EP3103116A1 - Improved frequency band extension in an audio signal decoder - Google Patents
Improved frequency band extension in an audio signal decoderInfo
- Publication number
- EP3103116A1 EP3103116A1 EP15705687.0A EP15705687A EP3103116A1 EP 3103116 A1 EP3103116 A1 EP 3103116A1 EP 15705687 A EP15705687 A EP 15705687A EP 3103116 A1 EP3103116 A1 EP 3103116A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- signal
- band
- decoded
- frequency
- frequency band
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005236 sound signal Effects 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000003044 adaptive effect Effects 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000002156 mixing Methods 0.000 claims abstract description 11
- 230000006872 improvement Effects 0.000 claims abstract description 6
- 239000000737 potassium alginate Substances 0.000 claims abstract description 5
- 239000000728 ammonium alginate Substances 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims abstract description 3
- 230000005284 excitation Effects 0.000 claims description 34
- 238000001228 spectrum Methods 0.000 claims description 25
- 238000001514 detection method Methods 0.000 claims description 11
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 230000015572 biosynthetic process Effects 0.000 description 30
- 238000003786 synthesis reaction Methods 0.000 description 30
- 238000001914 filtration Methods 0.000 description 29
- 230000006870 function Effects 0.000 description 14
- 238000012545 processing Methods 0.000 description 13
- 230000004044 response Effects 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 11
- 230000003595 spectral effect Effects 0.000 description 10
- 230000009466 transformation Effects 0.000 description 10
- 238000012952 Resampling Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 9
- 238000005070 sampling Methods 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000011282 treatment Methods 0.000 description 7
- 238000012805 post-processing Methods 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 239000004606 Fillers/Extenders Substances 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K3/00—Apparatus for stamping articles having integral means for supporting the articles to be stamped
- B41K3/54—Inking devices
- B41K3/56—Inking devices using inking pads
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/02—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with one or more flat stamping surfaces having fixed images
- B41K1/04—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with one or more flat stamping surfaces having fixed images with multiple stamping surfaces; with stamping surfaces replaceable as a whole
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/08—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with a flat stamping surface and changeable characters
- B41K1/10—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with a flat stamping surface and changeable characters having movable type-carrying bands or chains
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/08—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with a flat stamping surface and changeable characters
- B41K1/12—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor with a flat stamping surface and changeable characters having adjustable type-carrying wheels
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/36—Details
- B41K1/38—Inking devices; Stamping surfaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/36—Details
- B41K1/38—Inking devices; Stamping surfaces
- B41K1/40—Inking devices operated by stamping movement
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41K—STAMPS; STAMPING OR NUMBERING APPARATUS OR DEVICES
- B41K1/00—Portable hand-operated devices without means for supporting or locating the articles to be stamped, i.e. hand stamps; Inking devices or other accessories therefor
- B41K1/36—Details
- B41K1/38—Inking devices; Stamping surfaces
- B41K1/40—Inking devices operated by stamping movement
- B41K1/42—Inking devices operated by stamping movement with pads or rollers movable for inking
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/21—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being power information
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/26—Pre-filtering or post-filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
Definitions
- the present invention relates to the field of coding / decoding and audio-frequency signal processing (such as speech, music or other signals) for their transmission or storage.
- the invention relates to a method and a device for extending the frequency band in a decoder or a processor performing an audio-frequency signal improvement.
- the state of the art audio signal coding (mono) consists of perceptual encoding by transform or subband, with parametric high frequency band replication coding (SBR for Spectral).
- 3GPP AMR-WB Adaptive Multi-Rate Wideband
- codec and decoder which operates at an input / output frequency of 16 kHz and in which the signal is divided into two sub-bands, the low band (0-6.4 kHz) which is sampled at 12.8 kHz and coded by CELP model and the high band (6.4-7 kHz) which is parametrically reconstructed by "band extension" ( or BWE for "Bandwidth Extension” with or without additional information depending on the mode of the current frame.
- BWE Bandwidth Extension
- the limitation of the coded band of the AMR-WB codec at 7 kHz is essentially related to the fact that the transmission frequency response of the broadband terminals has been approximated at the time of standardization (ETSI / 3GPP then ITU-T T) according to the frequency mask defined in the ITU-T P.341 standard and more precisely by using a so-called "P341" filter defined in the ITU-T G.191 standard. which cuts frequencies above 7 kHz (this filter respects the mask defined in P.341).
- a signal sampled at 16 kHz may have a defined audio band of 0 to 8000 Hz; the AMR-WB codec thus introduces a limitation of the high band in comparison with the theoretical bandwidth of 8 kHz.
- the 3GPP AMR-WB speech codec was standardized in 2001 mainly for circuit-mode (CS) telephony applications on GSM (2G) and UMTS (3G). This same codec was also standardized in 2003 in ITU-T as Recommendation G.722.2 "Wideband coding speech at around 16kbit / s using Adaptive Multi-Rate Wideband (AMR-WB)".
- AMR-WB coding and decoding algorithm The details of the AMR-WB coding and decoding algorithm are not repeated here, a detailed description of this codec is found in the 3GPP specifications (TS 26.190, 26.191, 26.192, 26.193, 26.194, 26.204) and ITU-TG .722.2 (and the corresponding Appendices and Appendix) and in the article by B. Bessette et al. entitled "The adaptive multirate broadband speech coded (AMR-WB)", IEEE Transactions on Speech and Audio Processing, vol. 10, no. 8, 2002, pp. 620-636 and associated 3GPP and ITU-T standard source codes.
- AMR-WB adaptive multirate broadband speech coded
- the principle of band extension in the AMR-WB codec is rather rudimentary. Indeed, the high band (6.4-7 kHz) is generated by formatting a white noise through a temporal envelope (applied in the form of gains per subframe) and frequency (by the application of a linear prediction synthesis filter or LPC for "Linear Predictive Coding").
- This band extension technique is illustrated in Figure 1.
- a white noise, u HB1 (n), n 0, ⁇ ⁇ ⁇ , 79, is generated at 16 kHz per 5 ms subframe per linear congruent generator (block 100).
- This noise u HB1 (n) is shaped in time by applying gains per subframe; this operation is broken down into two processing steps (blocks 102, 106 or 109):
- a first factor is calculated (block 101) to set the white noise u HB1 (n) (block 101).
- the normalization of the energies is done by comparing blocks of different size (64 for u (n) and 80 for u HB1 (n)), without compensation of the differences of sampling frequencies (12.8 or 16 kHz) .
- u HB ⁇ n g HB u HB2 ⁇ n
- w sp is a weighting function that depends on Voice Activity Detection (VAD).
- VAD Voice Activity Detection
- the factor g HB in the decoding AMR-WB is bounded to take values in the interval [0.1, 1.0]. In fact, for the signals whose spectrum has more energy at high frequencies ⁇ e tiU close to -1, g sp close to 2), the gain g HB is usually underestimated.
- correction information is transmitted by the encoder AMR-WB and decoded (blocks 107, 108) in order to refine the estimated gain per subframe (4 bits every 5ms, ie 0.8 kbit / s) .
- a low-pass filter also FIR type (block 113) is added to the treatment to further attenuate frequencies above 7 kHz.
- the synthesis at high frequencies (HF) is finally added (block 130) to the low frequency synthesis (BF) obtained with the blocks 120 to 123 and resampled at 16 kHz (block 123).
- HF high frequencies
- BF low frequency synthesis
- the signal in the high band is white noise formatted (by temporal gains per subframe, filtering by 1 / A HB (z) and bandpass filtering), which is not a good general pattern of the signal in the 6.4-7 kHz band.
- white noise formatted by temporal gains per subframe, filtering by 1 / A HB (z) and bandpass filtering
- bandpass filtering which is not a good general pattern of the signal in the 6.4-7 kHz band.
- there are very harmonic music signals for which the 6.4-7 kHz band contains sinusoidal components (or tones) and no noise (or little noise) for these signals the band extension of the AMR-WB coding degrades. strongly the quality.
- the 7 kHz low-pass filter (block 113) introduces an offset of nearly 1 ms between the low and high bands, which can potentially degrade the quality of some signals by slightly desynchronizing the two bands at 23.85 kbit / s - this desynchronization can also be problematic when switching from 23.85 kbit / s to other modes.
- the estimate of gains per subframe is not optimal. In part, it is based on an equalization of the "absolute" energy per sub-frame (block 101) between signals at different frequencies: the artificial excitation at 16 kHz (white noise) and a signal at 12.8 kHz ( ACELP excitation decoded).
- the AMR-WB decoding algorithm has been improved in part with the development of the ITU-T G.718 scalable codec which was standardized in 2008.
- ITU-T G.718 includes an interoperable mode, for which core coding is compatible with 12.65 kbit / s G.722.2 (AMR-WB) coding; in addition, the G.718 decoder has the particularity of being able to decode a bit stream AMR-WB / G.722.2 at all possible bit rates of the AMR-WB codec (from 6.6 to 23.85 kbit / s).
- the G.718 interoperable decoder in low delay mode (G.718-LD) is illustrated in FIG. 2. Below are the improvements made to the bit-stream decoding functionality AMR- WB in the G.718 decoder, with references to Figure 1 when necessary:
- the band extension (described for example in clause 7.13.1 of Recommendation G.718, block 206) is identical to that of the AMR-WB decoder, except that the 6-7 kHz band-pass filter and the Synthesis 1 / A H B (Z) (blocks 111 and 112) are in reverse order.
- the 4 bits transmitted by AMR-WB encoder subframes are not used in the interoperable G.718 decoder; the synthesis of high frequencies (HF) at 23.85 kbit / s is therefore identical to 23.05 kbit / s which avoids the known problem of quality of AMR-WB decoding at 23.85 kbit / s.
- the low-pass filter at 7 kHz (block 113) is not used, and the specific decoding mode 23.85 kbit / s is omitted (blocks 107 to 109).
- a post-processing of the 16 kHz synthesis is implemented in G.718 by "noise gate” in block 208 (to “improve” the quality of silences by reducing the level) , high-pass filtering (block 209), low-frequency post filter (so-called “bass posfilter”) in block 210 attenuating inter-harmonic noise at low frequencies and conversion to 16-bit integers with saturation control (with control Gain or AGC) in block 211.
- the band extension in AMR-WB and / or G.718 codecs is still limited in several respects.
- the synthesis of high frequencies by shaped white noise is a very limited model of the signal in the frequency band above 6.4 kHz.
- the present invention improves the situation.
- the invention proposes a method of extending the frequency band of an audiofrequency signal during a decoding or improvement process comprising a step of obtaining the decoded signal in a first low band frequency band.
- the method is such that it comprises the following steps:
- band extension will be taken in the broad sense and will include not only the case of the extension of a subband at high frequencies but also the case of a replacement of subbands set to zero (of type "noise filling" in transform coding).
- both the taking into account of tonal components and a surround signal extracted from the signal resulting from the decoding of the low band makes it possible to perform the band extension with a signal model adapted to the true nature of the band. signal contrary to the use of artificial noise.
- the quality of the band extension is thus improved and in particular for certain types of signals such as music signals.
- the signal decoded in the low band has a part corresponding to the sound environment that can be transposed into high frequency so that a mix of harmonic components and the existing environment ensures a high band reconstructed consistent.
- the band extension is performed in the field of excitation and the decoded low band signal is a decoded low band excitation signal.
- the advantage of this embodiment is that a transformation without windowing (or equivalently with an implicit rectangular window of the length of the frame) is possible in the field of excitation. In this case no artifact (block effects) is audible.
- the extraction of the tonal components and the ambient signal is carried out according to the following steps:
- This embodiment allows accurate detection of tonal components.
- a power level control factor used for adaptive mixing is calculated based on the total energy of the decoded or decoded and extended low band signal and the tonal components.
- this control factor allows the combining step to adapt to the characteristics of the signal to optimize the relative proportion of the ambient signal in the mixture.
- the energy level is thus controlled to avoid audible artifacts.
- the decoded low band signal undergoes a step of subband decomposition by transform or filter bank, the extraction and combining steps then taking place in the frequency domain or in sub-bands. .
- the implementation of the band extension in the frequency domain makes it possible to obtain a fineness of frequency analysis which is not available with a temporal approach, and also makes it possible to have a frequency resolution sufficient to detect the tonal components. .
- this function includes a re-sampling of the signal by adding samples to the spectrum of this signal.
- Other ways of extending the signal are however possible, for example by translation in a sub-band processing.
- the present invention also relates to a frequency band extension device of an audiofrequency signal, the signal having been decoded in a first so-called low band frequency band.
- the device is such that it comprises:
- an extension module on at least a second frequency band greater than the first frequency band implemented on the decoded low band signal before the extraction module or on the combined signal after the combination module.
- This device has the same advantages as the method described above, which it implements.
- the invention relates to a decoder comprising a device as described.
- the invention relates to a storage medium, readable by a processor, integrated or not to the band expansion device, possibly removable, storing a computer program implementing a band extension method as described above.
- FIG. 1 illustrates a part of an AMR-WB decoder implementing frequency band extension steps of the state of the art and as previously described;
- FIG. 2 illustrates a decoder of the interoperable type G.718-LD at 16 kHz according to the state of the art and as described previously;
- FIG. 3 illustrates an interoperable decoder with the AMR-WB coding and integrating a band extension device according to one embodiment of the invention
- FIG. 4 illustrates in flowchart form the main steps of a band extension method according to one embodiment of the invention
- FIG. 5 illustrates an embodiment in the frequency domain of a band extension device according to the invention integrated in a decoder
- Figure 6 illustrates a hardware embodiment of a tape extender according to the invention
- FIG. 3 illustrates an exemplary decoder, compatible with the AMR-WB / G.722.2 standard, in which there is a postprocessing similar to that introduced in G.718 and described with reference to FIG. 2 and an improved band extension according to the extension method of the invention, implemented by the band extension device illustrated by block 309.
- the CELP decoding (BF for low frequencies) always operates at the internal frequency of 12.8 kHz, as in AMR-WB and G.718, and the band extension (HF for high frequencies) being the subject of the invention operates at the frequency of 16 kHz, the synthesis BF and HF are combined (block 312) at the frequency fs after adequate resampling (blocks 307 and 311).
- the combination of the low and high bands can be done at 16 kHz, after resampling the low band of 12.8 to 16 kHz, before resampling the combined signal at the frequency fs.
- the decoding according to FIG. 3 depends on the mode (or bit rate) AMR-WB associated with the current frame received.
- the decoding of the low band CELP part comprises the following steps:
- This excitation a) is used in the adaptive dictionary of the following subframe; it is then post-processed and one discerns as in G.718 the excitement a) (also noted exc) of its modified post-processed version u (n) (also noted exc2) which serves as input to the synthesis filter, 1 / ((z), in block 303.
- the post-treatments applied to the excitation can be modified (for example, the phase dispersion can be improved) or such post-treatments may be extended (for example, interharmonic noise reduction may be implemented) without affecting the nature of the band extension method of the invention.
- the post-treatments applied to the excitation can be modified (for example, the phase dispersion can be improved) or these post-treatments can be extended (for example, inter-harmonic noise reduction can be implemented), without affecting the nature of the band extension.
- the decoding of the low band described above assumes a current frame called "active" with a rate between 6.6 and 23.85 kbit / s.
- active a current frame
- some frames can be coded as "inactive” and in this case you can either transmit a silence descriptor (on 35 bits) or not transmit anything.
- SID frame of the AMR-WB encoder describes several parameters: ISF parameters averaged over 8 frames, average energy over 8 frames, "dithering flag" for the non-stationary noise reconstruction.
- This example decoder operates in the field of excitation and therefore comprises a step of decoding the low band excitation signal.
- the band extension device and the band extension method within the meaning of the invention also operates in a field different from the field of excitation and in particular with a low band decoded direct signal or a filter-weighted signal. perceptual.
- the decoder described makes it possible to extend the decoded low band (50-6400 Hz by taking into account the high-pass filtering at 50 Hz at the decoder, 0-6400 Hz in the general case ) to an extended band whose width varies, ranging from approximately 50-6900 Hz to 50-7700 Hz depending on the mode implemented in the current frame.
- the excitation for the high frequencies and generated in the frequency domain in a band of 5000 to 8000 Hz, to allow bandpass filtering of width 6000 to 6900 or 7700 Hz whose slope is not too stiff in the upper band rejected.
- the high band synthesis part is realized in the block 309 representing the band extension device according to the invention and which is detailed in FIG. 5 in one embodiment.
- a delay (block 310) is introduced to synchronize the outputs of the blocks 306 and 309 and the high band synthesized at 16 kHz is resampled from 16 kHz to the frequency fs (output of block 311).
- the extension method of the invention implemented in block 309 according to the first embodiment introduces preferentially no additional delay with respect to the low band reconstructed at 12.8 kHz; however, in variants of the invention (for example using a time / frequency transformation with overlap), a delay may be introduced.
- the low and high bands are then combined (added) in block 312 and the resulting synthesis is post-processed by high-order 50 Hz (type IIR) high-pass filtering whose coefficients depend on the frequency fs (block 313) and output post-processing with optional noise gate application similar to G.718 (block 314).
- high-order 50 Hz type IIR
- the band extension device according to the invention illustrated by the block 309 according to the embodiment of the decoder of FIG. 5, implements a band extension method (in the broad sense) now described with reference to FIG. figure 4.
- This extension device may also be independent of the decoder and may implement the method described in FIG. 4 to perform a band extension of an existing audio signal stored or transmitted to the device, with an analysis of the audio signal to extract it for example an excitation and an LPC filter.
- This device receives as input a decoded signal in a first so-called low-band frequency band u (ri) which may be in the field of excitation or that of the signal.
- a step of subband decomposition (E401b) by time frequency transform or filter bank is applied to the low band decoded signal to obtain the spectrum of the decoded low band signal U (k) for a implemented in the frequency domain.
- a step E401a for extending the decoded low band signal into a second frequency band greater than the first frequency band, to obtain an extended low band decoded signal U HB1 (k), can be performed on this decoded low band signal before or after the analysis step (subband decomposition).
- This extension step may comprise both a resampling step and an extension step or simply a translation step or frequency transposition as a function of the signal obtained at the input. It will be noted that in variants, step E401a may be performed at the end of the processing described in FIG. 4, that is to say on the combined signal, this processing then being mainly performed on the low band signal before expansion. , the result being equivalent.
- a step E402 for extracting a room signal (U HBA (k)) and tonal components (y (k)) is performed from the decoded (U (k)) or decoded and extended ( U HB1 (k)).
- Ambience is defined here as the residual signal that is obtained by suppressing in the existing signal the main (or dominant) harmonics (or tonal components).
- the high band (> 6 kHz) contains ambient information that is generally similar to that in the low band.
- the step of extracting the tonal components and the ambient signal comprises, for example, the following steps:
- This step can also be obtained by:
- obtaining the ambient signal by calculating an average of the decoded (or decoded and extended) low band signal
- the tonal components and the surround signal are then adaptively combined using energy level control factors in step E403 to obtain a so-called combined signal (U HB2 (k)).
- the extension step E401a can then be implemented if it has not already been performed on the decoded low band signal.
- the combination of these two types of signals makes it possible to obtain a combined signal with characteristics more adapted to certain types of signals, such as musical signals, and richer in frequency content and in the extended frequency band corresponding to the entire band of signals. frequency including the first and the second frequency band.
- the band extension according to the method improves the quality for this type of signals compared to the extension described in the AMR-WB standard.
- a synthesis step which corresponds to the analysis at 401b, is performed at E404b to bring the signal back into the time domain.
- an energy level adjustment step of the high band signal can be performed at E404a, before and / or after the synthesis step, by applying gain and / or adequate filtering. This step will be explained in more detail in the embodiment described in FIG. 5 for blocks 501 to 507.
- the band extension device 500 is described now with reference to FIG. 5 illustrating both this device and also processing modules suitable for implementation in a decoder of interoperable type with a coding AMR-WB.
- This device 500 implements the band extension method described above with reference to FIG. 4.
- the processing block 510 receives a decoded low band signal (u (n)).
- the band extension uses the decoded 12.8 kHz excitation (exc2 or u (n)) at the output of the block 302 of FIG. 3.
- This signal is broken down into frequency subbands by the subband decomposition module 510 (which implements step E401b of FIG. 4) which generally performs a transform or applies a filter bank, to obtain a sub-band decomposition U (k) of the signal u (n).
- a transformation without windowing (or equivalently with an implicit rectangular window of the length of the frame) is possible when the processing is performed in the field of excitation, and not the domain of the signal. In this case no artefact (block effects) is audible, which is an important advantage of this embodiment of the invention.
- the DCT-IV transformation is implemented by FFT according to the "Evolved ZXT (EDCT)" algorithm described in the article by D. M. Zhang, HT. Li, A Low Complexity Transform - Evolved DCT, IEEE 14th International Conference on Computational Science and Engineering (CSE), Aug. 2011, pp. 144-149, and implemented in ITU-T G.718 Annex B and G.729.1 Annex E.
- EDCT Evolved ZXT
- the DCT-IV transformation may be replaced by other short-term time-frequency transformations of the same length and in the field of excitation or in the domain of the signal. as an FFT (for "Fast Fourier Transform” in English) or a DCT-II (Discrete Cosine Transform - Type II).
- the DCT-IV can be replaced on the frame by a recovery-addition and windowing transformation of length greater than the length of the current frame, for example using an MDCT (for "Modified Discrete Cosine Tranform" in English).
- MDCT for "Modified Discrete Cosine Tranform" in English
- the subband decomposition is performed by the application of a real or complex filter bank, for example of the PQMF (Pseudo-QMF) type.
- a real or complex filter bank for example of the PQMF (Pseudo-QMF) type.
- PQMF Pulseudo-QMF
- the preferred embodiment in the invention can be applied by producing for example a transform of each subband and calculating the ambient signal in the range of absolute values, the tonal components always being obtained by difference between the signal (in absolute value) and the ambient signal.
- the complex module of the samples will replace the absolute value.
- the invention will be applied in a system using two subbands, the low band being analyzed by transform or filterbank.
- 0-6400 Hz (at 12.8 kHz) is then extended (block 511) into a spectrum of 320 samples covering the band 0-8000 Hz (at 16 kHz) in the following form:
- Block 511 implements step E401a of FIG. 4, that is to say the extension of the decoded low band signal.
- the original spectrum is preserved, in order to be able to apply a gradual attenuation response of the high-pass filter in this frequency band and also not to introduce audible defects during the addition step of the low frequency synthesis at high frequency synthesis.
- the generation of the extended over-sampled spectrum is carried out in a frequency band ranging from 5 to 8 kHz, thus including a second frequency band (6.4-8 kHz) greater than the first band of frequency (0-6.4 kHz).
- the extension of the decoded low band signal is performed at least on the second frequency band but also on a part of the first frequency band.
- This approach preserves the original spectrum in this band and avoids introducing distortions in the 5000-6000 Hz band during the addition of HF synthesis with BF synthesis - particularly the signal phase (implicitly represented in the DCT-IV domain) in this band is preserved.
- the 6000-8000 Hz band of U HB1 (k) is here defined by copying the 4000-6000 Hz band of U (k) since the value of start_band is preferably fixed at 160.
- the value of start_band can be made adaptive around the value of 160, without changing the nature of the invention.
- the details of the adaptation of the value start_band are not described here because they go beyond the scope of the invention without changing the scope.
- the high band (> 6 kHz) contains background information that is naturally similar to that in the low band.
- Ambience is defined here as the residual signal that is obtained by suppressing in the existing signal the main (or dominant) harmonics.
- the level of harmonicity in the 6000-8000 Hz band is generally correlated to that of the lower frequency bands.
- This decoded and extended low band signal is provided at the input of the extension device 500 and in particular at the input of the module 512.
- the block 512 for extracting tonal components and a room signal implements the step E402 of Figure 4 in the frequency domain.
- the extraction of the tonal components and the ambient signal is carried out according to the following operations: • Calculation of the total energy of the extended decoded low band signal ener HB : ener HB U HB1 (k) 2 + e
- L 80 and represents the length of the spectrum and the index i from 0 to L - 1 corresponds to the indices 240 + 31 of 240 to 319, ie the spectrum of 6 to 8 kHz.
- This variant has the defect of being more complex
- a non-uniform weighting may be applied to the averaged terms, or the median filtering may be replaced for example by other nonlinear filters of "stack filter” type.
- the residual signal is also calculated:
- This calculation therefore involves an implicit detection of the tonal components.
- the tonal parts are thus implicitly detected using the intermediate term y (i) representing an adaptive threshold.
- the detection condition being y (z)> 0.
- this ambient signal can be extracted from a low frequency signal or possibly another frequency band (or several frequency bands).
- the detection of peaks or tonal components can be done differently.
- This ambient signal could also be done on the decoded but not extended excitation, that is to say before the extension or spectral translation step, that is to say, for example on a portion of the low frequency signal rather than directly on the high frequency signal.
- the extraction of the tonal components and the ambient signal is performed in a different order and according to the following steps:
- This variant can for example be made in the following way: A peak (or tonal component) is detected at a line of index i in the amplitude spectrum * (i + 240 + 1)
- ⁇ 0, ..., L - 1.
- a sinusoidal model is applied in order to estimate the amplitude, frequency and possibly phase parameters of a tonal component associated with this peak.
- the details of this estimate are not presented here, but the estimate of the frequency can typically use a parabolic interpolation on 3 points to locate the maximum of the parabola approximating the 3 points of amplitude
- DCT-IV transform domain used here
- the absolute value of the spectral values will be replaced for example by the square of the spectral values, without changing the principle of the invention; in this case a square root will be needed to return to the signal domain, which is more complex to achieve.
- the combination module 513 performs a step of combining by adaptive mixing of the ambient signal and the tonal components.
- the factor ⁇ is> 1.
- the tonal components, detected line by line by the condition y (i)> 0, are reduced by the factor ⁇ ; the average level is amplified by the factor / ⁇ .
- a power level control factor is calculated based on the total energy of the decoded (or decoded and extended) low band signal and the tonal components.
- the energy adjustment is performed as follows:
- the adjustment factor is defined by the following equation:
- ⁇ avoids over-estimation of energy.
- we calculate /? so as to maintain the same ambient signal level with respect to the energy of the tonal components in the consecutive bands of the signal.
- the energy of the tonal components is calculated in three bands: 2000-4000 Hz, 4000-6000 Hz and 6000-8000 Hz, with
- V 4 ⁇ u a (k)
- N (£ 1 , £ 2 ) is the set of indices k for which the index coefficient k is classified as being associated with the tonal components. This set can be obtained for example by detecting the local peaks in U > lev (k) or lev ⁇ k) is calculated as the average level of the spectrum line by line.
- ⁇ is fixed so that the ratio between the energy of the tonal components in the 4-6 kHz and 6-8 kHz bands is the same as between the 2-4 kHz and 4-6 kHz bands: o _ PE N6 _ &
- the calculation of ⁇ may be replaced by other methods.
- the linear regression could for example be estimated in a supervised manner by estimating the factor ⁇ by giving the original high band in a base d 'learning. It will be noted that the method of calculating ⁇ does not limit the nature of the invention.
- the parameter ⁇ can be used to calculate ⁇ , taking into account that a signal with a surround signal added in a given band is generally perceived as stronger than a harmonic signal at the same energy in the same direction. bandaged. If we define a as the quantity of ambient signal added to the harmonic signal:
- the block 501 At the output of the band extension device 500, the block 501, in a particular embodiment, optionally carries out a dual operation of application of bandpass filter frequency response and deemphasis filtering (or deemphasis filtering). ) in the frequency domain.
- the deemphasis filtering may be performed in the time domain, after the block 502 or even before the block 510; however, in this case, the bandpass filtering performed in the block 501 may leave some low frequency components of very low levels which are amplified by de-emphasis, which may slightly discern the decoded low band. For this reason, it is preferred here to perform the deemphasis in the frequency domain.
- G deemph (k) is the frequency response of the filter l / (l - 0.68z _1 ) over a restricted discrete frequency band.
- G deem h (k) as:
- the definition of 0 k can be adjusted (for example for even frequencies).
- the HF synthesis is not de-emphasized.
- the high-frequency signal is on the contrary de-emphasized so as to bring it back to a domain coherent with the low-frequency signal (0-6.4 kHz) which leaves block 305 of FIG. 3. This is important for the estimation and subsequent adjustment of the energy of HF synthesis.
- the de-emphasis can be performed in an equivalent manner in the time domain after inverse DCT.
- band-pass filtering is applied with two separate parts: one fixed high-pass, the other adaptive low-pass (flow-rate function).
- This filtering is performed in the frequency domain.
- the partial low-pass filter response in the frequency domain is calculated as follows:
- N lp 60 to 6.6 kbit / s, 40 to 8.85 kbit / s, 20 at rates> 8.85 bit / s.
- G h (k), k 0, ⁇ ⁇ ⁇ , 55, is given for example in Table 1 below.
- G hp (k) may be modified while keeping a gradual attenuation.
- the low-pass filtering with variable bandwidth, G lp ⁇ k may be adjusted with different values or frequency support, without changing the principle of this filtering step.
- bandpass filtering can be adapted by defining a single filtering step combining the high-pass and low-pass filtering.
- the bandpass filtering may be performed equivalently in the time domain (as in block 112 of FIG. 1) with different filter coefficients according to the bit rate, after an inverse DCT step.
- it is advantageous to carry out this step directly in the frequency domain because the filtering is carried out in the field of LPC excitation and therefore the problems of circular convolution and edge effects are very limited in this field. .
- the inverse transform block 502 performs an inverse DCT on 320 samples to find the high frequency signal sampled at 16 kHz. Its implementation is identical to block 510 because the DCT-IV is orthonormed, except that the length of the transform is 320 instead of 256, and we obtain:
- the block 502 performs the synthesis corresponding to the analysis carried out in the block 510.
- the signal sampled at 16 kHz is then optionally scaled by gains defined by subframe of 80 samples (block 504).
- one calculates first (block 503) a gain g H Bi (m) per subframe by energy ratios of the subframes such that in each subframe of index 77 0, 1, 2 or 3 of the current frame:
- Block 504 scales the combined signal (included in step E404a of FIG. 4) according to the following equation:
- the realization of the block 503 differs from that of the block 101 of Figure 1, because the energy at the current frame is taken into account in addition to that of the subframe. This makes it possible to have the ratio of the energy of each subframe with respect to the energy of the frame. Energy ratios (or relative energies) are compared rather than the absolute energies between low band and high band.
- this scaling step makes it possible to keep in the high band the energy ratio between the subframe and the frame in the same way as in the low band.
- block 506 then scales the signal (included in step E404a of FIG. 4) according to the following equation:
- the gain g HB2 (m) is obtained from the block 505 by executing the blocks 103, 104 and 105 of the AMR-WB coding (the input of the block 103 being the decoded excitation in the low band, u (n)) .
- Blocks 505 and 506 are useful for adjusting the level of the LPC synthesis filter (block 507), here depending on the tilt of the signal. Other methods of calculating the gain g HB2 (m) are possible without changing the nature of the invention.
- this filtering can be done in the same way as described for the block 111 of FIG. 1 of the AMR-WB decoder, however the order of the filter goes to 20 at the rate of 6.6, which does not change. not significantly the quality of the synthesized signal.
- the coding of the low band (0-6.4 kHz) may be replaced by a CELP coder other than that used in AMR-WB, for example the CELP coder in G.718 to 8. kbit / s.
- a CELP coder other than that used in AMR-WB, for example the CELP coder in G.718 to 8. kbit / s.
- other encoders in wide band or operating at frequencies higher than 16 kHz in which the coding of the low band operates at an internal frequency at 12.8 kHz could be used.
- the invention can be obviously adapted to other sampling frequencies than 12.8 kHz, when a low frequency encoder operates at a sampling frequency lower than that of the original or reconstructed signal.
- the low band decoding does not use a linear prediction, it does not have an excitation signal to be extended, in this case it will be possible to carry out an LPC analysis of the reconstructed signal in the current frame and calculate an LPC excitation. so as to be able to apply the invention.
- the excitation or the low band signal (u (n)) is resampled, for example by linear interpolation or cubic "spline", of 12.8 to 16 kHz before transformation (for example DCT-IV) of length 320.
- This variant has the defect of being more complex, because the transform (DCT-IV) of the excitation or the signal is then calculated on a larger length and resampling is not performed in the transform domain.
- FIG. 6 represents an exemplary hardware embodiment of a band extension device 600 according to the invention. This may be an integral part of an audio-frequency signal decoder or equipment receiving decoded or non-decoded audio signals.
- This type of device comprises a PROC processor cooperating with a memory block BM having a memory storage and / or work MEM.
- Such a device comprises an input module E adapted to receive a decoded audio signal or extracted in a first frequency band said low band brought into the frequency domain (U (k)). It comprises an output module S adapted to transmit the extension signal in a second frequency band (U HB2 (k)) for example to a filtering module 501 of FIG. 5.
- the memory block may advantageously comprise a computer program comprising code instructions for implementing the steps of the band extension method in the sense of the invention, when these instructions are executed by the processor PROC, and in particular the steps for extracting (E402) tonal components and a surround signal from a signal derived from the decoded low band signal (U (k)), combining (E403) the tonal components (y (k)) and the ambient signal (U HBA k)) by adaptive mixing using energy level control factors to obtain an audio signal, said combined signal (U HB2 (k)), of extension (E401a) on at minus a second frequency band higher than the first frequency band of the low band decoded signal before the extraction step or the combined signal after the combining step.
- a computer program comprising code instructions for implementing the steps of the band extension method in the sense of the invention, when these instructions are executed by the processor PROC, and in particular the steps for extracting (E402) tonal components and a surround signal from a signal derived from the decoded low
- FIG. 4 repeats the steps of an algorithm of such a computer program.
- the computer program can also be stored on a memory medium readable by a reader of the device or downloadable in the memory space thereof.
- the memory MEM generally records all the data necessary for the implementation of the method.
- the device thus described may also include the low band decoding functions and other processing functions described for example in FIGS. 5 and 3 in addition to the band extension functions according to the invention.
Abstract
Description
Claims
Priority Applications (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
RS20210945A RS62160B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio signal decoder |
PL15705687T PL3103116T3 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio signal decoder |
EP17206569.0A EP3327722B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206567.4A EP3330967B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206563.3A EP3330966B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
SI201531646T SI3103116T1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio signal decoder |
DK17206563.3T DK3330966T3 (en) | 2014-02-07 | 2015-02-04 | IMPROVED FREQUENCY BAND SPREADING IN A DECODER FOR AUDIO FREQUENCY SIGNALS |
HRP20211187TT HRP20211187T1 (en) | 2014-02-07 | 2021-07-23 | Improved frequency band extension in an audio signal decoder |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1450969A FR3017484A1 (en) | 2014-02-07 | 2014-02-07 | ENHANCED FREQUENCY BAND EXTENSION IN AUDIO FREQUENCY SIGNAL DECODER |
PCT/FR2015/050257 WO2015118260A1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio signal decoder |
Related Child Applications (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17206563.3A Division EP3330966B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206563.3A Division-Into EP3330966B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206569.0A Division-Into EP3327722B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206569.0A Division EP3327722B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206567.4A Division EP3330967B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206567.4A Division-Into EP3330967B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3103116A1 true EP3103116A1 (en) | 2016-12-14 |
EP3103116B1 EP3103116B1 (en) | 2021-05-05 |
Family
ID=51014390
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17206563.3A Active EP3330966B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206567.4A Active EP3330967B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP15705687.0A Active EP3103116B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio signal decoder |
EP17206569.0A Active EP3327722B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17206563.3A Active EP3330966B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
EP17206567.4A Active EP3330967B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17206569.0A Active EP3327722B1 (en) | 2014-02-07 | 2015-02-04 | Improved frequency band extension in an audio frequency signal decoder |
Country Status (21)
Country | Link |
---|---|
US (5) | US10043525B2 (en) |
EP (4) | EP3330966B1 (en) |
JP (4) | JP6625544B2 (en) |
KR (5) | KR102426029B1 (en) |
CN (4) | CN108022599B (en) |
BR (2) | BR112016017616B1 (en) |
DK (2) | DK3330966T3 (en) |
ES (2) | ES2878401T3 (en) |
FI (1) | FI3330966T3 (en) |
FR (1) | FR3017484A1 (en) |
HR (2) | HRP20231164T1 (en) |
HU (2) | HUE055111T2 (en) |
LT (2) | LT3103116T (en) |
MX (1) | MX363675B (en) |
PL (2) | PL3103116T3 (en) |
PT (2) | PT3103116T (en) |
RS (2) | RS62160B1 (en) |
RU (4) | RU2682923C2 (en) |
SI (2) | SI3103116T1 (en) |
WO (1) | WO2015118260A1 (en) |
ZA (3) | ZA201606173B (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
MX347316B (en) * | 2013-01-29 | 2017-04-21 | Fraunhofer Ges Forschung | Apparatus and method for synthesizing an audio signal, decoder, encoder, system and computer program. |
FR3017484A1 (en) | 2014-02-07 | 2015-08-14 | Orange | ENHANCED FREQUENCY BAND EXTENSION IN AUDIO FREQUENCY SIGNAL DECODER |
EP2980794A1 (en) * | 2014-07-28 | 2016-02-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoder and decoder using a frequency domain processor and a time domain processor |
EP3382702A1 (en) * | 2017-03-31 | 2018-10-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for determining a predetermined characteristic related to an artificial bandwidth limitation processing of an audio signal |
CN109688531B (en) * | 2017-10-18 | 2021-01-26 | 宏达国际电子股份有限公司 | Method for acquiring high-sound-quality audio conversion information, electronic device and recording medium |
EP3518562A1 (en) | 2018-01-29 | 2019-07-31 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio signal processor, system and methods distributing an ambient signal to a plurality of ambient signal channels |
KR102308077B1 (en) * | 2019-09-19 | 2021-10-01 | 에스케이텔레콤 주식회사 | Method and Apparatus for Artificial Band Conversion Based on Learning Model |
WO2021143691A1 (en) * | 2020-01-13 | 2021-07-22 | 华为技术有限公司 | Audio encoding and decoding methods and audio encoding and decoding devices |
Family Cites Families (52)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU3352997A (en) | 1996-07-03 | 1998-02-02 | British Telecommunications Public Limited Company | Voice activity detector |
SE9700772D0 (en) * | 1997-03-03 | 1997-03-03 | Ericsson Telefon Ab L M | A high resolution post processing method for a speech decoder |
TW430778B (en) * | 1998-06-15 | 2001-04-21 | Yamaha Corp | Voice converter with extraction and modification of attribute data |
JP4135240B2 (en) * | 1998-12-14 | 2008-08-20 | ソニー株式会社 | Receiving apparatus and method, communication apparatus and method |
US6226616B1 (en) * | 1999-06-21 | 2001-05-01 | Digital Theater Systems, Inc. | Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility |
JP4792613B2 (en) * | 1999-09-29 | 2011-10-12 | ソニー株式会社 | Information processing apparatus and method, and recording medium |
US6704711B2 (en) * | 2000-01-28 | 2004-03-09 | Telefonaktiebolaget Lm Ericsson (Publ) | System and method for modifying speech signals |
DE10041512B4 (en) * | 2000-08-24 | 2005-05-04 | Infineon Technologies Ag | Method and device for artificially expanding the bandwidth of speech signals |
US7400651B2 (en) * | 2001-06-29 | 2008-07-15 | Kabushiki Kaisha Kenwood | Device and method for interpolating frequency components of signal |
KR100935961B1 (en) * | 2001-11-14 | 2010-01-08 | 파나소닉 주식회사 | Encoding device and decoding device |
EP1451812B1 (en) * | 2001-11-23 | 2006-06-21 | Koninklijke Philips Electronics N.V. | Audio signal bandwidth extension |
US20030187663A1 (en) * | 2002-03-28 | 2003-10-02 | Truman Michael Mead | Broadband frequency translation for high frequency regeneration |
JP2005531456A (en) * | 2002-06-28 | 2005-10-20 | ピレリ・プネウマティチ・ソチエタ・ペル・アツィオーニ | Tire characteristic parameter monitoring system |
US6845360B2 (en) * | 2002-11-22 | 2005-01-18 | Arbitron Inc. | Encoding multiple messages in audio data and detecting same |
NZ562190A (en) * | 2005-04-01 | 2010-06-25 | Qualcomm Inc | Systems, methods, and apparatus for highband burst suppression |
DE602006019723D1 (en) * | 2005-06-08 | 2011-03-03 | Panasonic Corp | DEVICE AND METHOD FOR SPREADING AN AUDIO SIGNAL BAND |
FR2888699A1 (en) * | 2005-07-13 | 2007-01-19 | France Telecom | HIERACHIC ENCODING / DECODING DEVICE |
US7546237B2 (en) * | 2005-12-23 | 2009-06-09 | Qnx Software Systems (Wavemakers), Inc. | Bandwidth extension of narrowband speech |
CN101089951B (en) * | 2006-06-16 | 2011-08-31 | 北京天籁传音数字技术有限公司 | Band spreading coding method and device and decode method and device |
JP5141180B2 (en) * | 2006-11-09 | 2013-02-13 | ソニー株式会社 | Frequency band expanding apparatus, frequency band expanding method, reproducing apparatus and reproducing method, program, and recording medium |
KR101379263B1 (en) * | 2007-01-12 | 2014-03-28 | 삼성전자주식회사 | Method and apparatus for decoding bandwidth extension |
US8229106B2 (en) * | 2007-01-22 | 2012-07-24 | D.S.P. Group, Ltd. | Apparatus and methods for enhancement of speech |
US8489396B2 (en) * | 2007-07-25 | 2013-07-16 | Qnx Software Systems Limited | Noise reduction with integrated tonal noise reduction |
US8041577B2 (en) * | 2007-08-13 | 2011-10-18 | Mitsubishi Electric Research Laboratories, Inc. | Method for expanding audio signal bandwidth |
CN101790757B (en) * | 2007-08-27 | 2012-05-30 | 爱立信电话股份有限公司 | Improved transform coding of speech and audio signals |
WO2009039897A1 (en) * | 2007-09-26 | 2009-04-02 | Fraunhofer - Gesellschaft Zur Förderung Der Angewandten Forschung E.V. | Apparatus and method for extracting an ambient signal in an apparatus and method for obtaining weighting coefficients for extracting an ambient signal and computer program |
US8688441B2 (en) * | 2007-11-29 | 2014-04-01 | Motorola Mobility Llc | Method and apparatus to facilitate provision and use of an energy value to determine a spectral envelope shape for out-of-signal bandwidth content |
RU2439720C1 (en) * | 2007-12-18 | 2012-01-10 | ЭлДжи ЭЛЕКТРОНИКС ИНК. | Method and device for sound signal processing |
EP2077551B1 (en) * | 2008-01-04 | 2011-03-02 | Dolby Sweden AB | Audio encoder and decoder |
US8483854B2 (en) * | 2008-01-28 | 2013-07-09 | Qualcomm Incorporated | Systems, methods, and apparatus for context processing using multiple microphones |
DE102008015702B4 (en) * | 2008-01-31 | 2010-03-11 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for bandwidth expansion of an audio signal |
US8831936B2 (en) * | 2008-05-29 | 2014-09-09 | Qualcomm Incorporated | Systems, methods, apparatus, and computer program products for speech signal processing using spectral contrast enhancement |
KR101381513B1 (en) * | 2008-07-14 | 2014-04-07 | 광운대학교 산학협력단 | Apparatus for encoding and decoding of integrated voice and music |
US8532983B2 (en) * | 2008-09-06 | 2013-09-10 | Huawei Technologies Co., Ltd. | Adaptive frequency prediction for encoding or decoding an audio signal |
US8352279B2 (en) * | 2008-09-06 | 2013-01-08 | Huawei Technologies Co., Ltd. | Efficient temporal envelope coding approach by prediction between low band signal and high band signal |
KR101369267B1 (en) * | 2008-12-15 | 2014-03-04 | 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. | Audio encoder and bandwidth extension decoder |
US8463599B2 (en) * | 2009-02-04 | 2013-06-11 | Motorola Mobility Llc | Bandwidth extension method and apparatus for a modified discrete cosine transform audio coder |
RU2452044C1 (en) * | 2009-04-02 | 2012-05-27 | Фраунхофер-Гезелльшафт цур Фёрдерунг дер ангевандтен Форшунг Е.Ф. | Apparatus, method and media with programme code for generating representation of bandwidth-extended signal on basis of input signal representation using combination of harmonic bandwidth-extension and non-harmonic bandwidth-extension |
CN101990253A (en) * | 2009-07-31 | 2011-03-23 | 数维科技(北京)有限公司 | Bandwidth expanding method and device |
JP5493655B2 (en) * | 2009-09-29 | 2014-05-14 | 沖電気工業株式会社 | Voice band extending apparatus and voice band extending program |
RU2568278C2 (en) * | 2009-11-19 | 2015-11-20 | Телефонактиеболагет Лм Эрикссон (Пабл) | Bandwidth extension for low-band audio signal |
JP5589631B2 (en) * | 2010-07-15 | 2014-09-17 | 富士通株式会社 | Voice processing apparatus, voice processing method, and telephone apparatus |
US9047875B2 (en) * | 2010-07-19 | 2015-06-02 | Futurewei Technologies, Inc. | Spectrum flatness control for bandwidth extension |
KR101826331B1 (en) * | 2010-09-15 | 2018-03-22 | 삼성전자주식회사 | Apparatus and method for encoding and decoding for high frequency bandwidth extension |
SG192718A1 (en) * | 2011-02-14 | 2013-09-30 | Fraunhofer Ges Forschung | Audio codec using noise synthesis during inactive phases |
US20140019125A1 (en) * | 2011-03-31 | 2014-01-16 | Nokia Corporation | Low band bandwidth extended |
PL2791937T3 (en) * | 2011-11-02 | 2016-11-30 | Generation of a high band extension of a bandwidth extended audio signal | |
TWI591620B (en) | 2012-03-21 | 2017-07-11 | 三星電子股份有限公司 | Method of generating high frequency noise |
US9228916B2 (en) * | 2012-04-13 | 2016-01-05 | The Regents Of The University Of California | Self calibrating micro-fabricated load cells |
KR101897455B1 (en) * | 2012-04-16 | 2018-10-04 | 삼성전자주식회사 | Apparatus and method for enhancement of sound quality |
US9666202B2 (en) * | 2013-09-10 | 2017-05-30 | Huawei Technologies Co., Ltd. | Adaptive bandwidth extension and apparatus for the same |
FR3017484A1 (en) * | 2014-02-07 | 2015-08-14 | Orange | ENHANCED FREQUENCY BAND EXTENSION IN AUDIO FREQUENCY SIGNAL DECODER |
-
2014
- 2014-02-07 FR FR1450969A patent/FR3017484A1/en active Pending
-
2015
- 2015-02-04 RU RU2016136008A patent/RU2682923C2/en active
- 2015-02-04 CN CN201711459701.1A patent/CN108022599B/en active Active
- 2015-02-04 RS RS20210945A patent/RS62160B1/en unknown
- 2015-02-04 CN CN201711459702.6A patent/CN107993667B/en active Active
- 2015-02-04 DK DK17206563.3T patent/DK3330966T3/en active
- 2015-02-04 PL PL15705687T patent/PL3103116T3/en unknown
- 2015-02-04 KR KR1020177037710A patent/KR102426029B1/en active IP Right Grant
- 2015-02-04 PT PT157056870T patent/PT3103116T/en unknown
- 2015-02-04 FI FIEP17206563.3T patent/FI3330966T3/en active
- 2015-02-04 KR KR1020177037700A patent/KR20180002906A/en not_active IP Right Cessation
- 2015-02-04 ES ES15705687T patent/ES2878401T3/en active Active
- 2015-02-04 DK DK15705687.0T patent/DK3103116T3/en active
- 2015-02-04 LT LTEP15705687.0T patent/LT3103116T/en unknown
- 2015-02-04 SI SI201531646T patent/SI3103116T1/en unknown
- 2015-02-04 EP EP17206563.3A patent/EP3330966B1/en active Active
- 2015-02-04 EP EP17206567.4A patent/EP3330967B1/en active Active
- 2015-02-04 EP EP15705687.0A patent/EP3103116B1/en active Active
- 2015-02-04 CN CN201580007250.0A patent/CN105960675B/en active Active
- 2015-02-04 CN CN201711459695.XA patent/CN108109632B/en active Active
- 2015-02-04 KR KR1020227007471A patent/KR102510685B1/en active IP Right Grant
- 2015-02-04 PL PL17206563.3T patent/PL3330966T3/en unknown
- 2015-02-04 RU RU2017144522A patent/RU2763481C2/en active
- 2015-02-04 PT PT172065633T patent/PT3330966T/en unknown
- 2015-02-04 KR KR1020167024350A patent/KR102380205B1/en active IP Right Grant
- 2015-02-04 RU RU2017144523A patent/RU2763547C2/en active
- 2015-02-04 BR BR112016017616-2A patent/BR112016017616B1/en active IP Right Grant
- 2015-02-04 BR BR122017027991-2A patent/BR122017027991B1/en active IP Right Grant
- 2015-02-04 EP EP17206569.0A patent/EP3327722B1/en active Active
- 2015-02-04 HU HUE15705687A patent/HUE055111T2/en unknown
- 2015-02-04 JP JP2016549732A patent/JP6625544B2/en active Active
- 2015-02-04 SI SI201531958T patent/SI3330966T1/en unknown
- 2015-02-04 RS RS20230844A patent/RS64614B1/en unknown
- 2015-02-04 MX MX2016010214A patent/MX363675B/en unknown
- 2015-02-04 US US15/117,100 patent/US10043525B2/en active Active
- 2015-02-04 KR KR1020177037706A patent/KR102380487B1/en active IP Right Grant
- 2015-02-04 WO PCT/FR2015/050257 patent/WO2015118260A1/en active Application Filing
- 2015-02-04 HR HRP20231164TT patent/HRP20231164T1/en unknown
- 2015-02-04 ES ES17206563T patent/ES2955964T3/en active Active
- 2015-02-04 RU RU2017144521A patent/RU2763848C2/en active
- 2015-02-04 LT LTEP17206563.3T patent/LT3330966T/en unknown
- 2015-02-04 HU HUE17206563A patent/HUE062979T2/en unknown
-
2016
- 2016-09-06 ZA ZA2016/06173A patent/ZA201606173B/en unknown
-
2017
- 2017-12-11 ZA ZA2017/08366A patent/ZA201708366B/en unknown
- 2017-12-11 ZA ZA2017/08368A patent/ZA201708368B/en unknown
-
2018
- 2018-01-12 US US15/869,560 patent/US10668760B2/en active Active
- 2018-06-18 US US16/011,153 patent/US10730329B2/en active Active
-
2019
- 2019-06-07 JP JP2019107008A patent/JP6775064B2/en active Active
- 2019-06-07 JP JP2019107007A patent/JP6775063B2/en active Active
- 2019-06-07 JP JP2019107009A patent/JP6775065B2/en active Active
-
2020
- 2020-07-13 US US16/926,818 patent/US11312164B2/en active Active
- 2020-07-27 US US16/939,104 patent/US11325407B2/en active Active
-
2021
- 2021-07-23 HR HRP20211187TT patent/HRP20211187T1/en unknown
Non-Patent Citations (2)
Title |
---|
None * |
See also references of WO2015118260A1 * |
Also Published As
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3020043B1 (en) | Optimized scale factor for frequency band extension in an audiofrequency signal decoder | |
EP3014611B1 (en) | Improved frequency band extension in an audio signal decoder | |
EP3330966B1 (en) | Improved frequency band extension in an audio frequency signal decoder | |
WO2014009657A1 (en) | Processing method for improving the quality of audio frequency signals |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20160804 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KANIEWSKA, MAGDALENA Inventor name: RAGOT, STEPHANE |
|
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20170720 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: KONINKLIJKE PHILIPS N.V. |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: KONINKLIJKE PHILIPS N.V. |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20201123 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1390825 Country of ref document: AT Kind code of ref document: T Effective date: 20210515 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602015068913 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: PT Ref legal event code: SC4A Ref document number: 3103116 Country of ref document: PT Date of ref document: 20210712 Kind code of ref document: T Free format text: AVAILABILITY OF NATIONAL TRANSLATION Effective date: 20210705 |
|
REG | Reference to a national code |
Ref country code: RO Ref legal event code: EPE |
|
REG | Reference to a national code |
Ref country code: HR Ref legal event code: TUEP Ref document number: P20211187T Country of ref document: HR |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20210721 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP Ref country code: FI Ref legal event code: FGE |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
REG | Reference to a national code |
Ref country code: GR Ref legal event code: EP Ref document number: 20210401962 Country of ref document: GR Effective date: 20210813 |
|
REG | Reference to a national code |
Ref country code: EE Ref legal event code: FG4A Ref document number: E021177 Country of ref document: EE Effective date: 20210719 |
|
REG | Reference to a national code |
Ref country code: NO Ref legal event code: T2 Effective date: 20210505 |
|
REG | Reference to a national code |
Ref country code: SK Ref legal event code: T3 Ref document number: E 37832 Country of ref document: SK |
|
REG | Reference to a national code |
Ref country code: HU Ref legal event code: AG4A Ref document number: E055111 Country of ref document: HU |
|
REG | Reference to a national code |
Ref country code: HR Ref legal event code: T1PR Ref document number: P20211187 Country of ref document: HR |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2878401 Country of ref document: ES Kind code of ref document: T3 Effective date: 20211118 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602015068913 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: HR Ref legal event code: ODRP Ref document number: P20211187 Country of ref document: HR Payment date: 20220127 Year of fee payment: 8 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20220208 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210505 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: MK Payment date: 20220125 Year of fee payment: 8 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: UEP Ref document number: 1390825 Country of ref document: AT Kind code of ref document: T Effective date: 20210505 |
|
REG | Reference to a national code |
Ref country code: HR Ref legal event code: ODRP Ref document number: P20211187 Country of ref document: HR Payment date: 20230124 Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20230222 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SM Payment date: 20230221 Year of fee payment: 9 Ref country code: RO Payment date: 20230206 Year of fee payment: 9 Ref country code: NO Payment date: 20230216 Year of fee payment: 9 Ref country code: MC Payment date: 20230216 Year of fee payment: 9 Ref country code: LU Payment date: 20230222 Year of fee payment: 9 Ref country code: LT Payment date: 20230124 Year of fee payment: 9 Ref country code: IE Payment date: 20230215 Year of fee payment: 9 Ref country code: FR Payment date: 20230223 Year of fee payment: 9 Ref country code: FI Payment date: 20230223 Year of fee payment: 9 Ref country code: ES Payment date: 20230321 Year of fee payment: 9 Ref country code: DK Payment date: 20230223 Year of fee payment: 9 Ref country code: CZ Payment date: 20230203 Year of fee payment: 9 Ref country code: CH Payment date: 20230307 Year of fee payment: 9 Ref country code: BG Payment date: 20230224 Year of fee payment: 9 Ref country code: AT Payment date: 20230215 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: TR Payment date: 20230124 Year of fee payment: 9 Ref country code: SK Payment date: 20230124 Year of fee payment: 9 Ref country code: SI Payment date: 20230124 Year of fee payment: 9 Ref country code: SE Payment date: 20230222 Year of fee payment: 9 Ref country code: RS Payment date: 20230124 Year of fee payment: 9 Ref country code: PT Payment date: 20230123 Year of fee payment: 9 Ref country code: PL Payment date: 20230125 Year of fee payment: 9 Ref country code: LV Payment date: 20230223 Year of fee payment: 9 Ref country code: IT Payment date: 20230220 Year of fee payment: 9 Ref country code: IS Payment date: 20230222 Year of fee payment: 9 Ref country code: HU Payment date: 20230127 Year of fee payment: 9 Ref country code: HR Payment date: 20230124 Year of fee payment: 9 Ref country code: GR Payment date: 20230220 Year of fee payment: 9 Ref country code: GB Payment date: 20230214 Year of fee payment: 9 Ref country code: EE Payment date: 20230222 Year of fee payment: 9 Ref country code: DE Payment date: 20220628 Year of fee payment: 9 Ref country code: BE Payment date: 20230222 Year of fee payment: 9 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230523 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: MT Payment date: 20230227 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: AL Payment date: 20230227 Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: HR Ref legal event code: ODRP Ref document number: P20211187 Country of ref document: HR Payment date: 20240123 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: MK Payment date: 20230126 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: LU Payment date: 20240226 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GR Payment date: 20240221 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IS Payment date: 20240229 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: LT Payment date: 20240122 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20240226 Year of fee payment: 10 Ref country code: ES Payment date: 20240307 Year of fee payment: 10 Ref country code: IE Payment date: 20240220 Year of fee payment: 10 |