EP3403260A1 - Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte - Google Patents

Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte

Info

Publication number
EP3403260A1
EP3403260A1 EP17711600.1A EP17711600A EP3403260A1 EP 3403260 A1 EP3403260 A1 EP 3403260A1 EP 17711600 A EP17711600 A EP 17711600A EP 3403260 A1 EP3403260 A1 EP 3403260A1
Authority
EP
European Patent Office
Prior art keywords
frequencies
audio
frequency
audio signal
selection criterion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17711600.1A
Other languages
German (de)
English (en)
Other versions
EP3403260B1 (fr
Inventor
Denis Perechnev
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ask Industries GmbH
Original Assignee
Ask Industries GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ask Industries GmbH filed Critical Ask Industries GmbH
Publication of EP3403260A1 publication Critical patent/EP3403260A1/fr
Application granted granted Critical
Publication of EP3403260B1 publication Critical patent/EP3403260B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G10L19/0208—Subband vocoders
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/0017—Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/012—Comfort noise or silence coding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
    • G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility

Definitions

  • the invention relates to a method for processing a lossy compressed audio signal.
  • the purpose of data compression is to reduce the data size of corresponding audio signals.
  • the data compression can always be lossy or not lossy.
  • the lossy data compression is to be considered in the following, which can be realized, for example, by data-related rejection of frequency components lying at the edge of the human audible range. The subjective perception of hearing by a listener should hardly be affected in this way.
  • lossy compressed audio signals Due to the comparatively reduced sound quality of lossy compressed audio signals, it is sometimes desirable to process lossy compressed audio signals; H. correspondingly discarded frequency components at least partially restore or replace them with comparable frequency components.
  • the invention is thus based on the object of specifying an improved method for processing a lossy compressed audio signal.
  • the object is achieved by a method according to claim 1.
  • the dependent claims relate to advantageous embodiments of the method.
  • the object is further achieved by the device according to claim 14 and by the audio device according to claim 15.
  • the method described herein generally serves to format a lossy compressed audio signal.
  • a lossy compressed audio file or a part of such act. Specifically, it may be z.
  • the audio file or parts of it may already be decoded.
  • suitable decoding algorithms via which an at least partial decoding of the mp3-coded audio file was made can be used for an mp3-coded audio file.
  • Under a treatment is basically an at least partial recovery missing, d. H. z. B. in the context of data compression discarded, frequency components or an at least partial replacement missing, d. H. z. B. rejected in the context of data compression, frequency components by comparable frequency components.
  • an at least partial replacement is missing, ie. H. z. B. in the context of data compression discarded, frequency components relevant.
  • a lossy compressed audio signal to be processed is provided.
  • the provision of a corresponding audio signal may in principle be via any physical or non-physical audio source, i. H. z. From an audio device for processing and outputting audio signals.
  • the audio signal is transmitted in a frequency spectrum.
  • energies of the audio signal are correlated with frequencies of the audio signal.
  • the content of the audio signal is limited to its energy, i. H. Amplitude or frequency components, examined and transferred the individual energy components of the audio signal data in a frequency-dependent representation or implemented.
  • the audio signal for this purpose is divided into individual, possibly overlapping, time intervals, which are individually transferred or converted into the frequency spectrum.
  • the transmission or conversion of the audio signal into the frequency spectrum takes place by means of suitable algorithms, i. H. z. By means of (faster) Fourier transform algorithms.
  • the length of the algorithms is basically variable.
  • the examination of the content of the audio signal for its energy components may include a classification and grouping of the energy components as well as an estimation of the energy components of the audio signal.
  • frequencies of local amplitude maxima are determined in the frequency spectrum.
  • the frequency spectrum is examined for local amplitude maxima and the frequencies associated with the respective amplitude maxima are determined.
  • Below a local amplitude maximum is an amplitude maximum value to understand in a defined frequency environment. The determination of local amplitude maxima takes place by means of suitable analysis algorithms.
  • a first selection criterion is determined. On the basis of the first selection criterion, the frequencies of two immediately successive (local) amplitude maxima are preselected, which frequencies satisfy the first selection criterion. In the fourth step, therefore, the frequencies of pairs of immediately successive amplitude maxima with respect to the first selection criterion are examined. Thus, in the fourth step, a pairwise examination of the frequencies of immediately successive amplitude maxima is carried out to determine whether the frequencies corresponding to the respective amplitude maxima satisfy the first selection criterion. In the further steps of the method, only the frequencies that satisfy the first selection criterion are typically considered. In the fourth step, a preselection of the frequencies to be considered below or the associated amplitude maxima ensues.
  • the first selection criterion typically describes a certain threshold frequency value (range) ("threshold".) Frequencies of directly successive amplitude maxima satisfy the first selection criterion, if their frequency difference exceeds the limit frequency value (range) described by the first selection criterion, see the formula given below I illustrated context:
  • ⁇ frequency difference of two immediately successive amplitude maxima
  • Af T limit frequency value (range).
  • the cutoff frequency value can be set by transmitting the preselected frequencies to a bark scale.
  • frequencies can generally be transmitted in a Bark scale.
  • the transmission of the preselected frequencies to a Bark scale is based on the relationship represented by Formula II below:
  • Bark For example: Bark; f: frequency value to be transmitted to the Bark scale.
  • the cutoff frequency value may basically correspond to a bark or to a bark matched or multiplied by an adjustment factor or multiplied by an adjustment factor.
  • the adjustment factor is typically between 0.7 and 1.1, in particular 0.9.
  • the cutoff frequency value thus typically corresponds to 0.7 to 1.1, in particular 0.9, Bark.
  • the adjustment factor gives some variability of the cutoff frequency value.
  • a second selection criterion is determined. Based on the second selection criterion, preselected frequencies of two immediately consecutive local amplitude maxima are selected (based on the first selection criterion) which satisfy the second selection criterion. In the fifth step, preselected frequencies are considered with respect to the second selection criterion. In the fifth step, an investigation of preselected frequencies is made as to whether they (additionally) satisfy the second selection criterion.
  • the second selection criterion can describe a limit energy value (range). Respective preselected frequencies are sufficient for the second selection criterion if the energy content between them falls below the limit energy value (range) ("threshold") described by the second selection criterion.
  • the limit energy value (range) can be defined by a defined limit energy content. Respective preselected frequencies satisfy the second selection criterion if they fall below the limit energy content described by the second selection criterion, cf. for this purpose the relationship represented by the following formula III:
  • S (f) the area described by the frequencies or frequency values fi, f 2 of the two immediately successive amplitude maxima (energy content between the frequencies or frequency values fi, f 2 of the two immediately successive amplitude maxima); T: limit energy content.
  • the limit energy value (range) can alternatively also be determined by a first energy curve emanating from the preselected frequency ("lower frequency”), which is associated with the lower (lower frequency) amplitude maximum, and one from the frequency (“upper frequency”), which the immediately following upper (in terms of frequency higher) amplitude maximum is associated, outgoing second energy profile is generated and the two energy profiles are transferred into the frequency spectrum.
  • the limit energy value is then defined by the respective energy progressions.
  • the first energy curve runs starting from the frequency of the (amplitude) lower amplitude maximum of the two immediately successive amplitude maxima in the direction of the frequency of the (frequency) upper (higher) amplitude maximum of the two immediately successive amplitude maxima.
  • the second energy course runs in the direction of the frequency of the (frequency-wise) lower (lower) amplitude maximum of the two immediately successive amplitude maxima.
  • the generated energy profiles can be transferred into the frequency spectrum in terms of data.
  • the actual frequency distribution between the frequencies and the energy curves defines a closed area or a closed area.
  • the range is defined in terms of frequency as a function of the frequencies of the two immediately adjacent amplitude maxima and, in terms of energy, by the actual frequency curve between the amplitude maxima and the energy profiles running between them.
  • the range typically only contains energy values> zero. If one considers the area geometrically with respect to the frequency spectrum, the area of the area immediately adjacent to the amplitude maxima, the energy or frequency gradients running between them and the frequency axis (x axis) corresponds to a geometrically defined area.
  • the generation of energy curves is typically based on a psychoacoustic model.
  • a psychoacoustic model is typically used, or the energy courses are derived from a psychoacoustic model.
  • the psychoacoustic model generally describes those frequency components of a particular sound which are distinct from the human ear in a particular sound environment, i. H. possibly in the presence of other noises, are perceptible.
  • a preferred psychoacoustic model is the model of spectral masking, which describes that human hearing ability can not or only with reduced sensitivity perceive certain frequency components of a particular sound.
  • These masking or masking effects are based essentially on the anatomical or mechanical conditions of the human inner ear and, for example, conditional that low-energy or low-pitched tones in the middle frequency range are imperceptible in the low frequency range with simultaneous reproduction of high-energy or loud sounds; the tones in the low frequency range mask the tones in the middle frequency range.
  • the energy courses are derived, in particular, from the hearing thresholds of the human hearing given by the respective psychoacoustic model at respective preselected frequencies. This means that the psychoacoustic model is applied to the frequencies of the two immediately successive amplitude maxima.
  • the first energy curve corresponds to the part of the hearing threshold derived for the frequency of the lower amplitude maximum from the psychoacoustic model, which extends in the direction of increasing frequencies.
  • the second energy curve corresponds to the part of the hearing threshold derived for the frequency of the upper amplitude maximum from the psychoacoustic model, which extends in the direction of falling frequencies.
  • an audio filling signal is generated or generated.
  • the audio filling signal is typically generated specifically with regard to the previously determined reprocessing frequency ranges within the audio signal to be processed.
  • the audio filling signal is thus typically generated specifically with regard to the frequency range defined by immediately successive frequencies satisfying both the first and the second selection criterion in order to fill it and to fill the "energy valley" between the frequencies at least in sections, in particular completely.
  • the generated audio filling signal therefore expediently has a frequency range lying between the frequencies of respective immediately successive amplitude maxima.
  • a seventh step of the method the actual processing of the audio signal is carried out by introducing the audio filling signal into respective frequency ranges between respective frequencies satisfying the first and second selection criteria, so that a respective frequency range is filled at least in sections, in particular completely, with the audio filling signal.
  • corresponding "energy valleys" resulting from the data compression of the audio signal are determined and, in the form of the audio filling signal generated with regard to the determined “energy valleys”, specifically filled with a specific data content, whereby a processing of the audio signal is realized. It follows that the processing according to the method of the audio signal, as mentioned above, in particular by an at least partial replacement of missing, d. H. z. B. in the context of data compression discarded, frequency components of the audio signal is realized.
  • the described steps of the method provide an improved method for processing a lossy compressed audio signal, in particular with regard to the efficiency of the processing and the quality of the processed audio signal.
  • the corresponding processed audio signal via at least one z. B. trained as a speaker device or at least to output such a comprehensive signal output device.
  • An optional eighth step of the method may thus provide for output of a processed audio signal via at least one signal output device.
  • a correspondingly processed stored audio signal can be output at a later time via at least one corresponding signal output device and / or transmitted to at least one communication partner via a suitable, in particular wireless, communication network.
  • An optional eighth step of the method may therefore also provide for storing a processed audio signal in at least one memory device and / or transmitting a processed audio signal to at least one communication partner.
  • the conditioned audio signal may be subjected to inverse Fourier transformation prior to output and / or storage and / or transmission.
  • one out of the selected frequency (“lower frequency”) associated with the lower (lower frequency) amplitude maximum third, energy curve and one of the selected frequency (“upper frequency”), which is the (frequency higher) amplitude maximum associated, outgoing, possibly fourth, energy curve is generated and these two energy profiles are transmitted to the frequency spectrum.
  • the possibly third energy curve runs starting from the frequency of the (amplitude) lower amplitude maximum of the two immediately successive amplitude maxima in the direction of the frequency of the (frequency) upper amplitude maximum of the two immediately successive amplitude maxima.
  • the possibly fourth energy curve runs starting from the frequency of the upper (higher) amplitude maximum of the two immediately successive amplitude maxima in the direction of the frequency of the (lower frequency) lower (lower) amplitude maximum of the two immediately successive amplitude maxima.
  • the generated energy curves can in turn be transferred into the frequency spectrum in terms of data.
  • the frequencies and the energy curves also define a closed area or a closed area.
  • the range is again defined by the frequencies of the two immediately successive amplitude maxima and, in terms of energy, by the energy progresses between them.
  • the range typically only contains energy values> zero. If one regards the area geometrically with respect to the frequency spectrum, the area corresponds again to the one defined by the two immediately adjacent amplitude maxima, the energy or frequency gradients running between them and the frequency axis (x-axis) Area.
  • the generation of the optionally third and fourth energy curves typically likewise takes place on the basis of a psychoacoustic model.
  • a psychoacoustic model is typically used, or the energy courses are derived from a psychoacoustic model.
  • the statements in connection with the first two energy curves apply analogously.
  • the possibly third and fourth energy courses are likewise derived, in particular, from the hearing thresholds of the human hearing given at respective preselected frequencies by the respective psychoacoustic model.
  • the possibly third energy curve corresponds to the part of the hearing threshold derived for the frequency of the lower amplitude maximum from the psychoacoustic model, which extends in the direction of increasing frequencies.
  • the possibly fourth energy curve corresponds to the part of the hearing threshold derived for the frequency of the upper amplitude maximum from the psychoacoustic model, which extends in the direction of falling frequencies.
  • the audio filling signal is subsequently introduced at least in sections, in particular completely, into the region of the frequency spectrum defined by the two preselected frequencies and the respective energy profiles.
  • the audio signal is processed in this case by introducing the audio filling signal into the frequency range of the frequency spectrum defined by the frequencies of the two immediately adjacent amplitude maxima and the respective energy profiles, so that the range of the frequency range defined by the frequencies of the two immediately successive amplitude maxima and the respective energy profiles Frequency spectrum at least partially, in particular completely, is filled with the audio filling signal or is.
  • the audio filling signal can be generated independently of or dependent on acoustic parameters of the audio signal to be processed, in particular with regard to the respective energy and frequency components of the audio signal.
  • the audio filling signal is expediently generated independently of acoustic parameters of the audio signal, ie, solely with regard to the at least partially filling of the region of the frequency spectrum defined by the frequencies of the two immediately adjacent amplitude maxima, since the computation effort for generating the audio filling signal may be such can be significantly reduced.
  • the filling or filling of the area of the frequency spectrum defined by the frequencies of the two immediately successive amplitude maxima can be dependent on specific acoustic parameters of the audio signal, in particular the amplitude and / or frequency response, or certain acoustic parameters of another audio signal to be processed, in particular of the amplitude and / or frequency response. In this way, a possibly more natural perception of the processed audio signal for the human ear can be realized.
  • the frequency spectrum into which the audio signal is transmitted according to the method can be a Bark scale.
  • the 24 individual barks or bands of the Bark scale are known to correspond to the 24 individual frequency groups of the human ear, d. H. those frequency ranges that are evaluated jointly by human hearing.
  • the individual barks or bands of the Bark scale contain different frequencies or frequency ranges or bandwidths. Possible frequency bands of the frequency spectrum can correspond to the 24 bars or bands of the Bark scale.
  • the invention further relates to a device for processing a lossy compressed audio signal according to the method as described above.
  • the device comprises at least one control device implemented in hardware and / or software, which is distinguished by the fact that it is used for
  • the device comprises a control device equipped or communicating with corresponding devices.
  • the device may be part of an audio device or an audio system for a motor vehicle.
  • the invention further relates to an audio device or an audio system for a motor vehicle.
  • the audio device may be part of a motor vehicle-side multimedia device for outputting multimedia contents, in particular of audio and / or video content, to occupants of a motor vehicle.
  • the audio device comprises at least one signal output device, i. H. z. B. a speaker device, which is set up for the acoustic output processed audio signals in at least a part of a passenger compartment forming interior of a motor vehicle.
  • the audio device is characterized in that it has at least one apparatus for processing lossy compressed audio signals for processing lossy compressed audio signals.
  • FIG. 1 is a schematic diagram of an apparatus for performing a method according to an embodiment
  • FIG. 2 is a block diagram of a method according to an embodiment
  • 3, 4 are each a schematic diagram of a psychoacoustic model according to a
  • FIGS. 5-8 each show a schematic representation of a frequency spectrum in which energies of an audio signal are correlated with frequencies of the audio signal, according to an exemplary embodiment.
  • Fig. 1 shows a schematic diagram of a device 1 for processing a lossy compressed audio signal 2.
  • the audio signal 2 it may, for. B. may be a lossy compressed audio file. Specifically, it may be z.
  • the audio file may already be at least partially decoded, for example, the audio file may include a piece of music.
  • the device 1 shown in the embodiment forms part of an audio device
  • the audio device 3 may be part of a motor vehicle-side multimedia device (not shown) for outputting multimedia contents, in particular of audio and / or video content, to occupants of the motor vehicle 4.
  • the audio device 3 comprises at least one z. B. designed as a speaker device or at least one such comprehensive signal output device 5, which is set up for the acoustic output processed audio signals 6 in at least a part of the passenger compartment forming interior 7 of the motor vehicle 4.
  • the device 1 comprises a central control device 8 implemented in hardware and / or software, which is set up to implement a method, explained in more detail below with reference to FIG. 2, for processing lossy compressed audio signals 2.
  • the device 1 comprises a control device 8 equipped with corresponding devices.
  • FIG. 2 shows a block diagram of an embodiment of a method for processing lossy compressed audio signals 2. The method can be carried out with the device 1 described above.
  • the lossy compressed audio signal 2 to be processed is provided.
  • the provision of the audio signal 2 may in principle be via any physical or non-physical audio source, i. H. z. B. of the audio device 3, take place.
  • the audio signal 2 z. B. from a data memory (not shown) of the audio device 3 are provided.
  • the audio signal 2 is transmitted in a frequency spectrum.
  • energies of the audio signal 2 are correlated with frequencies of the audio signal 2.
  • suitable algorithms d. H. z. B. by means of (faster) Fourier transformation algorithms, data transmitted in a frequency-dependent representation.
  • a corresponding frequency spectrum is u. a. in Fig. 5 in a schematic representation. ,
  • frequencies f, local amplitude maxima are determined in the frequency spectrum; the frequency spectrum is therefore examined for local amplitude maxima and the frequencies f, p associated with the respective amplitude maxima are determined.
  • a in Figs. 5 - 8 by a point graphically emphasized local amplitude maximum is to be understood as an amplitude maximum value in a defined frequency environment.
  • a first selection criterion is determined.
  • the frequencies f, of two immediately successive (local) amplitude maxima are preselected, which frequencies satisfy the first selection criterion.
  • the frequencies f, of pairs of directly successive amplitude maxima with respect to the first selection criterion are examined to determine whether the frequencies f satisfy the first selection criterion.
  • only the frequencies f 1 satisfying the first selection criterion are considered.
  • a preselection of the frequencies f.sub.i to be considered below ensues.
  • the first selection criterion describes a specific limit frequency value Af T.
  • Frequencies f of immediately successive amplitude maxima satisfy the first selection criterion if their frequency difference Af exceeds the limit frequency value Af T described by the first selection criterion, cf. for this purpose the relationship represented by the following formula:
  • ⁇ frequency difference of two immediately successive amplitude maxima
  • Af T limit frequency value
  • the cut-off frequency value Af T is set by transmitting the preselected frequencies f i into a bark scale.
  • the transmission of the preselected frequencies f i into a bark scale is based on the relationship represented by the following formula:
  • Bark For example: Bark; f: frequency value to be transmitted to the Bark scale.
  • both preselected frequencies f 1 and the limit frequency values Af T described by the first selection criterion can be transmitted to the bark scale.
  • the cut-off frequency value Af T may correspond to a bark or to a bark adapted or multiplied by an adjustment factor or multiplied by an adjustment factor.
  • the adjustment factor is typically between 0.7 and 1.1, in particular 0.9.
  • the cutoff frequency value thus typically corresponds to 0.7 to 1.1, in particular 0.9, Bark.
  • a second selection criterion is determined. Based on the second selection criterion, preselected frequencies f 1 are selected (based on the first selection criterion) which satisfy (additionally) the second selection criterion.
  • an investigation of preselected frequencies f is carried out to determine whether they (additionally) satisfy the second selection criterion.
  • the (additionally) the second selection criterion sufficient frequencies f can in turn be transferred to a Bark scale.
  • the second selection criterion can describe a limit energy value. Respective preselected frequencies f satisfy the second selection criterion if the energy content between them falls below the limit energy value described by the second selection criterion.
  • the limit energy value can be defined by a defined limit energy content T.
  • Respective preselected frequencies f satisfy the second selection criterion if they fall below the limit energy content T described by the second selection criterion, cf. for this purpose the relationship represented by the following formula:
  • S (f) the area described by the frequencies f- 1 , f 2 of the two immediately consecutive amplitudes (energy content between the frequencies f- 1 , f 2 of the two immediately successive amplitude maxima); T: limit energy content.
  • FIG. 6 illustrates the basic representation shown in FIG. 6 of a frequency spectrum containing two preselected frequencies f 1 , f 2 , which is also a section of another, namely the frequency spectrum shown in FIG. 5.
  • FIG. 6 illustrates the surface (hatched) described by the frequencies f- 1 , f 2 of the two immediately successive amplitude maxima, and the bounded energy content T represented by a horizontal line.
  • the hatched area corresponds to the integral represented by the above formula.
  • the limit energy value can alternatively also be determined by a first energy curve EV1 originating from the preselected frequency f- 1 ("lower frequency"), which is associated with the lower (frequency-lower) amplitude maximum, and one from the preselected frequency f2 ("upper Frequency) which is associated with the upper (frequency higher) amplitude maximum, outgoing second energy curve EV2 is generated and the two energy curves EV1, EV2 are transmitted into the frequency spectrum Energy curves EV1, EV2 defined.
  • the generated energy profiles EV1, EV2 can be transmitted into the frequency spectrum in terms of data.
  • the first energy curve EV1 runs starting from the lower frequency f-1 in the direction of the upper frequency f 2 .
  • the second energy curve EV2 runs from the upper frequency f 2 in the direction of the lower frequency,
  • a closed area or a closed area is defined.
  • the range is defined in terms of frequency share by the two frequencies f- 1, 2 and energy-wise by the actual frequency response and the energy progressions EV1, EV2 running between them.
  • the range typically only contains energy values> zero. If one considers the area geometrically with respect to the frequency spectrum, the area of the amplitude maxima defined by the frequencies f- 1, 2 of the two immediately adjacent amplitude maxima, the energy or frequency gradients running between them and the frequency axis (x-axis) corresponds geometrically Fig. 7 hatched area shown.
  • the generation of the energy curves EV1, EV2 is based on a psychoacoustic model.
  • a preferred psychoacoustic model is the model of spectral masking. It can be seen from FIG. 3 that the energy profiles EV1, EV2 are derived from the hearing thresholds of the human hearing given by the respective preselected frequencies f- 1, 2 by the respective psychoacoustic model. This means that the psychoacoustic model used is applied to the two frequencies f- 1, 2 respectively.
  • the first energy curve EV1 corresponds to the part of the hearing threshold derived from the psychoacoustic model for the lower frequency f-1, which extends in the direction of increasing frequencies (see left curly brace in FIG.
  • the second energy curve EV2 corresponds to the part of the hearing threshold derived for the upper frequency f 2 from the psychoacoustic model, which extends in the direction of falling frequencies (compare right curly brace in FIG.).
  • the energy profiles EV1, EV2 it is also possible for the energy profiles EV1, EV2 to intersect or intersect in a value range above the x-axis.
  • an audio fill signal AFS is generated or generated.
  • the audio filling signal AFS is generated specifically with regard to the previously determined reprocessing frequency ranges within the audio signal 2 to be processed.
  • the audio filling signal AFS is thus generated in a targeted manner with regard to the frequency range defined by both the first and the second selection criterion f 1 or f 1 2 of the two immediately successive amplitude maxima, in order to fill this out and between frequencies f, f.
  • the generated audio filling signal AFS therefore has a frequency range lying between the frequencies f, of respective immediately successive amplitude maxima.
  • the audio filling signal AFS can be generated as a function of or independent of acoustic parameters of the audio signal 2, in particular with respect to respective energy and frequency components of the audio signal 2.
  • the audio filling signal AFS is independent of acoustic parameters of the audio signal 2, ie alone in terms of filling the frequency share by the frequencies f- ⁇ , 2 and energy proportionately by the actual frequency response and running between them energy curves EV3, EV4 area , generated.
  • a seventh step S7 of the method the actual processing of the audio signal 2 takes place by introducing the audio filling signal AFS into respective frequency ranges between respective frequencies f corresponding to the first and second selection criteria f, so that a respective frequency range is filled with the audio filling signal AFS.
  • a further or third energy curve EV3 originating from the selected lower (lower) frequency f-1, which is associated with the lower (lower frequency) amplitude maximum, and one from the selected upper one (FIG. higher) frequency f 2 , which is the upper (frequency higher) amplitude maximum associated, outgoing further or fourth energy profile EV4 generated.
  • the third energy curve EV3 runs starting from the lower frequency f-1 in the direction of the upper frequency f 2 .
  • the fourth energy curve EV4 runs from the upper frequency f 2 in the direction of the lower frequency f
  • a closed area or a closed area is defined.
  • the range is defined in terms of frequency as a function of the frequencies f- 1, 2 of the amplitude maxima and, in terms of energy, by the actual frequency response and the energy courses EV3, EV4 running between them.
  • the range typically only contains energy values> zero. Looking at the area geometrically in terms of the frequency spectrum, corresponding to defined geometric of the area defined by the frequencies 2 of the two directly adjacent amplitude maxima, the extending between these energy or frequency gradients and the frequency axis (x-axis) in Fig. 8 hatched area shown.
  • the generation of the energy curves EV3, EV4 also takes place on the basis of a psychoacoustic model.
  • a preferred psychoacoustic model is also the model of spectral masking (see Fig. 4). It can be seen from FIG. 4 that the energy profiles EV3, EV4 are derived from the hearing thresholds of the human hearing given by the respective preselected frequencies f- 1, 2 by the respective psychoacoustic model. This also means here that the psychoacoustic model used is applied in each case to the two immediately successive frequencies fi , 2.
  • the third energy curve EV3 corresponds to the part of the auditory threshold derived from the psychoacoustic model for the lower frequency f-1, which extends in the direction of increasing frequencies (compare left curly brace in FIG.
  • the fourth energy curve EV4 corresponds to the part of the auditory threshold derived from the psychoacoustic model for the upper frequency f 2 , which extends in the direction of falling frequencies (see right-hand curly brace in FIG. 4, it is also possible here for the energy profiles EV3, EV4 to intersect or intersect in a value range above the x-axis.
  • the (first two) energy curves EV1, EV2 can differ from the third and fourth energy curves Ev3, EV4.
  • An optional eighth step S8 of the method may include outputting a processed audio signal 2 via at least one signal output device 5 and / or storing a processed audio signal 2 in at least one memory device (not shown) and / or transmitting a processed audio signal 2 to at least one communication partner (not shown).
  • the conditioned audio signal 2 may be subjected to an inverse Fourier transformation before output and / or storage and / or transmission.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Mathematical Physics (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

La présente invention concerne un procédé de mise en forme d'un signal audio 2 comprimé avec perte, qui comprend les étapes suivantes : - fournir un signal audio 2 comprimé avec perte, - transmettre le signal audio 2 dans un spectre de fréquence dans lequel des énergies du signal audio 2 sont corrélées avec des fréquences du signal audio 2, - déterminer les fréquences fi d'amplitudes crêtes locales dans le spectre de fréquence, - définir un premier critère de sélection et présélectionner les fréquences fi de deux amplitudes crêtes locales se succédant directement, lesdites fréquences satisfaisant au premier critère de sélection, - définir un second critère de sélection et sélectionner des fréquences fi présélectionnées de deux amplitudes crêtes locales se succédant directement, lesdites fréquences satisfaisant au premier critère de sélection et satisfaisant également au second critère de sélection, - produire un signal audio de remplissage (AFS) et mettre en forme le signal audio 2 par insertion du signal audio de remplissage (AFS) dans une plage de fréquences située entre les fréquences fi satisfaisant au second critère de sélection, de sorte que la plage de fréquences est remplie au moins par endroits, en particulier en totalité, par le signal audio de remplissage (AFS).
EP17711600.1A 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte Active EP3403260B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102016104665.5A DE102016104665A1 (de) 2016-03-14 2016-03-14 Verfahren und Vorrichtung zur Aufbereitung eines verlustbehaftet komprimierten Audiosignals
PCT/EP2017/055820 WO2017157841A1 (fr) 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte

Publications (2)

Publication Number Publication Date
EP3403260A1 true EP3403260A1 (fr) 2018-11-21
EP3403260B1 EP3403260B1 (fr) 2020-03-04

Family

ID=58358566

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17711600.1A Active EP3403260B1 (fr) 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte

Country Status (5)

Country Link
US (1) US10734000B2 (fr)
EP (1) EP3403260B1 (fr)
CN (1) CN108174614B (fr)
DE (1) DE102016104665A1 (fr)
WO (1) WO2017157841A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110491407B (zh) * 2019-08-15 2021-09-21 广州方硅信息技术有限公司 语音降噪的方法、装置、电子设备及存储介质
CN113192519B (zh) * 2021-04-29 2023-05-23 北京达佳互联信息技术有限公司 音频编码方法和装置以及音频解码方法和装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5479560A (en) 1992-10-30 1995-12-26 Technology Research Association Of Medical And Welfare Apparatus Formant detecting device and speech processing apparatus
DE10103134A1 (de) * 2001-01-24 2002-08-08 Harman Becker Automotive Sys Dekodiereinrichtung, Dekodierverfahren und Kraftfahrzeugaudiosystem mit einer derartigen Dekodiereinrichtung
US7469206B2 (en) * 2001-11-29 2008-12-23 Coding Technologies Ab Methods for improving high frequency reconstruction
US7447631B2 (en) * 2002-06-17 2008-11-04 Dolby Laboratories Licensing Corporation Audio coding system using spectral hole filling
DE50312002D1 (de) * 2003-07-24 2009-11-19 Palm Inc Verfahren und Vorrichtung zum Entzerren eines mit einem äusseren Störsignal behafteten Audiosignals
US7562021B2 (en) * 2005-07-15 2009-07-14 Microsoft Corporation Modification of codewords in dictionary used for efficient coding of digital media spectral data
BR112012026324B1 (pt) * 2010-04-13 2021-08-17 Fraunhofer - Gesellschaft Zur Förderung Der Angewandten Forschung E. V Codificador de aúdio ou vídeo, decodificador de aúdio ou vídeo e métodos relacionados para o processamento do sinal de aúdio ou vídeo de múltiplos canais usando uma direção de previsão variável
JP2013148724A (ja) * 2012-01-19 2013-08-01 Sony Corp 雑音抑圧装置、雑音抑圧方法およびプログラム
JP5949270B2 (ja) * 2012-07-24 2016-07-06 富士通株式会社 オーディオ復号装置、オーディオ復号方法、オーディオ復号用コンピュータプログラム
RU2660605C2 (ru) * 2013-01-29 2018-07-06 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Концепция заполнения шумом
CN116665683A (zh) * 2013-02-21 2023-08-29 杜比国际公司 用于参数化多声道编码的方法
EP2830060A1 (fr) * 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Remplissage de bruit de codage audio multicanal
EP2830064A1 (fr) * 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Appareil et procédé de décodage et de codage d'un signal audio au moyen d'une sélection de tuile spectrale adaptative
EP2960903A1 (fr) * 2014-06-27 2015-12-30 Thomson Licensing Procédé et appareil de détermination de la compression d'une représentation d'une trame de données HOA du plus petit nombre entier de bits nécessaires pour représenter des valeurs de gain non différentielles
EP3162087B1 (fr) * 2014-06-27 2021-03-17 Dolby International AB Représentation de trames de données hoa codées qui comprend des valeurs de gain non différentielles associées à des signaux de canaux de trames spécifiques parmi les trames de données d'une représentation de trames de données hoa
CN110459229B (zh) * 2014-06-27 2023-01-10 杜比国际公司 用于解码声音或声场的高阶高保真度立体声响复制（hoa）表示的方法

Also Published As

Publication number Publication date
US10734000B2 (en) 2020-08-04
US20190080702A1 (en) 2019-03-14
CN108174614A (zh) 2018-06-15
WO2017157841A1 (fr) 2017-09-21
EP3403260B1 (fr) 2020-03-04
DE102016104665A1 (de) 2017-09-14
CN108174614B (zh) 2018-12-28

Similar Documents

Publication Publication Date Title
DE69122648T2 (de) Digitale Teilbandkodierungsvorrichtung
DE112016006218B4 (de) Schallsignal-Verbesserungsvorrichtung
EP1953739B1 (fr) Procédé et dispositif pour réduire le bruit dans un signal décodé
EP2919652B1 (fr) Traitement de signaux audio pour la thérapie des acouphènes
EP2372702A1 (fr) Encastreur destiné à l'encastrement d'un filigrane dans la représentation d'une information, détecteur pour la détection d'un filigrane dans la représentation d'une information, procédé, programme informatique et signal informatique
EP2372634A2 (fr) Encastreur destiné à l'encastrement d'un filigrane dans la représentation d'une information, détecteur destiné à la détection d'un filigrane dans la représentation d'une information, procédé et programme informatique
DE102010039589A1 (de) Hörhilfe- und/oder Tinnitus-Therapie-Gerät
DE60311619T2 (de) Datenreduktion in Audiokodierern unter Ausnutzung nichtharmonischer Effekte
DE102006047983A1 (de) Verarbeitung eines Eingangssignals in einem Hörgerät
EP3197181B1 (fr) Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif
EP3403260B1 (fr) Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte
DE102021004108B3 (de) Verfahren zur Maskierung von unerwünschten Störgeräuschen und Fahrzeug
EP1351550A1 (fr) Procédé d'adaptation d'une amplification de signal dans une prothèse auditive et prothèse auditive
EP1212751B1 (fr) Procede d'attenuation de bruits parasites dans un champ de signal
EP3232691B1 (fr) Procédé de détermination de la réponse d'impulsion spatiale d'un habitacle d'un véhicule automobile
DE102009030551A1 (de) Verfahren zum lautheitsbasierten Einstellen der Verstärkung eines Hörgeräts und zugehöriges Hörgerät
DE102018207530A1 (de) Vorrichtung und Verfahren für Verbesserung der Privatsphäre
EP1130577B1 (fr) Procédé de reconstruction des basses fréquences du signal de parole à partir de fréquences moyennes
DE102022124486B3 (de) Verfahren und Audiowiedergabevorrichtung zum Kompensieren eines Geräuschs in einem Innenraum eines Kraftfahrzeugs sowie zugehöriges Kraftfahrzeug
DE602004007249T2 (de) Einheitliche Behandlung von aufgelösten und nicht-aufgelösten Oberwellen
DE19626329A1 (de) Verfahren zur Analyse von Schallereignissen
EP3834723A1 (fr) Procédé de détermination du seuil auditif d'un sujet humain
DE102014003057B4 (de) Verfahren zur Rekonstruierung hoher Frequenzen bei verlustbehafteter Audiokomprimierung
EP3965434A1 (fr) Procédé de sonorisation améliorée de plusieurs zones de sonorisation
DE102021103769A1 (de) Verfahren zur Signalverarbeitung

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180606

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190717

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191122

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1241292

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502017004091

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200604

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200604

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200605

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200704

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502017004091

Country of ref document: DE

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200313

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200313

26N No opposition filed

Effective date: 20201207

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200504

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1241292

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220313

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220313

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240326

Year of fee payment: 8