EP3403260B1 - Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte - Google Patents

Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte Download PDF

Info

Publication number
EP3403260B1
EP3403260B1 EP17711600.1A EP17711600A EP3403260B1 EP 3403260 B1 EP3403260 B1 EP 3403260B1 EP 17711600 A EP17711600 A EP 17711600A EP 3403260 B1 EP3403260 B1 EP 3403260B1
Authority
EP
European Patent Office
Prior art keywords
frequencies
audio
frequency
audio signal
energy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP17711600.1A
Other languages
German (de)
English (en)
Other versions
EP3403260A1 (fr
Inventor
Denis Perechnev
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ask Industries GmbH
Original Assignee
Ask Industries GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ask Industries GmbH filed Critical Ask Industries GmbH
Publication of EP3403260A1 publication Critical patent/EP3403260A1/fr
Application granted granted Critical
Publication of EP3403260B1 publication Critical patent/EP3403260B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G10L19/0208—Subband vocoders
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/0017—Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/012—Comfort noise or silence coding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
    • G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility

Definitions

  • the invention relates to a method for processing a lossy compressed audio signal.
  • the data compression of audio signals or information, such as. B. music files is known in and of itself.
  • the purpose of data compression is to reduce the data size of corresponding audio signals.
  • data compression can be lossy or not lossy.
  • the lossy data compression should be considered in particular, which can be implemented, for example, by data-based rejection of frequency components lying on the edge of the human hearing range. The subjective hearing perception by a listener should hardly be impaired in this way.
  • lossy compressed audio signals Because of the reduced sound quality in comparison of lossy compressed audio signals, it is sometimes desirable to prepare lossy compressed audio signals, i. H. to at least partially restore correspondingly rejected frequency components or to replace them with comparable frequency components.
  • EP 1501190 A1 discloses a method and a device for equalizing an audio signal with an external interference signal to increase the intelligibility of the audio signal.
  • the invention is therefore based on the object of specifying an improved method for processing a lossy compressed audio signal.
  • the object is achieved by a method according to claim 1.
  • the dependent claims relate to advantageous embodiments of the method.
  • the object is further achieved by the device according to claim 14 and by the audio device according to claim 15.
  • the method described herein is generally used to process a lossy compressed audio signal.
  • a processable or processed audio signal it can e.g. B. a lossy compressed audio file or part of such.
  • it can be e.g. B. a lossy compressed audio file using an mp3 algorithm, d. H. is an mp3-encoded audio file or mp3 file.
  • the audio file or parts of it can already be decoded.
  • suitable decoding algorithms can be used, by means of which an at least partial decoding of the mp3-coded audio file was carried out. The same applies, of course, to audio data that was not coded using an mp3 algorithm, but rather using other algorithms.
  • the audio file can e.g. B.
  • audio signals e.g. B. include a piece of music.
  • Processing is generally an at least partial restoration that is missing, i. H. z. B. discarded as part of data compression, frequency components or an at least partial replacement missing, d. H. z. B. in the context of data compression discarded to understand frequency components by comparable frequency components.
  • lossy compressed audio signals in particular require an at least partial replacement of missing, ie. H. z. B. discarded in the context of data compression, frequency components relevant.
  • a lossy compressed audio signal to be processed is provided.
  • a corresponding audio signal can in principle be provided via any physical or non-physical audio source, that is to say, for example, from an audio device for processing and outputting audio signals.
  • the audio signal is transmitted into a frequency spectrum.
  • energies of the audio signal are correlated with frequencies of the audio signal.
  • the content of the audio signal is reduced to its energy, i.e. H. Amplitude or frequency components are examined and the individual energy components of the audio signal are transmitted or converted in terms of data into a frequency-dependent representation.
  • the audio signal is typically divided into individual, possibly overlapping, time intervals, which are individually transmitted or converted into the frequency spectrum.
  • the transmission or conversion of the audio signal into the frequency spectrum takes place by means of suitable algorithms, i. H. z. B. using (faster) Fourier transform algorithms.
  • the length of the algorithms is basically variable.
  • the examination of the content of the audio signal for its energy components can include a classification and grouping of the energy components and an estimate of the energy components of the audio signal.
  • frequencies of local amplitude maxima are determined in the frequency spectrum.
  • the frequency spectrum is examined for local amplitude maxima and the frequencies associated with the respective amplitude maxima are determined.
  • Below a local amplitude maximum is an amplitude maximum value to be understood in a defined frequency range.
  • Local amplitude maxima are determined using suitable analysis algorithms.
  • a first selection criterion is defined. On the basis of the first selection criterion, the frequencies of two immediately following (local) amplitude maxima are preselected, which frequencies meet the first selection criterion. In the fourth step, the frequencies of pairs of immediately consecutive amplitude maxima are examined with regard to the first selection criterion. In the fourth step, the frequencies of immediately successive amplitude maxima are then examined in pairs to determine whether the frequencies associated with the respective amplitude maxima meet the first selection criterion. In the further steps of the method, only the frequencies that meet the first selection criterion are typically considered. In the fourth step, the frequencies to be considered below and the associated amplitude maxima are preselected.
  • the first selection criterion typically describes a specific limit frequency value (range) (“threshold”). Frequencies of immediately successive amplitude maxima satisfy the first selection criterion if their frequency difference exceeds the limit frequency value (range) described by the first selection criterion, cf. the relationship represented by the formula I shown below: ⁇ f i > ⁇ f T
  • ⁇ f i applies: frequency difference between two immediately following amplitude maxima; ⁇ f T : cutoff frequency value (range).
  • Both the preselected frequencies and the limit frequency value described by the first selection criterion can be transferred to the Bark scale using the relationship represented by Formula II.
  • the limit frequency value can fundamentally correspond to a bark or a bark adjusted by an adaptation factor or multiplied by an adaptation factor.
  • the adjustment factor is typically between 0.7 and 1.1, in particular 0.9.
  • the limit frequency value thus typically corresponds to 0.7 to 1.1, in particular 0.9, Bark.
  • the frequency difference of the respective frequencies should correspond to a bark or almost a bark in order to meet the first selection criterion.
  • a certain variability of the limit frequency value is given by the adjustment factor.
  • a second selection criterion is defined. On the basis of the second selection criterion, (based on the first selection criterion) preselected frequencies of two immediately successive local amplitude maxima are selected which satisfy the second selection criterion. In the fifth step, preselected frequencies are considered with regard to the second selection criterion. In the fifth step, preselected frequencies are then examined to determine whether (additionally) they meet the second selection criterion.
  • the second selection criterion can describe a limit energy value (range). Respective preselected frequencies meet the second selection criterion if the energy content between them falls below the limit energy value (range) described by the second selection criterion (“threshold”).
  • the limit energy value (range) can be defined by a specified limit energy content. Respective preselected frequencies meet the second selection criterion if they fall below the limit energy content described by the second selection criterion, cf. the relationship represented by the formula III shown below: ⁇ f 1 f 2 S f 2 df ⁇ T
  • S (f) the area described by the frequencies or frequency values f 1 , f 2 of the two immediately successive amplitude maxima (energy content between the frequencies or frequency values f 1 , f 2 of the two immediately successive amplitude maxima); T: Limit energy content.
  • the limit energy value (range) can alternatively also be determined by using a first energy curve starting from the preselected frequency ("lower frequency”) which belongs to the lower (lower frequency) amplitude maximum and a frequency ("upper frequency") which immediately follows the upper (frequency higher) maximum amplitude is associated, outgoing second energy curve is generated and the two energy curves are transferred to the frequency spectrum.
  • the limit energy value is then defined by the respective energy profiles.
  • the first energy curve runs from the frequency of the (frequency-wise) lower amplitude maximum of the two immediately following amplitude maxima in the direction of the frequency of the (frequency-wise) upper (higher) amplitude maximum of the two immediately successive amplitude maxima.
  • the second energy curve starts from the frequency of the (in terms of frequency) upper amplitude maximum of the two immediately following amplitude maxima in the direction of the frequency of (in terms of frequency) lower (lower) amplitude maximum of the two immediately following amplitude maxima.
  • the energy profiles generated can be transferred into the frequency spectrum in terms of data.
  • a closed area or area is defined by the actual frequency curve between the frequencies and the energy curves.
  • the range is defined in terms of frequency by the frequencies of the two immediately adjacent amplitude maxima and in terms of energy by the actual frequency profile between the amplitude maxima and the energy profiles running between them. The range typically only contains energy values ⁇ zero. If one looks at the area geometrically in relation to the frequency spectrum, the area corresponds to the area geometrically defined by the two immediately adjacent amplitude maxima, the energy or frequency profiles running between these and the frequency axis (x-axis).
  • the energy profiles are typically generated on the basis of a psychoacoustic model.
  • a psychoacoustic model is therefore typically used to generate the energy profiles, or the energy profiles are derived from a psychoacoustic model.
  • the psychoacoustic model generally describes those frequency components of a certain sound which are heard by the human ear in a certain noise environment, i.e. H. if necessary in the presence of other noises.
  • a preferred used psychoacoustic model is the model of spectral masking or masking, by which it is described that the human hearing ability cannot perceive certain frequency components of a certain noise or only with reduced sensitivity.
  • These masking or masking effects are essentially based on the anatomical or mechanical conditions of the human inner ear and, for example, mean that low-energy or quiet tones in the middle frequency range are not perceivable with simultaneous reproduction of high-energy or loud tones in the low frequency range; the tones in the low frequency range mask the tones in the medium frequency range.
  • the energy profiles are derived in particular from the hearing thresholds of human hearing given by the respective psychoacoustic model at the respective preselected frequencies. This means that the psychoacoustic model is applied to the frequencies of the two immediately following amplitude maxima.
  • the first energy curve corresponds to the part of the hearing threshold derived from the psychoacoustic model for the frequency of the lower amplitude maximum, which extends in the direction of increasing frequencies.
  • the second energy curve corresponds to the part of the hearing threshold derived from the psychoacoustic model for the frequency of the upper amplitude maximum, which extends in the direction of falling frequencies.
  • an audio fill signal is generated or generated.
  • the audio fill signal is typically generated in a targeted manner with regard to the previously determined frequency ranges to be processed within the audio signal to be processed.
  • the audio fill signal is therefore typically generated in a targeted manner with regard to the frequency range defined by immediately successive frequencies that satisfy both the first and the second selection criterion in order to fill it and to fill the "energy valley" between the frequencies, at least in sections, in particular completely.
  • the generated audio fill signal therefore expediently has a frequency range lying between the frequencies of respective immediately successive amplitude maxima.
  • the generation of the audio fill signal takes place e.g. B. by means of a suitable signal generator.
  • a seventh step of the method the actual processing of the audio signal is carried out by introducing the audio fill signal into respective frequency ranges between respective frequencies satisfying the first and second selection criteria, so that a respective frequency range is filled with the audio fill signal at least in sections, in particular completely.
  • corresponding "energy troughs" resulting from the data compression of the audio signal are determined and specifically filled with a certain data content in the form of the audio fill signal generated with regard to the determined "energy troughs", as a result of which the audio signal is processed. It follows from this that the processing of the audio signal according to the method, as mentioned above, in particular by an at least partial replacement of missing, ie. H. z. B. in the context of data compression discarded, frequency components of the audio signal is realized.
  • the steps of the method described result in an improved method for processing a lossy compressed audio signal, in particular with regard to the efficiency of the processing and the quality of the processed audio signal.
  • an optional eighth step of the method it is possible to process the correspondingly prepared audio signal via at least one z. B. trained as a speaker device or at least one such comprehensive signal output device.
  • An optional eighth step of the method can then provide for the output of a processed audio signal via at least one signal output device.
  • the eighth step of the method it is possible to store the correspondingly processed audio signal in a memory device, ie. H. z. B. a hard disk space to store (between).
  • a correspondingly prepared, stored audio signal can be output at a later point in time via at least one corresponding signal output device and / or transmitted to at least one communication partner via a suitable, in particular wireless, communication network.
  • An optional eighth step of the method can therefore (also) provide for storing a processed audio signal in at least one storage device and / or transmitting a processed audio signal to at least one communication partner.
  • the processed audio signal can be subjected to an inverse Fourier transformation before it is output and / or stored and / or transmitted.
  • the third energy profile if any, proceeds from the frequency of the (in terms of frequency) lower amplitude maximum of the two immediately following amplitude maxima in the direction of the frequency of (in terms of frequency) the upper amplitude maximum of the two immediately following amplitude maxima.
  • the fourth energy curve if any, starts from the frequency of the (frequency-wise) upper (higher) amplitude maximum of the two immediately successive amplitude maxima in the direction of the frequency of the (frequency-wise) lower (lower) amplitude maximum of the two immediately successive amplitude maxima.
  • the energy profiles generated can in turn be transmitted in terms of data to the frequency spectrum.
  • a closed area or area is also defined by the frequencies and the energy profiles.
  • the range is in turn defined by the frequencies of the two immediately following amplitude maxima and in terms of energy by the energy profiles running between them.
  • the range typically only contains energy values ⁇ zero. If the area is viewed geometrically with respect to the frequency spectrum, the area in turn corresponds to the area defined geometrically by the two immediately adjacent amplitude maxima, the energy or frequency courses running between these and the frequency axis (x-axis) Area.
  • the generation of the third and fourth energy profiles is typically also carried out on the basis of a psychoacoustic model.
  • a psychoacoustic model is therefore typically also used to generate the energy profiles, or the energy profiles are derived from a psychoacoustic model.
  • the explanations in connection with the first two energy profiles apply analogously.
  • the third and fourth energy profiles are likewise derived in particular from the hearing thresholds of human hearing given by the respective psychoacoustic model at the respective preselected frequencies.
  • the possibly third energy profile corresponds to the part of the hearing threshold derived from the psychoacoustic model for the frequency of the lower amplitude maximum, which extends in the direction of increasing frequencies.
  • the possibly fourth energy curve corresponds to the part of the hearing threshold derived from the psychoacoustic model for the frequency of the upper amplitude maximum, which extends in the direction of falling frequencies.
  • the audio fill signal is subsequently introduced at least in sections, in particular completely, into the region of the frequency spectrum defined by the two preselected frequencies and the respective energy profiles.
  • the audio signal is processed here by introducing the audio fill signal into the frequency range of the frequency spectrum defined by the frequencies of the two immediately adjacent amplitude maxima and the respective energy profiles, so that the range of the frequency spectrum defined by the frequencies of the two immediately adjacent amplitude maxima and the respective energy profiles Frequency spectrum at least in sections, in particular completely, is or will be filled with the audio fill signal.
  • the audio fill signal can be generated as a function of or independently of acoustic parameters of the audio signal to be processed, in particular with regard to the respective energy and frequency components of the audio signal.
  • the audio fill signal is expediently generated independently of acoustic parameters of the audio signal, that is to say solely with regard to the at least partial filling of the range of the frequency spectrum defined by the frequencies of the two immediately adjacent amplitude maxima, since the computing effort for generating the audio fill signal may be such can be significantly reduced.
  • the filling or filling of the range of the frequency spectrum defined by the frequencies of the two immediately consecutive amplitude maxima can be carried out depending on certain acoustic parameters of the audio signal, in particular the amplitude and / or frequency curve. or certain acoustic parameters of a further audio signal to be processed, in particular of the amplitude and / or frequency curve. In this way, a more natural perception of the prepared audio signal for the human ear can be realized.
  • a Bark scale can be used as the frequency spectrum in which the audio signal is transmitted according to the method.
  • the 24 individual barks or bands of the Bark scale are known to correspond to the 24 individual frequency groups of human hearing, i.e. H. those frequency ranges that are jointly evaluated by human hearing.
  • the individual barks or bands of the Bark scale contain different frequencies or frequency ranges or bandwidths. Possible frequency bands of the frequency spectrum can correspond to the 24 barks or bands of the Bark scale.
  • the device comprises a control device equipped or communicating with appropriate devices.
  • the device can be part of an audio device or an audio system for a motor vehicle.
  • the invention further relates to an audio device or an audio system for a motor vehicle.
  • the audio device can be part of a motor vehicle-side multimedia device for outputting multimedia content, in particular audio and / or video content, to occupants of a motor vehicle.
  • the audio device comprises at least one signal output device, i. H. z. B. a loudspeaker device, which is set up for acoustic output of prepared audio signals in an at least part of a passenger compartment of an interior of a motor vehicle.
  • the audio device is characterized in that it has at least one device for processing lossy compressed audio signals as described for the preparation of lossy compressed audio signals.
  • Fig. 1 shows a schematic diagram of a device 1 for processing a lossy compressed audio signal 2.
  • the audio signal 2 may, for. B. is a lossy compressed audio file. Specifically, it can be e.g. B. can be a lossy compressed mp3-encoded audio file ("mp3 file") by means of an mp3 algorithm.
  • the audio file can already be at least partially decoded.
  • the audio file can e.g. B. include a piece of music.
  • the device 1 shown in the exemplary embodiment forms part of an audio device 3 or an audio system of a motor vehicle 4.
  • the audio device 3 can be part of a motor vehicle-side multimedia device (not shown) for outputting multimedia content, in particular audio and / or video content, to occupants of the motor vehicle 4.
  • the audio device 3 comprises at least one z. B. designed as a loudspeaker device or at least one such comprehensive signal output device 5, which is set up for acoustic output of prepared audio signals 6 in an at least part of the passenger compartment interior 7 of the motor vehicle 4.
  • the device 1 comprises a central hardware and / or software implemented control device 8, which is set up in the following with reference to Fig. 2 to implement the described method for processing lossy compressed audio signals 2.
  • Steps S1-S7 (S8) carried out according to the method can be carried out in separate hardware and / or software implemented devices (not shown) of the control device 8.
  • the device 1 comprises a control device 8 equipped with corresponding devices.
  • Fig. 2 shows a block diagram of an embodiment of a method for processing lossy compressed audio signals 2. The method can be carried out with the device 1 described above.
  • the lossy compressed audio signal 2 to be processed is provided.
  • the provision of the audio signal 2 can in principle be via any physical or non-physical audio source, i. H. z. B. from the audio device 3. Specifically, the audio signal 2 z. B. from a data memory (not shown) of the audio device 3.
  • the audio signal 2 is transmitted into a frequency spectrum.
  • energies of the audio signal 2 are correlated with frequencies of the audio signal 2.
  • the content of the audio signal 2 is examined for its energy, that is to say amplitude and frequency components, and the individual energy components of the audio signal 2 are transmitted in terms of data into a frequency-dependent representation by means of suitable algorithms, for example by means of (faster) Fourier transformation algorithms.
  • a corresponding frequency spectrum is included in Fig. 5 shown in a schematic diagram ..
  • frequencies f i of local amplitude maxima are determined in the frequency spectrum; the frequency spectrum is therefore examined for local amplitude maxima and the frequencies f i associated with the respective amplitude maxima are determined.
  • the frequencies f i associated with the respective amplitude maxima are determined.
  • Under one in the 5 - 8 by a dot graphically highlighted local amplitude maximum is to be understood as an amplitude maximum value in a defined frequency environment range.
  • a first selection criterion is defined. On the basis of the first selection criterion, the frequencies f i of two immediately following (local) amplitude maxima are preselected, which frequencies meet the first selection criterion. In the fourth step S4, the frequencies f i of pairs of immediately consecutive amplitude maxima are examined with regard to the first selection criterion to determine whether the frequencies f i meet the first selection criterion. In the further steps S5-S7 of the method, only the frequencies f i which meet the first selection criterion are considered. The fourth step S4 then preselects the frequencies f i to be considered below.
  • the first selection criterion describes a certain limit frequency value ⁇ f T.
  • Frequencies f i of immediately successive amplitude maxima satisfy the first selection criterion if their frequency difference ⁇ f i exceeds the limit frequency value ⁇ f T described by the first selection criterion, cf. the relationship represented by the following formula: ⁇ f i > ⁇ f T
  • ⁇ f i applies: frequency difference between two immediately following amplitude maxima; ⁇ f T : cutoff frequency value.
  • the limit frequency value ⁇ f T is determined by transferring the preselected frequencies f i to a Bark scale.
  • Both the preselected frequencies f i and the limit frequency values ⁇ f T described by the first selection criterion can be transferred to the Bark scale using the relationship represented by the above formula.
  • the limit frequency value ⁇ f T can correspond to a bark or to a bark adjusted by an adaptation factor or multiplied by an adaptation factor.
  • the adjustment factor is typically between 0.7 and 1.1, in particular 0.9.
  • the limit frequency value thus typically corresponds to 0.7 to 1.1, in particular 0.9, Bark.
  • a second selection criterion is defined in the fifth step S5 of the method.
  • preselected frequencies f i are selected which (additionally) meet the second selection criterion.
  • preselected frequencies f i are then examined to determine whether they (additionally) meet the second selection criterion.
  • the frequencies f i (additionally) satisfying the second selection criterion can in turn be transferred to a Bark scale.
  • the second selection criterion can describe a limit energy value. Respective preselected frequencies f i satisfy the second selection criterion if the amount of energy between them falls below the limit energy value described by the second selection criterion.
  • the limit energy value can be defined by a defined limit energy content T.
  • Respective preselected frequencies f i meet the second selection criterion if they fall below the limit energy content T described by the second selection criterion, cf. the relationship represented by the following formula: ⁇ f 1 f 2 S f 2 df ⁇ T
  • S (f) the area described by the frequencies f 1 , f 2 of the two immediately consecutive amplitudes (energy content between the frequencies f 1 , f 2 of the two immediately consecutive amplitude maxima); T: Limit energy content.
  • FIG. 6 Schematic representation of a frequency spectrum containing two preselected frequencies f 1 , f 2 , which is also a section of another, namely the one in Fig. 5 shown frequency spectrum is to refer.
  • Out Fig. 6 the area described by the frequencies f 1 , f 2 of the two immediately successive amplitude maxima (hatched) and the limit energy content T represented by a horizontal line are illustrated.
  • the hatched area corresponds to the integral represented by the above formula.
  • the limit energy value can alternatively also be determined by a first energy curve EV1 starting from the preselected frequency f 1 ("lower frequency"), which belongs to the lower (lower in frequency) amplitude maximum, and a first energy curve EV1 from the preselected frequency f 2 ("upper frequency ), which is associated with the upper (higher frequency) amplitude maximum, the outgoing second energy curve EV2 is generated and the two energy curves EV1, EV2 are transmitted into the frequency spectrum Energy profiles EV1, EV2 defined.
  • first energy curve EV1 starting from the preselected frequency f 1
  • upper frequency which is associated with the upper (higher frequency) amplitude maximum
  • the generated energy profiles EV1, EV2 can be transmitted in terms of data into the frequency spectrum.
  • the first energy curve EV1 proceeds from the lower frequency f 1 in the direction of the upper frequency f 2 .
  • the second energy curve EV2 proceeds from the upper frequency f 2 in the direction of the lower frequency f 1 .
  • a closed area or area is defined by the actual frequency curve between the frequencies f 1, 2 and the energy curves EV1, EV2.
  • the range is defined in terms of frequency share by the two frequencies f 1, 2 and in terms of energy share by the actual frequency curve and the energy curves EV1, EV2 running between them.
  • the range typically only contains energy values ⁇ zero. If one looks at the area geometrically with respect to the frequency spectrum, the area corresponds to the geometrically defined amplitude or maxima defined by the frequencies f 1, 2 of the two immediately adjacent amplitude maxima, and the frequency axis (x-axis) between them Fig. 7 hatched area.
  • the energy profiles EV1, EV2 are generated on the basis of a psychoacoustic model.
  • a preferred used psychoacoustic model is the spectral masking or masking model.
  • the energy profiles EV1, EV2 are derived from the hearing thresholds of the human ear given the respective preselected frequencies f 1, 2 by the respective psychoacoustic model. This means that the psychoacoustic model used is applied to the two frequencies f 1, 2 .
  • the first energy curve EV1 corresponds to the part of the hearing threshold derived from the psychoacoustic model for the lower frequency f 1 , which extends in the direction of increasing frequencies (see left curly bracket in Fig. 3 ).
  • the second energy curve EV2 corresponds to the part of the hearing threshold derived from the psychoacoustic model for the upper frequency f 2 , which extends in the direction of falling frequencies (cf. right-hand curly bracket in Fig. 3 ).
  • the energy profiles EV1, EV2 intersect or intersect in a value range above the x-axis.
  • a suitable signal generator is used generates an audio fill signal AFS.
  • the audio fill signal AFS is generated in a targeted manner with regard to the previously determined frequency ranges to be processed within the audio signal 2 to be processed.
  • the audio fill signal AFS is thus generated in a targeted manner with regard to the frequency range defined by the frequencies f i and f 1, 2 of the two immediately successive amplitude maxima, which satisfy both the first and the second selection criterion, in order to fill this and that between the frequencies f i fill given "energy valley".
  • the generated audio fill signal AFS therefore has a frequency range lying between the frequencies f i of the respective directly successive amplitude maxima.
  • the audio fill signal AFS can be generated as a function of or independently of acoustic parameters of the audio signal 2, in particular with regard to the respective energy and frequency components of the audio signal 2.
  • the audio filling signal AFS becomes independent of acoustic parameters of the audio signal 2, ie solely with regard to the filling of the frequency component with frequencies f 1, 2 and energy component with the actual frequency profile and the range defined between these energy profiles EV3, EV4. generated.
  • a seventh step S7 of the method the actual processing of the audio signal 2 takes place by introducing the audio fill signal AFS into respective frequency ranges between respective frequencies f i satisfying the first and second selection criteria, so that a respective frequency range is filled with the audio fill signal AFS.
  • a further or third energy curve EV3 starting from the selected lower (lower) frequency f 1 , which is associated with the lower (lower in frequency) amplitude, and one from the selected upper (higher ) Frequency f 2 , which is associated with the upper (higher frequency) amplitude maximum, generates outgoing further or fourth energy curve EV4.
  • Fig. 8 it can be seen that the generated energy profiles EV3, EV4 - analogously to the energy profiles EV1, EV2 - are transmitted to the frequency spectrum in terms of data.
  • the third energy curve EV3 proceeds from the lower frequency f 1 in the direction of the upper frequency f 2 .
  • the fourth energy curve EV4 proceeds from the upper frequency f 2 in the direction of the lower frequency f 1 .
  • a closed area or area is defined by the actual frequency curve between the frequencies f 1, 2 and the energy curves EV3, EV4.
  • the range is defined in terms of frequency share by the frequencies f 1, 2 of the amplitude maxima and in terms of energy share by the actual frequency curve and the energy curves EV3, EV4 running between them.
  • the range typically only contains energy values ⁇ zero. Looking at the area geometrically with respect to the frequency spectrum, corresponds to the range of the geometrically defined energy or frequency profiles and the frequency axis (x-axis) defined by the frequencies f 1, 2 of the two immediately adjacent amplitude maxima, in Fig. 8 hatched area.
  • the energy profiles EV3, EV4 are also generated on the basis of a psychoacoustic model.
  • a preferred used psychoacoustic model is the model of spectral masking or masking (cf. Fig. 4 ).
  • the energy profiles EV3, EV4 are derived from the hearing thresholds of the human ear given the respective preselected frequencies f 1, 2 by the respective psychoacoustic model.
  • this means that the psychoacoustic model used is applied to the two immediately successive frequencies f 1, 2 .
  • the third energy curve EV3 corresponds to the part of the hearing threshold derived from the psychoacoustic model for the lower frequency f 1 , which extends in the direction of increasing frequencies (see left brace in Fig.
  • the fourth energy curve EV4 corresponds to the part of the hearing threshold derived from the psychoacoustic model for the upper frequency f 2 , which extends in the direction of falling frequencies (see right curly bracket in Fig. 4 ).
  • the energy profiles EV3, EV4 intersect or intersect in a range of values above the x-axis.
  • the (first two) energy profiles EV1, EV2 can differ from the third and fourth energy profiles Ev3, EV4.
  • "energy valleys” resulting from the data compression of the audio signal 2 are determined and specifically filled with a certain data content in the form of the audio fill signal AFS generated with regard to the determined “energy valleys", whereby the audio signal 2 is processed. It follows from this that the processing of the audio signal 2 according to the method by an at least partial replacement of missing, ie. H. z. B. in the context of data compression discarded, frequency components of the audio signal 2 is realized.
  • An optional eighth step S8 of the method can output a processed audio signal 2 via at least one signal output device 5 and / or store a processed audio signal 2 in at least one storage device (not shown) and / or transmit a processed audio signal 2 to at least one communication partner (not shown).
  • the processed audio signal 2 can be subjected to an inverse Fourier transformation before it is output and / or stored and / or transmitted.
  • the described steps S1-S7 (S8) of the method result in an improved method for processing a lossy compressed audio signal 2, in particular with regard to the efficiency of the processing and the quality of the processed audio signal 6.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Circuit For Audible Band Transducer (AREA)

Claims (15)

  1. Procédé servant à préparer un signal audio (2) comprimé avec perte, caractérisé par des étapes suivantes :
    - de fourniture d'un signal audio (2) comprimé avec perte, où il s'agit d'un fichier audio comprimé avec perte, déjà décodé,
    - de transfert du signal audio (2) dans un spectre de fréquences, dans lequel des énergies du signal audio (2) sont mises en corrélation avec des fréquences du signal audio (2),
    - de détermination des fréquences (fi) de crêtes d'amplitude locales dans le spectre de fréquences,
    - de fixation d'un premier critère de sélection et de présélection des fréquences (fi) de deux crêtes d'amplitude locales se suivant l'une l'autre directement, lesquelles fréquences répondent au premier critère de sélection,
    - de fixation d'un deuxième critère de sélection et de sélection de fréquence (fi) présélectionnées, répondant au premier critère de sélection, de deux crêtes d'amplitude locales se suivant l'une l'autre directement, lesquelles fréquences répondent en plus au deuxième critère de sélection,
    - de production d'un signal audio de remplissage (AFS), et
    - de préparation du signal audio (2) en introduisant le signal audio de remplissage (AFS) dans une plage de fréquences entre les fréquences (fi) répondant au deuxième critère de sélection si bien que la plage de fréquences est remplie au moins par endroits, en particulier totalement, avec le signal audio de remplissage (AFS).
  2. Procédé selon la revendication 1, caractérisé en ce que les fréquences (fi) répondent au premier critère de sélection quand leur différence de fréquence dépasse en valeur absolue une valeur de fréquence limite (Δfi) décrite par le premier critère de sélection.
  3. Procédé selon la revendication 2, caractérisé en ce que la valeur de fréquence limite (Δfi) est fixée par transfert des fréquences (fi) dans une échelle de Bark, dans lequel la valeur de fréquence limite (Δfi) correspond à un bark ou à un bark adapté par l'intermédiaire d'un facteur d'adaptation.
  4. Procédé selon la revendication 3, caractérisé en ce que le facteur d'adaptation utilisé correspond à une valeur entre 0,7 et 1,1 bark, en particulier 0,9 bark.
  5. Procédé selon l'une quelconque des revendications précédentes, caractérisé en ce que les fréquences (fi) répondent au deuxième critère de sélection quand le contenu en énergie entre les fréquences (fi) présente une valeur inférieure en valeur absolue à une valeur d'énergie limite.
  6. Procédé selon la revendication 5, caractérisé en ce que la valeur d'énergie limite est définie par un contenu en énergie limite (T) fixé.
  7. Procédé selon la revendication 5, caractérisé en ce que la valeur d'énergie limite est fixée en ce qu'une première courbe énergétique (EV1) partant de la fréquence inférieure (f1) sélectionnée et une courbe énergétique (EV2) partant de la fréquence supérieure (f2) sélectionnée sont produites et les deux courbes énergétiques (EV1, EV2) sont transférées dans le spectre de fréquences, dans lequel la valeur d'énergie limite est définie par les courbes énergétiques (EV1, EV2) respectives.
  8. Procédé selon la revendication 7, caractérisé en ce que la production de la première et de la deuxième courbe énergétique (EV1, EV2) est effectuée sur la base d'un modèle psychoacoustique.
  9. Procédé selon l'une quelconque des revendications précédentes, caractérisé en ce qu'avant le traitement du signal audio (2) en introduisant le signal audio de remplissage (AFS) dans la plage de fréquences entre les fréquences (fi) répondant au deuxième critère de sélection, de sorte que la plage de fréquences est remplie au moins par endroits, en particulier totalement, avec le signal audio de remplissage (AFS),
    une troisième courbe énergétique (EV3) éventuellement partant de la fréquence inférieure (f1) sélectionnée et une quatrième courbe énergétique (EV4) éventuellement partant de la fréquence supérieure (f2) sélectionnée sont produites et les deux courbes énergétiques (EV3, EV4) sont transférées dans le spectre de fréquences.
  10. Procédé selon la revendication 9, caractérisé en ce que le signal audio de remplissage (AFS) est introduit au moins par endroits, en particulier totalement, dans une plage, définie par les deux fréquences (f1, f2) sélectionnées et les courbes énergétiques (EV3, EV4) respectives, du spectre de fréquences.
  11. Procédé selon la revendication 9 ou 10, caractérisé en ce que la production des courbes énergétiques (EV3, EV4) est effectuée sur la base d'un modèle psychoacoustique.
  12. Procédé selon l'une quelconque des revendications précédentes, caractérisé en ce que le signal audio de remplissage (AFS) est produit en fonction ou indépendamment de paramètres acoustiques du signal audio (2).
  13. Procédé selon la revendication 12, caractérisé en ce que le signal audio de remplissage (AFS) est produit en fonction de paramètres acoustiques du signal audio (2), dans lequel le remplissage de la plage (A) est effectué en fonction de paramètres acoustiques donnés du signal audio (2) ou d'un autre signal audio (2) à traiter.
  14. Dispositif (1) servant à traiter un signal audio (2) comprimé avec perte selon un procédé selon l'une quelconque des revendications précédentes, caractérisé par au moins un système de commande (8), lequel est mis au point pour
    - fournir un signal audio (2) comprimé avec perte,
    - transférer le signal audio (2) dans un spectre de fréquences, dans lequel des énergies du signal audio (2) sont mises en corrélation avec des fréquences du signal audio (2),
    - déterminer des fréquences (fi) de crêtes d'amplitude locales dans le spectre de fréquences,
    - fixer un premier critère de sélection et présélectionner les fréquences (fi) de deux crêtes d'amplitude locales se suivant l'une l'autre directement, lesquelles fréquences répondent au premier critère de sélection,
    - fixer un deuxième critère de sélection et sélectionner des fréquences (fi) présélectionnées répondant au premier critère de sélection de deux crêtes d'amplitude locales se suivant l'une l'autre directement, lesquelles fréquences répondent en plus au deuxième critère de sélection,
    - produire un signal audio de remplissage (AFS), et
    - traiter le signal audio (2) en introduisant le signal audio de remplissage (AFS) dans une plage entre les fréquences (fi) répondant au deuxième critère de sélection si bien que la plage est remplie au moins par endroits, en particulier totalement, avec le signal audio de remplissage (AFS).
  15. Système audio (3) pour un véhicule automobile (4), comprenant au moins un système d'émission de signaux (5), lequel est mis au point pour l'émission acoustique de signaux audio (6) traités dans un espace intérieur (7), formant au moins une partie d'un habitacle, d'un véhicule automobile (4), caractérisé en ce que le système audio servant à traiter des signaux audio (2) comprimés avec perte présente au moins un dispositif (1) selon la revendication 14.
EP17711600.1A 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte Active EP3403260B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102016104665.5A DE102016104665A1 (de) 2016-03-14 2016-03-14 Verfahren und Vorrichtung zur Aufbereitung eines verlustbehaftet komprimierten Audiosignals
PCT/EP2017/055820 WO2017157841A1 (fr) 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte

Publications (2)

Publication Number Publication Date
EP3403260A1 EP3403260A1 (fr) 2018-11-21
EP3403260B1 true EP3403260B1 (fr) 2020-03-04

Family

ID=58358566

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17711600.1A Active EP3403260B1 (fr) 2016-03-14 2017-03-13 Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte

Country Status (5)

Country Link
US (1) US10734000B2 (fr)
EP (1) EP3403260B1 (fr)
CN (1) CN108174614B (fr)
DE (1) DE102016104665A1 (fr)
WO (1) WO2017157841A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110491407B (zh) * 2019-08-15 2021-09-21 广州方硅信息技术有限公司 语音降噪的方法、装置、电子设备及存储介质
CN113192519B (zh) * 2021-04-29 2023-05-23 北京达佳互联信息技术有限公司 音频编码方法和装置以及音频解码方法和装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5479560A (en) 1992-10-30 1995-12-26 Technology Research Association Of Medical And Welfare Apparatus Formant detecting device and speech processing apparatus
DE10103134A1 (de) * 2001-01-24 2002-08-08 Harman Becker Automotive Sys Dekodiereinrichtung, Dekodierverfahren und Kraftfahrzeugaudiosystem mit einer derartigen Dekodiereinrichtung
EP1423847B1 (fr) * 2001-11-29 2005-02-02 Coding Technologies AB Reconstruction des hautes frequences
US7447631B2 (en) * 2002-06-17 2008-11-04 Dolby Laboratories Licensing Corporation Audio coding system using spectral hole filling
DE50312002D1 (de) * 2003-07-24 2009-11-19 Palm Inc Verfahren und Vorrichtung zum Entzerren eines mit einem äusseren Störsignal behafteten Audiosignals
US7562021B2 (en) * 2005-07-15 2009-07-14 Microsoft Corporation Modification of codewords in dictionary used for efficient coding of digital media spectral data
PL3779977T3 (pl) * 2010-04-13 2023-11-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dekoder audio do przetwarzania audio stereo z wykorzystaniem zmiennego kierunku predykcji
JP2013148724A (ja) 2012-01-19 2013-08-01 Sony Corp 雑音抑圧装置、雑音抑圧方法およびプログラム
JP5949270B2 (ja) * 2012-07-24 2016-07-06 富士通株式会社 オーディオ復号装置、オーディオ復号方法、オーディオ復号用コンピュータプログラム
KR101757347B1 (ko) * 2013-01-29 2017-07-26 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에.베. 지각적 변환 오디오 코딩에서의 노이즈 채움
EP2959479B1 (fr) * 2013-02-21 2019-07-03 Dolby International AB Procédés d'encodage multicanal paramétrique
EP2830061A1 (fr) * 2013-07-22 2015-01-28 Fraunhofer Gesellschaft zur Förderung der angewandten Forschung e.V. Appareil et procédé permettant de coder et de décoder un signal audio codé au moyen de mise en forme de bruit/ patch temporel
EP2830060A1 (fr) * 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Remplissage de bruit de codage audio multicanal
KR20230162157A (ko) * 2014-06-27 2023-11-28 돌비 인터네셔널 에이비 Hoa 데이터 프레임 표현의 데이터 프레임들 중 특정 데이터 프레임들의 채널 신호들과 연관된 비차분 이득 값들을 포함하는 코딩된 hoa 데이터 프레임 표현
EP2960903A1 (fr) * 2014-06-27 2015-12-30 Thomson Licensing Procédé et appareil de détermination de la compression d'une représentation d'une trame de données HOA du plus petit nombre entier de bits nécessaires pour représenter des valeurs de gain non différentielles
CN117636885A (zh) * 2014-06-27 2024-03-01 杜比国际公司 用于解码声音或声场的高阶高保真度立体声响复制（hoa）表示的方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
US20190080702A1 (en) 2019-03-14
EP3403260A1 (fr) 2018-11-21
US10734000B2 (en) 2020-08-04
CN108174614B (zh) 2018-12-28
WO2017157841A1 (fr) 2017-09-21
DE102016104665A1 (de) 2017-09-14
CN108174614A (zh) 2018-06-15

Similar Documents

Publication Publication Date Title
DE69122648T2 (de) Digitale Teilbandkodierungsvorrichtung
EP0661905B1 (fr) Procédé d'adaptation de prothèse auditive, dispositif à cet effet et prothèse auditive
DE112016006218B4 (de) Schallsignal-Verbesserungsvorrichtung
EP1953739B1 (fr) Procédé et dispositif pour réduire le bruit dans un signal décodé
EP1386307B1 (fr) Procede et dispositif pour determiner un niveau de qualite d'un signal audio
EP3197181B1 (fr) Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif
EP2919652B1 (fr) Traitement de signaux audio pour la thérapie des acouphènes
DE102010039589A1 (de) Hörhilfe- und/oder Tinnitus-Therapie-Gerät
DE102006047983A1 (de) Verarbeitung eines Eingangssignals in einem Hörgerät
DE102017222750A1 (de) Regel- oder Steuervorrichtung und Verfahren zur Verbesserung einer Geräuschqualität eines Klimatisierungssystems
EP3403260B1 (fr) Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte
EP1351550B1 (fr) Procédé d'adaptation d'une amplification de signal dans une prothèse auditive et prothèse auditive
EP1212751B1 (fr) Procede d'attenuation de bruits parasites dans un champ de signal
EP3232691B1 (fr) Procédé de détermination de la réponse d'impulsion spatiale d'un habitacle d'un véhicule automobile
DE102009030551A1 (de) Verfahren zum lautheitsbasierten Einstellen der Verstärkung eines Hörgeräts und zugehöriges Hörgerät
EP3232684B1 (fr) Procédé de transmission d'un signal audio provenant d'un émetteur à un récepteur
EP1130577B1 (fr) Procédé de reconstruction des basses fréquences du signal de parole à partir de fréquences moyennes
DE102018207530A1 (de) Vorrichtung und Verfahren für Verbesserung der Privatsphäre
EP3217392B1 (fr) Audibilité de signaux ultrasonores à large bande
DE102023107308B3 (de) Psychoakustische Kalibrierung eines Audiowiedergabesystems
EP2024022B1 (fr) Dispositif et procédé pour générer un modèle d'activité filtré, séparateur de sources, procédé pour générer un signal audio épuré, et programme informatique
WO2022175349A2 (fr) Procédé et agencement de traitement de signal
EP3965434A1 (fr) Procédé de sonorisation améliorée de plusieurs zones de sonorisation
EP3834723A1 (fr) Procédé de détermination du seuil auditif d'un sujet humain
DE19626329A1 (de) Verfahren zur Analyse von Schallereignissen

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180606

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190717

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191122

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1241292

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502017004091

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200604

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200604

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200605

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200704

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502017004091

Country of ref document: DE

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200313

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200313

26N No opposition filed

Effective date: 20201207

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200504

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200304

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1241292

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220313

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220313

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240326

Year of fee payment: 8