EP2402943B1 - Dispositif et procédé de production d'un signal d'environnement - Google Patents

Dispositif et procédé de production d'un signal d'environnement Download PDF

Info

Publication number
EP2402943B1
EP2402943B1 EP11182965.1A EP11182965A EP2402943B1 EP 2402943 B1 EP2402943 B1 EP 2402943B1 EP 11182965 A EP11182965 A EP 11182965A EP 2402943 B1 EP2402943 B1 EP 2402943B1
Authority
EP
European Patent Office
Prior art keywords
signal
block
transient
synthesis
examination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP11182965.1A
Other languages
German (de)
English (en)
Other versions
EP2402943A2 (fr
EP2402943A3 (fr
Inventor
Oliver Hellmuth
Jürgen HERRE
Stephan Geyersberger
Andreas Walther
Christiaan Janssen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP2402943A2 publication Critical patent/EP2402943A2/fr
Publication of EP2402943A3 publication Critical patent/EP2402943A3/fr
Application granted granted Critical
Publication of EP2402943B1 publication Critical patent/EP2402943B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 
    • H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation  of the pseudo five- or more-channel type, e.g. virtual surround
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments

Definitions

  • Multi-channel audio is becoming more and more popular.
  • Such playback systems generally consist of three speakers L (left), C (center) and R (right), which are typically located in front of the user, and two speakers Ls and Rs located behind the user, and typically one of them LFE channel, which is also called low-frequency effect channel or subwoofer.
  • LFE channel which is also called low-frequency effect channel or subwoofer.
  • Such a channel scenario is in Fig. 10 and in Fig. 11 indicated. While the positioning of the speakers L, C, R, Ls, Rs, with respect to the user as in the FIGS.
  • Such a multi-channel system provides several advantages over a typical stereo reproduction, which is a two-channel reproduction, such as in Fig. 9 is shown.
  • Fig. 11 Another alternative concept, called in-the-band concept, is in Fig. 11 shown schematically.
  • Each type of sound ie direct sound sources and ambient sounds, are all positioned around the listener.
  • the position of a sound is independent of its characteristics (direct sound sources or ambient sounds) and depends only on the specific design of the algorithm, as described in eg Fig. 11 is shown. So was in Fig. 11 the upmix algorithm determines that the two instruments 1100 and 1102 are positioned laterally relative to the listener while the two instruments 1104 and 1106 are positioned in front of the user.
  • the two rear speakers Ls, Rs now also contain portions of the two instruments 1100 and 1102 and no longer just ambient sounds, as in Fig. 10 was still the case where the same instruments were all positioned in front of the user.
  • the extraction or partial extraction and partial synthesis of such ambient signals is one risky affair, as it would be annoying for a user to include information from sound sources in the surround channels which the user identifies as coming from the front, ie from the left channel, middle channel and right channel. For this reason, generating ambient signals would be very "defensive” to make sure that no user-perceived artifacts are generated.
  • the other extreme case if one acts too defensively in the generation of the ambient signals, is that a very quiet, or barely perceptible ambient signal is extracted, or that the ambient signal has only noise, but no longer has any special information, so that the ambient signal contributes very little to the enjoyment of listening and could actually be left out altogether in this case.
  • the U.S. Patent No. 4,076,969A discloses an impulse noise reduction system for detecting unwanted pop and click sound signals generated by mechanical defects on a recording medium.
  • a few of fully or partially correlated signals are generated from the recorded sound information by means of a stereo recording device.
  • the output signals of the stereo recording apparatus are combined so that desired sound signals generated by horizontal movement on the recording medium are subtracted and undesired signals generated by vertical movement on the recording medium are added.
  • the resulting unwanted signal is detected as a noise signal.
  • a control signal that starts before the leading edge of the detected unwanted sound signal and that has a width that is greater than the width of the detected unwanted sound signal is used in the system to eliminate the unwanted signals.
  • the specialist publication " Restoration of Historical Recordings by Means of Digital Signal Processing” refers to improving the overall quality of the records that have been degraded by multiple glitches.
  • impulsive noises generated by scratches and cracks in the splitting material are to be eliminated.
  • the continuous broadband surface noise is to be reduced.
  • the present invention is based on the recognition that the artifacts that are most negatively perceived by listeners in ambient signals are artifacts that cause the listener to think that a direct sound source is in the rear speaker, although he or she is Sound source perceives from the front.
  • Characteristics for the perception of direct sound sources are transient processes, ie signal fine structures in the time signal, which concern a (fast) change over a change threshold from a quiet state to a loud state or from a loud state to a quiet state, or one (Strong) energy increase above a threshold of change in special bands and especially in the upper bands within a certain time.
  • a transient period is thus detected in an examination signal, in which a transient region is present in the examination signal. Then, with the help of a synthesis signal generator, a synthesis signal generated for the transient period, which is configured to generate the synthesis signal to have a shallower time course than the examination signal in the transient region, wherein the synthesis signal generator is further configured to generate the synthesis signal so that it to its intensity of a preceding or succeeding portion of the examination signal by less than a predetermined threshold. This generated synthesis signal is then used by a signal substitutor instead of the examination signal in the transient period to obtain the ambient signal.
  • the extraction of an ambient signal-like signal from a two-channel stereo input signal is improved, or a post-processing of an existing signal, e.g. already an extracted raw ambient signal is made.
  • the examination signal is the actual two-channel stereo signal or respectively one channel of the two-channel signal, while in the second case the examination signal is already an extracted environment signal or a presynthesized environment signal.
  • the inventive concept is particularly useful for the upmix concept, which has also been presented as a "direct ambience concept".
  • the concept according to the invention can also be advantageous for the "in-the-band" concept, since it also leads to an improved environmental signal which, on the one hand, no longer has any interfering artifacts, but on the other hand still contains enough information for a user to benefit from the ambient signal.
  • the inventive concept ensures that the surround signal is a continuous, uninterrupted, diffused sound signal, since intermittent ambient sound, which is obtained, for example, if transients were simply completely eliminated, would be considered uncomfortable by the user or even errors in the high-mix process would be perceived.
  • an ambience-like signal for the rear channels is extracted from the stereo signal.
  • the difference between the original right and left channels is simply used.
  • the back channels are created in this way, they often have transient-like components of direct sound sources.
  • These transients may be tones, such as, for example, musical beginnings or parts of percussive instruments.
  • a transient perceived behind the listener while a direct sound source (to which the transient typically belongs) is positioned in front of the listener has a negative impact on the localization of the direct sound source.
  • the direct sound source thus appears either wider than the original or, even more damaging, perceived as an independent direct sound source behind the user, both effects being very undesirable, especially for the direct ambience concept.
  • this problem is addressed by suppressing transients in the ambient-like signal, and minimizing the effect of this suppression on the remaining signal, ie, preserving the continuity of the signal by providing only limited intensity variations be admitted for the transient period.
  • the signal generated for the transient period, before being used by the signal substituter is mixed with the signal originally present in the transient period, which is achieved, for example, by overlapping processing.
  • a cross-fading may be performed to slowly fade in a cross-fade range from the signal before the transient period to the signal in the transient period, or slowly out of the transient period hide.
  • the blanking out of the transient period into the original signal when no more transient is detected is preferred for an artifact-free listening impression, since it is intended to ensure that when there is a non-artifact-related examination signal, the transition from the synthesis signal to the original one Examination signal no cracking or something similar arises.
  • a manipulation of the signal in the transient period is performed in the frequency domain by randomizing signs of spectral values or, more generally, phases of spectral values, which inevitably results in a smoothing of the temporal fine structure of this frequency domain manipulated signal.
  • Another spectral processing is to perform a prediction on the frequency of the spectral values and then use the prediction spectral values as spectral values of the synthesis signal, since the prediction over the frequency leads to a smoothing of the corresponding time signal.
  • the intensity of the transient period is to limit the change of spectral values from one block to another, this limitation being global , ie for all spectral values equal or selective, ie only for certain spectral values, which have a particularly large change, can take place.
  • Fig. 1 shows an inventive device for generating an ambient signal 10, which is suitable for broadcasting via speakers for which no separate speaker signal has been transmitted.
  • Such speakers are typically the rear speakers or surround speakers, as in Ls, Rs in 10 and FIG. 11 For example, shown.
  • the device shown comprises a transient detector 11 for detecting a transient period (at 20 in Fig. 2 shown) in which an examination signal has a transient region.
  • a transient detector 11 for detecting a transient period (at 20 in Fig. 2 shown) in which an examination signal has a transient region.
  • any other methods for transient detection can be used, such as those found in an MPEG4 audio encoder, in which is switched depending on a transient detection of short to long windows.
  • transient detectors are used which can detect fast and strong changes in the envelope of a time signal.
  • Exemplary magnitudes to be detected are changes in the envelope that affect changes equal to or greater than 100% of the amplitude of the envelope over a period of 1 ms.
  • the transient detector 11 is coupled to a synthesis signal generator 12, which is designed to generate a synthesis signal 13 which satisfies the two conditions, on the one hand the transient condition and, on the other hand, the continuity condition.
  • the transient condition is that the synthesis signal has a shallower time course than the examination signal in the transient region
  • the continuity condition is that the intensity of the synthesis signal in the transient region is less than a preset one of an intensity of a preceding or succeeding portion of the examination signal Threshold deviates.
  • the threshold can also be realized by a confidence interval of 80% or less, which is determined based on the historical values.
  • Intensity measures that can be used for the present invention include the energy obtained by adding the sample squares or spectral value squares of a block, or a measure of performance that can be obtained considering the temporal block length, or a measure of weighting or unweighted adding up the magnitude of spectral values in a band, this particular measure, which is also an intensity, also being referred to as high frequency content if the band in which it is added is the upper frequency band of the examination signal, or generally higher frequencies versus lower Frequencies are more heavily weighted or have a greater impact on the end result.
  • the synthesis signal generator then generates a synthesis signal that is used by a signal substituter 14 to use the synthesis signal instead of the corresponding portion of the original examination signal to finally provide the ambient signal 10.
  • the signal substitute 14 thus receives in addition to the synthesis signal via the line 13 and the examination signal via a line 15, as in Fig. 1 is indicated.
  • the transient detector 11 receives the examination signal via an input line 16 and provides via an output line 17 a transient information to the synthesis signal generator 12, so that this generates the synthesis signal using the examination signal, which is provided to him via a line 18.
  • non-overlapping block processing as in FIG Fig. 2a represented or an overlapping block processing as in Fig. 2b shown used.
  • an examination signal 21 is divided into preferably equal blocks of a specific block length.
  • the transient detector detects a transient 22 in the transient period 20.
  • the transient 22 is thus in the transient period 20 of FIG Fig. 2a , which causes the transient detector 11 to provide an output signal via its output line 17, which tells the synthesis signal generator 12 that it now has to start with signal synthesis.
  • the block of the exam signal corresponding to the transient period 20 is now synthesized by the synthesis signal generator and then substituted by the signal substituter 14 of the original block of the examination signal in the ambient signal.
  • a processing of the block of the examination signal is performed, which takes place in the frequency domain.
  • the synthesis signal has a sample which may differ significantly from a sample which is the last sample of the preceding block in the examination signal.
  • it is used in the art Fig. 2a it is preferred to superimpose a block before a transient period into the synthesis signal in the transient period, for example by adding the first sample of the generated synthesis signal to the eg last 10 samples of the previous block weighted according to the suppression function, for example according to FIG Fade in Fig.
  • the last sample of the previous block is still in accordance with the blanking in Fig. 3 the first and the first sample samples of the synthesized block, respectively, weighted according to the fade-in function are added in the transient period to provide a fade.
  • the rear Crossfade area so if the transient period back into the non-transient-afflicted block of the ambient signal, proceed.
  • FIG. Fig. 2b To further reduce such block boundary artifacts, as shown in FIG Fig. 2b is shown, overlapping processing is preferred.
  • the transient detector then detects at the in Fig. 2b shown embodiment block areas, which are shown with ringed numbers (1), (2), (3), (4), (5), (6).
  • a transient is detected at 22. This causes it to be compared to Fig. 2a gives a larger transient period 20 since the transient at position 22 has been detected in both block 4 and block 5. Therefore, the synthesis signal generator 12 of Fig. 1 Generate synthesis signals for both block 4 and block 5.
  • the regions A, B, C are signaled by the signal substituter 14 of FIG Fig. 1
  • the section A is generated by the addition of the second half of the non-transient-related block 3 of the examination signal with the first half of the synthesis signal generated for the block 4.
  • the second part B of the transient period 20 is provided by adding the second half of the synthesis signal generated for block 4 to the first half of the synthesis signal generated for block 5 and from the signal substituter as a corresponding portion of the environmental signal 10 substituted.
  • the third portion C of the transient period 20 is generated by adding the second half of the synthesizer signal generator generated block # 5 to the first half of the block # 6, which is no longer transient, and written into the ambient signal by the signal substituter 14.
  • this skip function can be used to provide soft block transition from a non-synthesized block to a synthesized block in block processing with non-overlapping blocks, and also to provide a smooth transition from a synthesized block back to a non-synthesized block .
  • a corresponding cross-fade function can also be used to overshadow back to the original examination signal, in particular when a synthesis signal has been generated by a specific specified number of blocks. Given the likelihood that the synthesis signal has moved relatively far from the probe due to the extrapolation, an abrupt return to the probe would, in some cases, result in audible artifacts.
  • a synthesis signal is generated, which consists of 90% of the last synthesized block and 10% of the current block of investigation. In the next block, the ratio could then be changed to 80%: 20%, until then after a certain number of blocks, the synthesis signal is completely hidden and the current non-transient-related examination signal is fully displayed again.
  • the time signal which represents a block of the examination signal, is converted into a frequency domain representation or a subband representation by a converter 40, which may comprise a transformation or an analysis filter bank.
  • the spectral representation in the form of spectral coefficients or the subband signals are then optionally information, as shown at 41 replaced by an extrapolated spectral representation or extrapolated subband signals, if it is a block of the time signal in which a transient has been detected.
  • the spectral representation is then optionally supplied, using additional information due to extrapolation, to a smoother 42, which influences the spectral values in such a way that the temporal course of the underlying signal is smoothed.
  • the smoother 42 will affect the subband signals so that the timing of the signal underlying the subband signals is smoother than before smoothing.
  • an inverse conversion into the time domain takes place using either an inverse transformation or a synthesis filterbank to finally arrive at a timing signal 44 which is smoother than the timing signal at the input of the stage 40, however, has energy that has not been significantly affected by smoothing.
  • the smoothing has been done so that the energy of the smoothed time signal 44 does not differ from the energy of the previous time signal any more than the threshold.
  • an overall e-energy manipulation of the energy of the time signal may occur.
  • the transient is attenuated while the tonal components continue to be synthesized or past, by synthesizing the signal in the transient period by a prediction using a non-transient signal from the past.
  • the smoothing has caused the energy over the block to be more evenly distributed, thus producing a smoother timing, but without losing the block's energy Sampling of the examination signal considerably to change. This is sufficient in most cases and ensures that the user hears a test signal that always satisfies the continuity condition. Only when the transient leads to a considerable increase in energy on the entire block, the smoothing alone, so the more even distribution of energy over the block, will not be sufficient and then a controlled signal limitation can be made.
  • matrix decoders such as Dolby Pro Logic II or Logic 7, have the ability to upmix non-preprocessed 2-channel stereo files in multichannel surround files, although they have not been designed immediately for this task. These matrix decoders are often incapable of rejecting transient tones in the back channels, resulting in a signal that does not meet the requirements for transient-free and amplitude-continuity.
  • a transient suppression is produced without affecting the continuity of the synthesis signal or ambient signal.
  • an input signal e.g. a high-mix signal as obtained by a matrix up-mixer for the back channels, or a signal with similar characteristics and a similar range of application is analyzed to detect if a transient is present.
  • substitution signal When a transient is detected, the currently processed block is replaced with a substitution signal having a flat (non-transient) time envelope.
  • This substitution signal is either generated from previous signal portions in which no transient was present, or is generated from the currently processed block by a processing step that flattens the temporal envelope of the signal, or is generated by a combination of both methods.
  • substitution signal generated by previous sections is generated, for example, by extrapolating previous energy levels of the signal or by copying / repeating previous signal sections without a transient portion of the signal.
  • flattening of the fine time structure or the fine timing signal on the basis of the currently processed block may be performed be, as described below with reference to the Figures 5a, 5b or 5c is shown.
  • the absolute values of the spectral coefficients can be randomized within a limited range extending around the extrapolated spectral coefficients or amounts thereof, as will be described later Fig. 5c is pictured.
  • the phases or signs of the spectral coefficients of the processed block in which the transient is located may be randomized by a randomizer 50.
  • a short-time spectrum of the considered block of the examination signal is generated, and the complex spectral values obtained there are calculated in magnitude and phase, in order then to randomize the phases of the spectral values.
  • the signs can also be randomized to obtain a short-term spectrum with randomized phases / signs, which has a has a flatter time course of the corresponding time signal.
  • Fig. 5b An alternative implementation is in Fig. 5b represented by a predictor 51, which is designed to perform a prediction of the short-term spectrum over the frequency.
  • a predictor 51 is designed to perform a prediction of the short-term spectrum over the frequency.
  • Such a predictor is in J. Herre, JD Johnston: "Exploiting Both Time and Frequency Structure in a System that Uses presented to Analysis / Synthesis Filterbank with High Frequency Resolution", 103rd AES Convention, New York 1997, Preprint 4519 ,
  • a short-term spectrum is generated which has a transient course in its assigned time signal.
  • a current spectral value of the short-term spectrum is predicted using a previous or a plurality of previous spectral values, and then the predicted spectral value could be subtracted from the actual spectral value to obtain a residual spectral value.
  • the residual spectral value of a typical prediction over frequency represents the value of interest and information-bearing information along with coefficients of a prediction filter
  • a given prediction filter is preset and the spectral values of the short-term spectrum are replaced by the spectral values predicted using this prediction filter, while the prediction error signal is no longer used.
  • a preferred simple embodiment of the prediction filter is simply that a value of a spectral line lower in the index is used as the prediction value for a current spectral line.
  • the extrapolated signal may be blended with the original signal after a specified period of time rather than hard switching, to avoid long-term extrapolation artefacts.
  • Fig. 6 it is preferred, as it is based on Fig. 6 is shown to detect tonal components / bands by a detector 60 and not to be influenced by the synthesis signal generator, but to combine in a mixer / combiner 61 with transient band synthesis signals to then, after a transformation into the time domain, the could take place in block 61, to obtain a time signal with a shallower time course, which still includes the tonal bands, that is, portions that were not transient, in an unchanged form.
  • Fig. 5c an implementation of the present invention, which does not require an implicit and no longer explicit transient detector.
  • a measure of the intensity of a processed signal block is, for example, the energy or radio frequency content (HFC) or other measure based on the spectral values, time samples, energy, power or other amplitude-related measure of the signal. It is then determined in a device 54 whether an intensity increases from one block to the next above a threshold.
  • HFC radio frequency content
  • the spectral values of the processed block are limited so that their intensity does not exceed the intensity of the previous signal block by more than the particular relative or exceeds absolute threshold such that at least the overall dominance of transients is reduced.
  • This limitation takes place in a device 55 which is designed to, when a need for a limitation has been detected, ie when an implicit transient has been detected, either individually or globally limit spectral values.
  • An individual limitation would be that an increase in energy is calculated for spectral values or for bands and the spectral values or the energy bands increase only up to a maximum energy increase and values exceeding this are cut off.
  • the synthesis signal generator 12 is configured to randomize complex spectral values representing a short-term spectrum of the block comprising the transient period 20 with respect to their phase or their sign.
  • the synthesis signal generator 12 is configured to perform prediction processing 51 over frequency to obtain a prediction spectrum whose associated time signal has a flatter timing than a time signal associated with a spectrum prior to prediction processing versus frequency.
  • the transient detector 11 is configured to calculate a high-frequency content for a block of the examination signal 61, wherein the transient detector 11 is designed to be the weighted HF content with a moving average over a plurality of preceding or following Compare blocks without transient 73, wherein the transient detector 11 is adapted to detect a transient for a block when the RF content of a current block exceeds the moving average by more than a threshold c.
  • the transient detector is configured to use a threshold that is selected depending on the manner of calculating the moving average and closer to unity when the moving average is more in the past and farther from one, when the past comparatively weakens in the moving average.
  • the synthesis signal generator 12 is configured to calculate the synthesis signal from signal portions of the examination signal before or after the transient period, from the examination signal in the transient period after smoothing the time course thereof or from a combination of the signal portions of the examination signal and the examination signal after a smoothing.
  • the synthesis signal generator 12 is configured to copy signal portions of the examination signal before or after the transient period.
  • the synthesis signal generator 12 is configured to randomize extrapolated spectral values derived from the examination signal outside the transient period in a predetermined range.
  • the synthesis signal generator 12 is configured to, when the transient period lasts longer than a predetermined time, mix synthesis signal values with signal values of the examination signal for times later than the predetermined period.
  • the signal substitute 14 is configured to fade into a transient period from a portion prior to the transient period according to a fade function, or to fade from the transient period into a portion after the transient period according to a fade function.
  • the synthesis signal generator 12 is configured to compute 40, 41, 42 a short-term spectrum of the synthesis signal to convert the short-term spectrum into a time representation 43 representing the synthesis signal 44.
  • the synthesis signal generator 12 is configured to compute a short-term spectrum of the synthesis signal with subband signals 40, 41, 42, and to convert the short-term spectrum with subband signals into a timing representative of the synthesis signal 43.
  • the synthesis signal generator 12 is configured to generate the synthesis signal such that the predetermined threshold is less than or equal to a factor of two.
  • the synthesis signal generator 12 is configured to use a band-selective preset threshold or a single threshold for the entire spectrum.
  • the apparatus further comprises extraction means for processing a left channel signal and a right channel signal to extract the examination signal.
  • the present invention further relates to a method for generating an ambient signal suitable for transmission via loudspeakers Ls, Rs for which no suitable loudspeaker signal is present, comprising the following steps: Detecting 11 a transient period 20 in which an examination signal has a transient region 22 ; Generating 12 a synthesis signal for the transient period 20, wherein the synthesis signal generator 12 is adapted to generate a synthesis signal having a shallower time course than the examination signal in the transient period 20, and the intensity of an intensity of a preceding or succeeding portion of the examination signal deviates less than a predetermined threshold; and substituting 14 the examination signal in the transient period 20 by the synthesis signal to obtain the surrounding signal.
  • the method according to the invention can be implemented in hardware or in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronic readable control signals that can interact with a programmable computer system such that the method is executed.
  • the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer.
  • the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)

Claims (10)

  1. Dispositif pour générer un signal d'environnement qui convient pour la diffusion par l'intermédiaire de haut-parleurs (Ls, Rs) pour lesquels aucun signal de haut-parleur approprié n'est présent, aux caractéristiques suivantes:
    un détecteur de transitoire (11) destiné à détecter un transitoire (22) dans un bloc d'un signal de test, pour obtenir une période transitoire (20) qui présente le bloc du signal de test;
    un générateur de signal de synthèse (12) destiné à générer un signal de synthèse pour le période transitoire (20), le générateur de signal de synthèse (12) étant conçu pour générer un signal de synthèse qui présente une évolution temporelle plus plane que le signal de test dans la période transitoire (20), et dont l'intensité s'écarte d'une intensité d'un segment du signal de test précédant la période transitoire (20) ou suivant la période transitoire (20) de moins d'un seuil prédéterminé; et
    un substituteur de signal (14) destiné à substituer le signal de test dans la période transitoire par le signal de synthèse, pour obtenir le signal d'environnement,
    le détecteur de transitoire (11) étant conçu pour calculer une teneur en hautes fréquences pondérée pour le bloc du signal de test (71), la teneur en hautes fréquences pondérée étant une somme pondérée de valeurs absolues de toutes les lignes de fréquence dans le bloc avec des facteurs de pondération croissant de basses à hautes fréquences;
    dans lequel le détecteur de transitoire (11) est conçu pour comparer la teneur en hautes fréquences pondérée du bloc avec une valeur moyenne mobile (72) des teneurs en hautes fréquences pondérées d'une pluralité de blocs sans transitoire précédant ou suivant le bloc (73),
    dans lequel le détecteur de transitoire (11) est conçu pour détecter un transitoire pour le bloc lorsque la teneur en hautes fréquences pondérée du bloc excède la valeur moyenne mobile de plus d'un seuil (c),
    dans lequel le générateur de signal de synthèse est conçu pour calculer, lors de la génération du signal de synthèse, pour chaque valeur spectrale d'un spectre de courte durée d'une pluralité de blocs qui constituent le segment du signal de test précédant la période transitoire ou suivant la période transitoire, une valeur moyenne à l'aide de valeurs spectrales correspondantes de la pluralité de blocs (74), pour obtenir un spectre de valeur moyenne pour le bloc,
    pour calculer, pour les valeurs spectrales du spectre de valeur moyenne pour le bloc, des écarts qui sont différents pour les valeurs spectrales du spectre de valeur moyenne et inférieurs à un écart maximum (Δmax),
    pour additionner les écarts et les valeurs spectrales de valeur moyenne (75), pour obtenir les valeurs spectrales pour le bloc, et
    pour convertir les valeurs spectrales pour le bloc en une représentation temporelle (43, 76) qui représente le signal de synthèse (44).
  2. Dispositif selon la revendication 1, qui est conçu pour un traitement de bloc, pour diviser le signal de test en blocs successifs et venant en recouvrement ou ne venant pas en recouvrement de valeurs de balayage discrètes dans le temps.
  3. Dispositif selon la revendication 1, dans lequel le seuil (c) est choisi en fonction du type de calcul de la valeur moyenne mobile et est plus proche de un lorsque le passé est plus fort dans la valeur moyenne mobile, et est plus éloignée de un lorsque le passé est relativement plus faible dans la valeur moyenne mobile.
  4. Dispositif selon l'une des revendications précédentes,
    dans lequel le générateur de signal de synthèse (12) est conçu pour calculer le signal de synthèse à partir de segments du signal de test avant ou après la période transitoire, à partir du signal de test pendant la période transitoire après lissage de l'évolution temporelle de cette dernière ou à partir d'une combinaison des segments du signal de test et du signal de test après lissage.
  5. Dispositif selon l'une des revendications précédentes,
    dans lequel le générateur de signal de synthèse (12) est conçu pour générer le signal de synthèse de sorte que le seuil prédéterminé soit inférieur ou égal à un facteur de 2.
  6. Dispositif selon l'une des revendications précédentes,
    dans lequel le seuil prédéterminé est un seuil préréglé de manière sélective par bande ou un seuil unique pour l'ensemble du spectre.
  7. Dispositif selon l'une des revendications précédentes, qui présente par ailleurs les caractéristiques suivantes:
    un moyen d'extraction destiné à traiter un signal de canal gauche et un signal de canal droit, pour extraire le signal de test.
  8. Dispositif selon l'une des revendications précédentes, qui présente par ailleurs la caractéristique suivante:
    un mélangeur de 2 à 3 (82) destiné à générer un canal gauche, un canal droit et un canal central à partir d'un signal stéréo transmis; et
    dans lequel le générateur de signal de synthèse (12) est conçu pour fournir le signal d'environnement pour un canal arrière gauche (Ls) et le signal d'environnement pour un canal arrière droit (Rs), le canal arrière gauche et le canal arrière droit étant les canaux pour lesquels aucun signal de haut-parleur approprié n'est présent, ou
    dans lequel le générateur de signal de synthèse (12) est conçu pour mettre à échelle le signal de test de sorte que soient obtenues pour le canal arrière gauche et le canal arrière droit des versions mises à échelle différentes du signal d'environnement, ou
    dans lequel le générateur de signal de synthèse (12) est conçu pour calculer, pour le canal arrière gauche, un premier signal d'environnement et pour calculer, pour le canal arrière droit, un deuxième signal d'environnement qui est différent du premier signal d'environnement.
  9. Procédé pour générer un signal d'environnement qui convient pour la diffusion par l'intermédiaire de haut-parleurs (Ls, Rs) pour lesquels aucun signal de haut-parleur approprié n'est présent, aux étapes suivantes consistant à:
    détecter (11) un transitoire (22) dans un bloc d'un signal de test, pour obtenir une période transitoire (20) qui présente le bloc du signal de test;
    générer (12) un signal de synthèse pour le période transitoire (20), le générateur de signal de synthèse (12) étant conçu pour générer un signal de synthèse qui présente une évolution temporelle plus plane que le signal de test dans la période transitoire (20), et dont l'intensité s'écarte d'une intensité d'un segment du signal de test précédant la période transitoire (20) ou suivant la période transitoire (20) de moins d'un seuil prédéterminé; et
    substituer (14) le signal de test dans la période transitoire (20) par le signal de synthèse, pour obtenir le signal d'environnement,
    dans lequel à l'étape consistant à détecter (11) est calculés une teneur en hautes fréquences pondérée pour le bloc du signal de test (71), la teneur en hautes fréquences pondérée étant une somme pondérée de valeurs absolues de toutes les lignes de fréquence dans le bloc avec des facteurs de pondération croissant de basses à hautes fréquences;
    dans lequel à l'étape consistant à détecter (11) est comparée la teneur en hautes fréquences pondérée du bloc avec une valeur moyenne mobile (72) des teneurs en hautes fréquences pondérées d'une pluralité de blocs sans transitoire précédant ou suivant le bloc (73),
    dans lequel à l'étape consistant à détecter (11) est détecté un transitoire pour le bloc lorsque la teneur en hautes fréquences pondérée du bloc excède la valeur moyenne mobile de plus d'un seuil (c),
    dans lequel à l'étape consistant à générer (12) est calculée, lors de la génération du signal de synthèse, pour chaque valeur spectrale d'un spectre de courte durée d'une pluralité de blocs qui constituent le segment du signal de test précédant la période transitoire ou suivant la période transitoire, une valeur moyenne à l'aide de valeurs spectrales correspondantes de la pluralité de blocs (74), pour obtenir un spectre de valeur moyenne pour le bloc,
    dans lequel à l'étape consistant à générer (12) sont calculés, pour les valeurs spectrales du spectre de valeur moyenne pour le bloc, des écarts qui sont différents pour les valeurs spectrales du spectre de valeur moyenne et inférieurs à un écart maximum (Δmax),
    dans lequel à l'étape consistant à générer (12) sont additionnés les écarts et les valeurs spectrales de valeur moyenne (75), pour obtenir les valeurs spectrales pour le bloc, et
    dans lequel, à l'étape consistant à générer (12), les valeurs spectrales pour le bloc sont converties en une représentation temporelle (43, 76) qui représente le signal de synthèse (44).
  10. Programme d'ordinateur pour réaliser un procédé selon la revendication 9 lorsque le procédé est exécuté sur un ordinateur.
EP11182965.1A 2006-04-12 2007-01-30 Dispositif et procédé de production d'un signal d'environnement Active EP2402943B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US74471806P 2006-04-12 2006-04-12
DE102006017280A DE102006017280A1 (de) 2006-04-12 2006-04-12 Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals
EP07703145.8A EP2005421B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé pour la génération d'un signal d'ambiance

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
EP07703145.8A Division EP2005421B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé pour la génération d'un signal d'ambiance
EP07703145.8 Division 2007-01-30

Publications (3)

Publication Number Publication Date
EP2402943A2 EP2402943A2 (fr) 2012-01-04
EP2402943A3 EP2402943A3 (fr) 2012-06-20
EP2402943B1 true EP2402943B1 (fr) 2016-08-24

Family

ID=38514551

Family Applications (3)

Application Number Title Priority Date Filing Date
EP11182960.2A Active EP2402942B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé de production d'un signal d'environnement
EP11182965.1A Active EP2402943B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé de production d'un signal d'environnement
EP07703145.8A Active EP2005421B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé pour la génération d'un signal d'ambiance

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP11182960.2A Active EP2402942B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé de production d'un signal d'environnement

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP07703145.8A Active EP2005421B1 (fr) 2006-04-12 2007-01-30 Dispositif et procédé pour la génération d'un signal d'ambiance

Country Status (11)

Country Link
US (2) US8577482B2 (fr)
EP (3) EP2402942B1 (fr)
JP (1) JP4664431B2 (fr)
CN (1) CN101421779B (fr)
CA (1) CA2643862C (fr)
DE (1) DE102006017280A1 (fr)
ES (1) ES2604133T3 (fr)
HK (1) HK1124951A1 (fr)
PL (1) PL2402943T3 (fr)
WO (1) WO2007118533A1 (fr)
ZA (1) ZA200809604B (fr)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE458361T1 (de) * 2005-12-13 2010-03-15 Nxp Bv Einrichtung und verfahren zum verarbeiten eines audio-datenstroms
GB2437337A (en) * 2006-04-21 2007-10-24 Snell & Wilcox Ltd Measuring block artefacts in video data using an auto-correlation function
US8379868B2 (en) * 2006-05-17 2013-02-19 Creative Technology Ltd Spatial audio coding based on universal spatial cues
JP2007316254A (ja) * 2006-05-24 2007-12-06 Sony Corp オーディオ信号補間方法及びオーディオ信号補間装置
CA2697920C (fr) 2007-08-27 2018-01-02 Telefonaktiebolaget L M Ericsson (Publ) Detecteur de transitoires et procede pour prendre en charge le codage d'un signal audio
DE102007048973B4 (de) 2007-10-12 2010-11-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Erzeugen eines Multikanalsignals mit einer Sprachsignalverarbeitung
RU2565008C2 (ru) * 2008-03-10 2015-10-10 Фраунхофер-Гезелльшафт цур Фёрдерунг дер ангевандтен Форшунг Е.Ф. Устройство и метод для обработки аудио сигнала, содержащего переходный сигнал
EP2272169B1 (fr) * 2008-03-31 2017-09-06 Creative Technology Ltd. Décomposition adaptative de signaux audio en composantes primaires et ambiantes
BRPI0823033B1 (pt) * 2008-12-11 2020-12-29 Fraunhofer-Gesellschaft Zür Förderung Der Angewandten Forschung E.V aparelho para gerar um sinal de áudio multi-canal
EP2214165A3 (fr) 2009-01-30 2010-09-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Appareil, procédé et programme informatique pour manipuler un signal audio comportant un événement transitoire
CN102157151B (zh) 2010-02-11 2012-10-03 华为技术有限公司 一种多声道信号编码方法、解码方法、装置和系统
JP5307770B2 (ja) * 2010-07-09 2013-10-02 シャープ株式会社 音声信号処理装置、方法、プログラム、及び記録媒体
US8489391B2 (en) * 2010-08-05 2013-07-16 Stmicroelectronics Asia Pacific Pte., Ltd. Scalable hybrid auto coder for transient detection in advanced audio coding with spectral band replication
HUE045991T2 (hu) * 2013-02-05 2020-01-28 Ericsson Telefon Ab L M Hangkeret-elvesztés elrejtése
DE102014214143B4 (de) 2014-03-14 2015-12-31 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Verarbeiten eines Signals im Frequenzbereich
US9820073B1 (en) 2017-05-10 2017-11-14 Tls Corp. Extracting a common signal from multiple audio signals
CN112731519A (zh) * 2019-10-28 2021-04-30 中国石油天然气集团有限公司 确定颤动时间区间的方法和装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4076969A (en) * 1975-04-07 1978-02-28 Singer & Singer Impulse noise reduction system
US4819269A (en) * 1987-07-21 1989-04-04 Hughes Aircraft Company Extended imaging split mode loudspeaker system
JPH0715800A (ja) * 1993-06-21 1995-01-17 Toshiba Corp サラウンド回路
US5610986A (en) * 1994-03-07 1997-03-11 Miles; Michael T. Linear-matrix audio-imaging system and image analyzer
US5886276A (en) 1997-01-16 1999-03-23 The Board Of Trustees Of The Leland Stanford Junior University System and method for multiresolution scalable audio signal encoding
US7231060B2 (en) 1997-08-26 2007-06-12 Color Kinetics Incorporated Systems and methods of generating control signals
US6928169B1 (en) * 1998-12-24 2005-08-09 Bose Corporation Audio signal processing
JP4870896B2 (ja) * 2000-07-19 2012-02-08 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ ステレオサラウンド及び／又はオーディオ中央信号を得るマルチチャンネルステレオコンバータ
KR20050021484A (ko) * 2002-07-16 2005-03-07 코닌클리케 필립스 일렉트로닉스 엔.브이. 오디오 코딩
SG108862A1 (en) 2002-07-24 2005-02-28 St Microelectronics Asia Method and system for parametric characterization of transient audio signals
US7949522B2 (en) * 2003-02-21 2011-05-24 Qnx Software Systems Co. System for suppressing rain noise
US7353169B1 (en) * 2003-06-24 2008-04-01 Creative Technology Ltd. Transient detection and modification in audio signals
JP2007505346A (ja) * 2003-09-09 2007-03-08 コニンクリユケ フィリップス エレクトロニクス エヌ．ブイ． 遷移のオーディオ信号成分の符号化
US7970144B1 (en) * 2003-12-17 2011-06-28 Creative Technology Ltd Extracting and modifying a panned source for enhancement and upmix of audio signals
SE0400997D0 (sv) * 2004-04-16 2004-04-16 Cooding Technologies Sweden Ab Efficient coding of multi-channel audio
US7876909B2 (en) * 2004-07-13 2011-01-25 Waves Audio Ltd. Efficient filter for artificial ambience

Also Published As

Publication number Publication date
EP2402943A2 (fr) 2012-01-04
CN101421779B (zh) 2013-04-17
CA2643862A1 (fr) 2007-10-25
EP2402943A3 (fr) 2012-06-20
HK1124951A1 (en) 2009-07-24
ES2604133T3 (es) 2017-03-03
JP4664431B2 (ja) 2011-04-06
EP2005421B1 (fr) 2013-06-26
JP2009533910A (ja) 2009-09-17
EP2402942A2 (fr) 2012-01-04
PL2402943T3 (pl) 2017-02-28
DE102006017280A1 (de) 2007-10-18
CN101421779A (zh) 2009-04-29
EP2402942A3 (fr) 2012-05-23
CA2643862C (fr) 2014-12-16
EP2402942B1 (fr) 2016-06-01
ZA200809604B (en) 2010-03-31
US8577482B2 (en) 2013-11-05
US20120195434A1 (en) 2012-08-02
US9326085B2 (en) 2016-04-26
US20070242833A1 (en) 2007-10-18
WO2007118533A1 (fr) 2007-10-25
EP2005421A1 (fr) 2008-12-24

Similar Documents

Publication Publication Date Title
EP2402943B1 (fr) Dispositif et procédé de production d'un signal d'environnement
EP2206113B1 (fr) Dispositif et procédé permettant de générer un signal multicanal par traitement d'un signal vocal
DE102006050068B4 (de) Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals aus einem Audiosignal, Vorrichtung und Verfahren zum Ableiten eines Mehrkanal-Audiosignals aus einem Audiosignal und Computerprogramm
DE602004002390T2 (de) Audiocodierung
DE602005005186T2 (de) Verfahren und system zur schallquellen-trennung
DE69731677T2 (de) Verbessertes Kombinationsstereokodierverfahren mit zeitlicher Hüllkurvenformgebung
DE69933659T2 (de) Verfahren und system zur räumlichen kodierung mit niedriger bitrate
EP1864279B1 (fr) Dispositif et procede pour produire un flux de donnees et pour produire une representation multicanaux
EP1854334B1 (fr) Dispositif et procede de production d'un signal stereo code d'un morceau audio ou d'un flux de donnees audio
EP1763870B1 (fr) Production d'un signal multicanal code, et decodage d'un signal multicanal code
DE602004008613T2 (de) Treueoptimierte kodierung mit variabler rahmenlänge
EP1145227B1 (fr) Procede et dispositif pour masquer une erreur dans un signal audio code, et procede et dispositif de decodage d'un signal audio code
DE69827775T2 (de) Tonkanalsmischung
EP2036400A1 (fr) Génération de signaux décorrélés
DE10148351B4 (de) Verfahren und Vorrichtung zur Auswahl eines Klangalgorithmus
DE4335739A1 (de) Verfahren zur Steuerung des Signal-/Rausch-Abstandes bei rauschbehafteten Tonaufnahmen
DE4445983C2 (de) Verfahren zur Rauschunterdrückung und Vorrichtungen zur Durchführung der Verfahren
EP1518441B1 (fr) Procede et dispositif pour empecher une retroaction
DE102016107799B3 (de) Verfahren zur Verarbeitung eines FM-Stereosignals
WO2005083702A1 (fr) Dispositif et procede d'ecriture sur un cd audio, et cd audio

Legal Events

Date Code Title Description
AC Divisional application: reference to earlier application

Ref document number: 2005421

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

RIN1 Information on inventor provided before grant (corrected)

Inventor name: JANSSEN, CHRISTIAAN

Inventor name: WALTHER, ANDREAS

Inventor name: HERRE, JUERGEN

Inventor name: GEYERSBERGER, STEPHAN

Inventor name: HELLMUTH, OLIVER

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/02 20060101AFI20120516BHEP

Ipc: H04S 5/00 20060101ALI20120516BHEP

17P Request for examination filed

Effective date: 20121219

17Q First examination report despatched

Effective date: 20140611

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 502007015054

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0019020000

Ipc: H04S0005000000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 5/04 20060101ALN20160222BHEP

Ipc: G10L 19/02 20130101ALI20160222BHEP

Ipc: H04S 5/00 20060101AFI20160222BHEP

Ipc: G10L 19/008 20130101ALN20160222BHEP

INTG Intention to grant announced

Effective date: 20160314

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AC Divisional application: reference to earlier application

Ref document number: 2005421

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 824017

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160915

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502007015054

Country of ref document: DE

RIN2 Information on inventor provided after grant (corrected)

Inventor name: HELLMUTH, OLIVER

Inventor name: GEYERSBERGER, STEFAN

Inventor name: JANSSEN, CHRISTIAAN

Inventor name: HERRE, JUERGEN

Inventor name: WALTHER, ANDREAS

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161125

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161226

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2604133

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20170303

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502007015054

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170131

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161124

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20170526

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160824

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170131

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170130

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170130

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 824017

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20070130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160824

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161224

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230123

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20230130

Year of fee payment: 17

Ref country code: PL

Payment date: 20230120

Year of fee payment: 17

Ref country code: IT

Payment date: 20230131

Year of fee payment: 17

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230512

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240123

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20240216

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240119

Year of fee payment: 18

Ref country code: CZ

Payment date: 20240118

Year of fee payment: 18

Ref country code: GB

Payment date: 20240124

Year of fee payment: 18