EP2005421A1 - Dispositif et procédé pour la génération d'un signal d'ambiance - Google Patents
Dispositif et procédé pour la génération d'un signal d'ambianceInfo
- Publication number
- EP2005421A1 EP2005421A1 EP07703145A EP07703145A EP2005421A1 EP 2005421 A1 EP2005421 A1 EP 2005421A1 EP 07703145 A EP07703145 A EP 07703145A EP 07703145 A EP07703145 A EP 07703145A EP 2005421 A1 EP2005421 A1 EP 2005421A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- signal
- transient
- synthesis
- block
- examination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004519 manufacturing process Methods 0.000 title 1
- 230000001052 transient effect Effects 0.000 claims abstract description 135
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 81
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 81
- 230000003595 spectral effect Effects 0.000 claims description 65
- 238000012545 processing Methods 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 26
- 238000001228 spectrum Methods 0.000 claims description 21
- 230000007613 environmental effect Effects 0.000 claims description 13
- 238000009499 grossing Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 230000002123 temporal effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 2
- 238000011835 investigation Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 10
- 239000000523 sample Substances 0.000 description 9
- 230000008859 change Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 5
- 230000005236 sound signal Effects 0.000 description 5
- 230000001629 suppression Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000013213 extrapolation Methods 0.000 description 4
- 230000007774 longterm Effects 0.000 description 4
- 238000012805 post-processing Methods 0.000 description 4
- 238000006467 substitution reaction Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000002238 attenuated effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 235000009508 confectionery Nutrition 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005562 fading Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000009527 percussion Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
- H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation of the pseudo five- or more-channel type, e.g. virtual surround
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
Definitions
- the present invention relates to audio signal processing, and more particularly to concepts for generating ambient signals (ambience signals) for loudspeakers in a multi-channel scenario for which no dedicated loudspeaker signal has been transmitted.
- Multi-channel audio is becoming more and more popular.
- Such playback systems generally consist of three speakers L (left), C (center) and R (right), which are typically located in front of the user, and two speakers Ls and Rs located behind the user, and typically one of them LFE channel, also called low frequency effect channel or subwoofer.
- LFE channel also called low frequency effect channel or subwoofer.
- the positioning of the LFE channel is not so critical because the ear can not locate at such low frequencies and thus the LFE channel can be located anywhere where it does not bother due to its considerable size.
- Such a multi-channel system provides several advantages over a typical stereo reproduction, which is a two-channel reproduction such as shown in FIG. 9.
- the listener has a better feeling of "immersing" in the audio scene due to the two rear speakers Ls and Rs.
- the first option is to play the left and right channels through the left and right speakers of the multi-channel playback system.
- a disadvantage of this solution is that you do not exploit the variety of existing speakers, so that you do not take advantage of the presence of the center speaker and the two rear speakers advantageous.
- Another option is to convert the two channels into a multi-channel signal. This can be done during playback or through special preprocessing, which can be done by all six loudspeakers of the existing NEN 5.1 playback system advantageously exploits and thus leads to an improved listening experience when the upmixing or the "upmix" of two channels on 5 or 6 channels is performed without errors.
- the direct sound sources are reproduced by the three front channels so that they are perceived by the user at the same position as in the original two-channel version.
- the original two-channel version is shown schematically in Fig. 9, using the example of various drum instruments.
- FIG. 11 Another alternative concept, which is referred to as "in-the-band" concept, is shown schematically in FIG. 11.
- Each type of sound ie direct sound sources and ambient sounds
- the position of a sound is independent of its characteristics (direct sound sources or ambient sounds) and depends only on the specific design of the algorithm, as described e.g. in Fig. 11 is shown.
- the upmix algorithm it has been determined by the upmix algorithm that the two instruments 1100 and 1102 are positioned laterally with respect to the listener while the two instruments 1104 and 1106 are positioned in front of the user.
- the two rear speakers Ls, Rs now also contain portions of the two instruments 1100 and 1102 and no longer only ambient sounds, as was the case with FIG. 10, where the same instruments are all positioned in front of the user have been.
- ambient signals would be very "defensive” to make sure that no user-perceived artifacts are generated.
- the other extreme case if one acts too defensively in the generation of the ambient signals, is that a very quiet, or barely perceptible ambient signal is extracted, or that the ambient signal has only noise, but no longer has any special information, so that the ambient signal contributes very little to the enjoyment of listening and could actually be left out altogether in this case.
- the problem with the generation of the ambient signal is thus that on the one hand generates an ambient signal that includes information that goes beyond normal noise, but the environment signal not audible artifacts leads, so that a proper measure between audibility and information content is maintained.
- the object of the present invention is to provide a concept for generating an environmental signal in which audible artifacts are reduced.
- This object is achieved by a device for generating an environmental signal according to claim 1, a method for generating an environmental signal according to claim 21 or a computer program according to claim 22.
- the present invention is based on the recognition that the artifacts heard by listeners in ambient signals are the artifacts that cause the listener to think that the rear speaker is a direct source of sound, even though it senses that source of sound from the front.
- Characteristics for the perception of direct sound sources are transient processes, ie signal fine structures in the time signal, which concern a (fast) change over a change threshold from a quiet state to a loud state or from a loud state to a quiet state, or the ei - relate to a (strong) increase in energy above a threshold of change in specific bands and in particular in the upper bands within a certain period of time.
- transient processes are, for example, the insertion of an instrument or the impact of a percussion instrument, or the end of a sound that does not fade away slowly but ends abruptly.
- a listener perceives such transient processes as characteristic of direct sound sources, which according to the invention are eliminated from an ambient signal, so that the ambient loudspeakers are provided with an ambient signal generated according to the invention which comprises no or only strongly attenuated transients.
- the suppression of a transient in the ambient signal does not lead to an excessive amplitude modulation.
- variations in amplitude that is to say volume, even if they are not transient, ie below the transient threshold, but above a certain threshold of variation, are disturbing for the user and thus, if such amplitude variations due to the simple elimination of a transient in an ambient signal, the listener would recognize it as an artifact or error.
- a transient period is thus detected in an examination signal, in which a transient region is present in the examination signal.
- a synthesis signal for the transient period is generated, which is designed to produce the synthesis signal such that it has a flatter time course than the examination signal in the transient region, wherein the synthesis signal generator is further configured to provide the synthesis signal to generate that it differs by less than a predetermined threshold in view of its intensity of a preceding or subsequent portion of the examination signal.
- This generated synthesis signal is then used by a signal substitutor instead of the examining signal in the transient period to obtain the surround signal.
- the extraction of an ambient signal-like signal from a two-channel stereo input signal is improved, or a post-processing of an existing signal, e.g. already an extracted raw ambient signal is made.
- the examination signal is the actual two-channel stereo signal or respectively one channel of the two-channel signal, while in the second case the examination signal is already an extracted environment signal or a presynthesized environment signal.
- the inventive concept is particularly useful for the upmix concept, which has also been presented as a "direct ambience concept".
- the concept according to the invention can also be advantageous for the "in-the-band" concept, since it also leads to an improved environmental signal which, on the one hand, no longer has any interfering artifacts, but on the other hand still contains enough information for a user to benefit from the ambient signal.
- the ambient signal generation according to the invention results in the ambient signal not having any relevant parts of direct Sound sources has, in particular, no transients are included or transients are included only in very strongly damped form. Otherwise, the listener would perceive direct sound sources behind them, which would conflict with the experience of the user, who typically only perceives sound sources from the front.
- the inventive concept ensures that the surround signal is a continuous, uninterrupted, diffused sound signal, since intermittent ambient sound, which is obtained, for example, if transients were simply completely eliminated, would be considered uncomfortable by the user or even errors in the high-mix process would be perceived.
- an ambience-like signal for the rear channels is extracted from the stereo signal.
- the difference between the original right and left channels is simply used.
- the back channels are created in this way, they often have transient-like components of direct sound sources.
- These transients can be tones, such as note beginnings or parts of percussive instruments.
- a transient perceived behind the listener while a direct sound source (to which the transient typically belongs) is positioned in front of the listener has a negative impact on the localization of the direct sound source.
- the direct sound source thus appears either wider than the original or, even more damaging, perceived as an independent direct sound source behind the user, both effects being very undesirable, especially for the direct ambience concept.
- this problem is addressed by suppressing transients in the ambient-like signal and minimizing the effect of this suppression on the remaining signal, ie, maintaining the continuity of the signal by allowing only limited intensity variations for the transient period ,
- the signal generated for the transient period, before being used by the signal substituter is mixed with the signal originally present in the transient period, which is achieved, for example, by overlapping processing.
- a cross-fading may be performed to slowly fade in a cross-fade range from the signal before the transient period into the signal in the transient period or out of the transient period slowly fade out again.
- the suppression of the transient period into the original signal, when no more transient is detected is preferred for an artifact-free listening impression, since it is to be ensured that when a non-artifact-containing examination signal is present, no change occurs due to the transition from the synthesis signal to the original examination signal Cracking or something similar arises.
- a manipulation of the signal in the transient period in the frequency domain is performed by randomizing signs of spectral values or, more generally, phases of spectral values, which inevitably leads to a smoothing of the temporal fine structure of this signal manipulated in the frequency domain.
- Another spectral processing is to perform a prediction on the frequency of the spectral values and then the Predictive spectral values to be used as spectral values of the synthesis signal, since the prediction on the frequency leads to a smoothing of the corresponding time signal.
- FIG. 1 shows a block diagram of the device according to the invention for generating an ambient signal
- Figure 2a is a schematic representation of block processing for non-overlapping blocks, but with cross-fade area
- Fig. 2b is a schematic representation of the synthesis signal generation in overlapping blocks
- FIG. 3 shows a specific implementation of a crossfade with a fade-in function and a fade-out function that can be used for FIG. 2a or FIG. 2b;
- FIG. 3 shows a specific implementation of a crossfade with a fade-in function and a fade-out function that can be used for FIG. 2a or FIG. 2b;
- FIG. 4 shows a block diagram of a preferred implementation with a processing in the frequency domain
- Fig. 5a shows an alternative implementation of frequency domain processing
- FIG. 5b shows another alternative frequency domain processing
- 5c shows a preferred implementation of an intensity-based processing
- Fig. 6 shows an implementation for preserving tonal regions in the synthesis signal
- Fig. 7 is a block diagram of a preferred embodiment based on the high frequency content HFC
- FIG. 8 shows a preferred implementation of the device according to the invention with additional functionality for generating the direct sound channels L, R, C;
- Fig. 9 is a stereo playback scenario
- Fig. 10 is a multi-channel playback scenario in which all direct sound sources are reproduced by the front channels.
- FIG. 11 shows a multi-channel reproduction scenario in which sound sources can also be reproduced by rear channels.
- Fig. 1 shows an inventive device for generating an ambient signal 10, which is suitable for broadcasting via speakers for which no separate speaker signal has been transmitted. Such speakers are typically the rear speakers or surround speakers, as shown at Ls, Rs in FIGS. 10 and 11, for example.
- the device shown in FIG. 1 comprises a transient detector 11 for detecting a transient period (shown at 20 in FIG. 2) in which an examination signal has a transient range.
- a transient detector 11 for detecting a transient period (shown at 20 in FIG. 2) in which an examination signal has a transient range.
- any other methods for transient detection can also be used, such as those found in an MPEG4 audio coder, for example, depending on a transient detection of short to long windows is switched.
- transient detectors are used which can detect fast and strong changes in the envelope of a time signal.
- Exemplary magnitudes to be detected are changes in the envelope that affect changes equal to or greater than 100% of the amplitude of the envelope over a period of 1 ms.
- the transient detector 11 is coupled to a synthesis signal generator 12, which is designed to generate a synthesis signal 13 which fulfills the two conditions, on the one hand the transient condition and, on the other hand, the continuity condition.
- the transient condition is that the synthesis signal has a shallower time course than the examination signal in the transient region
- the continuity condition is that the intensity of the synthesis signal in the transient region is less than a preset one of an intensity of a preceding or succeeding portion of the examination signal Threshold deviates.
- the intensity of the signal in the transient region is at most 1.5 times or 0.66 times the intensity of a preceding non-transient portion or subsequent non-transient portion of the examination signal. This will ensure that a transient suppression will not lead to a disturbing amplitude variation or intensity variation.
- the threshold can also be realized by a confidence interval of 80% or less, which is determined based on the historical values.
- Intensity measures that can be used for the present invention include the energy obtained by adding the sample squares or spectral value squares of a block or a measure of performance that can be obtained taking the temporal block length into account or else A measure that sums up or sums amounts of spectral values in a band, weighted or unweighted, where this particular measure, which is also an intensity, is also referred to as the high frequency content, if the band in which it is added is the upper frequency band of the examination signal, or generally higher frequencies are more heavily weighted towards lower frequencies or have a greater impact on the final result.
- the synthesis signal generator then generates a synthesis signal that is used by a signal substituter 14 to use the synthesis signal instead of the corresponding portion of the original examination signal to finally provide the ambient signal 10.
- the signal substitute 14 thus receives in addition to the synthesis signal via the line 13 and the examination signal via a line 15, as indicated in Fig. 1.
- the transient detector 11 receives the examination signal via an input line 16 and provides via an output line 17 a transient information to the synthesis signal generator 12, so that this generates the synthesis signal using the examination signal, which is provided to him via a line 18.
- non-overlapping block processing as shown in Figure 2a or overlapping block processing as shown in Figure 2b is employed. In the non-overlapping block processing in Fig.
- an examination signal 21 is divided into preferably equal blocks of a specific block length.
- the transient detector detects a transient 22 in the transient period 20.
- the transient 22 thus lies in the transient period 20 of FIG. 2 a, which results in the transient detector 11 providing an output signal via its output line 17 which tells the synthesis signal generator 12 that it now has to start with a signal synthesis. While the blocks preceding and following the transient period 20 directly represent the corresponding parts of the surround signal 10 except for a transition in a cross-fade area 23, the block of the examination signal corresponding to the transient period 20 is now synthesized by the synthesis signal generator and then by the signal substituter 14 is used instead of the original block of the examination signal in the ambient signal.
- a processing of the block of the examination signal is performed, which takes place in the frequency domain.
- the synthesis signal has a sample which may differ significantly from a sample which is the last sample of the preceding block in the examination signal.
- it is preferred in the embodiment shown in FIG. 2a to fade from a block before a transient period into the synthesis signal in the transient period for example by adding the first sample of the generated synthesis signal to the eg last 10 samples of the previous block weighted according to the skip function are added, and
- the last sample of the previous block is still weighted according to the fade-in function in FIG. 3 to the first or the first sample following the sampled values of the synthesized block Transient period added to create a crossfade. Accordingly, it is possible to proceed in the rear cross-fade range, ie, when the transient period returns to the non-transient-afflicted block of the ambient signal.
- the transient detector then detects block areas which are represented by ringing numbers (1), (2), (3), (4), (5), (6).
- a transient is detected at 22. This results in a larger transient period 20 compared to FIG. 2a, since the transient at position 22 has been detected in both block 4 and block 5. Therefore, the synthesis signal generator 12 of FIG. 1 will generate synthesis signals for both block 4 and block 5.
- the section A is generated by the addition of the second half of the non-transient-related block 3 of the examination signal with the first half of the synthesis signal generated for the block 4, namely by the sections A, B, C generated by the synthesis signal generators.
- the second part B of the transient period 20 is supplied by adding the second half of the synthesis signal generated for the block 4 to the first half of the synthesis signal generated for the block 5, and from the signal substitute as corresponding section of the ambient signal 10 substituted.
- the third portion C of the transient period 20 is generated by adding the second half of the synthesizer signal generator generated block # 5 to the first half of the block # 6, which is no longer transient, and written into the ambient signal by the signal substituter 14.
- this skip function can be used to provide a smooth block transition from a non-synthesized block to a synthesized block in block processing with non-overlapping blocks and also a smooth transition from a synthesized block back to a non-synthesized block to accomplish.
- a corresponding cross-fade function can also be used to overshadow back to the original examination signal, in particular when a synthesis signal has been generated by a specific specified number of blocks. Given the likelihood that the synthesis signal has moved relatively far from the probe due to the extrapolation, an abrupt return to the probe would, in some cases, result in audible artifacts.
- a synthesis signal consisting of 90% of the last synthesized block and block for a block in which no more transient has already been detected to 10% from the current study block.
- the ratio could then be changed to 80%: 20% until, after a certain number of blocks, the synthesis signal is completely blanked out and the current non-transient examination signal is completely displayed again.
- the time signal which represents a block of the examination signal, is converted into a frequency domain representation or a subband representation by a converter 40, which may comprise a transformation or an analysis filter bank.
- the spectral representation in the form of spectral coefficients or the subband signals are then, as shown at 41, optionally replaced by information about an extrapolated spectral representation or extrapolated subband signals, if it is a block of the time signal in which a transient has been detected.
- the spectral representation is then optionally supplied, using additional information due to extrapolation, to a smoother 42, which influences the spectral values in such a way that the temporal course of the underlying signal is smoothed.
- the smoother 42 will affect the subband signals so that the timing of the signal underlying the subband signals is smoother than before smoothing.
- an inverse conversion into the time domain takes place, using either a back transformation or a synthesis filter bank, to finally arrive at a time signal 44, which has a smoother profile than the time signal at the input of the stage 40, but that has an energy that has not been significantly affected by the smoothing.
- the smoothing has been done so that the energy of the smoothed time signal 44 does not differ from the energy of the previous time signal any more than the threshold.
- an overall energy manipulation of the time signal energy may take place.
- the transient is attenuated while the tonal components continue to be synthesized or synthesized from the past by synthesizing the signal in the transient period by a prediction using a non-transient signal from the past.
- the smoothing has caused the energy over the block to be more evenly distributed, thus producing a smoother timing, but without losing the block's energy Significantly changing samples of the examination signal. This is sufficient in most cases and ensures that the user hears an examining signal that always satisfies the continuity condition. Only when the transient leads to a considerable energy increase on the whole block, the smoothing alone, ie the more even distribution of the energy over the block, will not be sufficient anymore and then a controlled signal limitation can be made.
- Matrix decoders such as Dolby Pro Logic II or Logic 7, have the ability to upmix non-preprocessed 2-channel stereo files into multichannel surround files, although they have not been designed directly for this task are. These Matrix decoders are often unable to suppress transient tones in the rear channels, resulting in a signal that does not meet the requirements for transient freedom and amplitude continuity.
- a transient suppression is produced without adversely affecting the continuity of the synthesis signal or ambient signal.
- an input signal e.g. a high-mix signal as obtained by a matrix up-mixer for the back channels, or a signal with similar characteristics and a similar range of application is analyzed to detect if a transient is present.
- substitution signal When a transient is detected, the currently processed block is replaced with a substitution signal having a flat (non-transient) time envelope.
- This substitution signal is either generated from previous signal portions in which no transient was present, or is generated from the currently processed block by a processing step that flattens the temporal envelope of the signal, or is generated by a combination of both methods.
- substitution signal generated by previous sections is obtained, for example, by extrapolating previous signal energy levels or by copying Reproduce previous signal sections without a transient region of the signal generated.
- the absolute values of the spectral coefficients can be randomized within a limited range extending around the extrapolated spectral coefficients or amounts thereof, as will be described later in connection with FIG. 5c.
- the phases or signs of the spectral coefficients of the processed block in which the transient is located may be randomized by a randomizer 50.
- a short-time spectrum of the considered block of the examination signal is generated, and the complex spectral values obtained there are calculated in magnitude and phase, in order then to randomize the phases of the spectral values.
- the signs can also be randomized to obtain a short-term spectrum with randomized phases / signs, which has a has a flatter time course of the corresponding time signal.
- FIG. 5b An alternative implementation is illustrated in FIG. 5b by means of a predictor 51, which is designed to perform a prediction of the short-term spectrum over the frequency.
- a predictor 51 is described in J. Herre, JD Johnston: "Exploiting Both Time and Frequency Structure in a system did Uses of Analysis / Synthesis Filter Bank with High Frequency Resolution", 103 rd AES Convention, New York 1997, Preprint 4519 shown.
- a short-term spectrum is generated which has a transient course in its assigned time signal.
- a current spectral value of the short-term spectrum is predicted using a previous or a plurality of previous spectral values, and then the predicted spectral value could be subtracted from the actual spectral value to obtain a residual spectral value.
- the residual spectral value of a typical prediction over frequency represents the value of interest and information-bearing information along with coefficients of a prediction filter
- a given prediction filter is preset and the spectral values of the short-term spectrum are replaced by the spectral values predicted using this prediction filter, while the prediction error signal is no longer used.
- the actually erroneous predictive spectral values thus obtained now have a flatter time course than the original short-term spectrum, but still have approximately the same energy, so that both the transient condition and the continuity condition, as shown in connection with the synthesis signal generator 12 of FIG has been fulfilled.
- a preferred simple embodiment of the prediction filter exists simply in that a value of a spectral line lower in the index is used as a prediction value for a current spectral line.
- the extrapolated signal may be blended with the original signal after a specified period of time, rather than hard switching, to avoid long-term extrapolation artifacts.
- tonal components / bands by a detector 60 and not to influence them by the synthesis signal generator, but to combine them in a mixer / combiner 61 with transient band synthesis signals to then , after being transformed into the time domain that could take place in block 61, obtain a time signal with a shallower time course, but which still comprises the tonal bands, ie, portions that were not transient, in an unchanged form.
- stationary / tonal frequency components in the input signal which were present during the duration of the transient, for example, in only parts of the spectrum, are detected and a substitution signal is generated, which consists of an extrapolation of the past stationary / tonal signal components and those detected in the current block stationary / tonal frequency components.
- FIG. 5c An implementation of the present invention, which uses an implicit and no longer explicit transient detector, is shown below with reference to FIG. 5c.
- a device 53 for calculating the intensity of a block and of a previous block is shown in FIG. 5c.
- a measure of the intensity of a processed signal block is, for example, the energy or radio frequency content (HFC) or other measure based on the spectral values, time samples, energy, power or other amplitude-related measure based on the signal. It is then determined in a device 54 whether an intensity increases from one block to the next above a threshold.
- HFC radio frequency content
- the spectral values of the processed block are limited so that their intensity does not exceed the intensity of the previous signal block by more than the determined relative or absolute threshold such that at least the overall dominance of transients is reduced.
- This limitation takes place in a device 55 which is designed to, when a need for a limitation has been detected, ie when an implicit transient has been detected, either individually or globally limit spectral values.
- An individual limitation would be that an increase in energy is calculated for spectral values or for bands and the spectral values or the energy bands increase only up to a maximum energy increase and values exceeding this are cut off.
- the means 55 for limiting the spectral values therefore limits the spectral values individually or globally, an individual limitation being that only those spectral values which increase above a threshold are limited and preferably limited to this threshold, while the other spectral values, the do not rise so much, not be influenced. Alternatively, however, in certain cases it will be more convenient and simpler in terms of computational effort, then, if too much increase has been detected, all spectral values will be bounded by the same absolute or relative measure.
- the block Z / F represents a time / frequency domain conversion 57, wherein a conversion from the time to the frequency domain may also be a filtering by means of an analysis filter bank, that in this case the spectral representation consists of subband signals and not individual spectral components.
- the transient detector as shown at 11 in Fig. 1, in this embodiment, includes high frequency content (HFC) calculating means 71 for each block, followed by means for calculating the long term HFC-72.
- a comparator 73 detects if there is a transient or if there is a transient period in which a transient exists.
- means 71 is configured to calculate the weighted radio frequency content (HFC) for each block of the original left signal and the original right signal.
- HFC radio frequency content
- the HFC is the weighted sum of the absolute values of all the frequency lines in a block, with increasing weighting factors from lower to higher frequencies.
- the HFC is calculated as follows:
- X (f) are the spectral coefficients for certain frequencies
- w (f) are weighting factors for certain frequencies.
- all spectral components can be used to calculate the HFC.
- the calculation of the HFC can also be carried out starting from a limit which is approximately in the middle range of the spectrum, so that the low spectral coefficients play no role in the calculation of the HFC.
- the threshold depends on the type of moving average. If the moving average is an average in which the past is weighted more heavily than the more recent block, ie a slow average, then the threshold is closer to unity than in the case where the past is less heavily weighted in the moving average. Here the threshold would be further away from one.
- SW SW m + RN- ⁇ max .
- the extrapolated values are blended in with the original values when a fixed time interval has elapsed, such as having three blocks of synthesis signals of which then must be returned to the original signal. If, however, the transient period is shorter than three blocks, then it is preferred not to carry out the cross-fading, because then it is assumed that the extrapolated signals have not yet moved so much away from the original signals.
- the crossfading can take place either before conversion into the time domain or, preferably, after conversion into the time domain, as shown at 76 in FIG. 7, in order to obtain the synthesis signal.
- the inventive concept may be integrated into an environment signal extraction process or used as a separate post-processing step using an existing environmental signal, but still includes undesirable transients prior to processing in accordance with the invention.
- the processing steps according to the invention can be carried out in the frequency domain per frequency line or in subbands. However, they can also be performed only partially in the frequency range, typically above a certain frequency limit, or else in the time domain exclusively or in a combination of time and frequency range.
- the ambient signal generation device is not only designed to generate ambient signals for a left surround channel output 80 and a right surround channel output 81.
- the device according to the invention additionally comprises in addition a high mixer 82 for generating signals for the left channel L, the right channel R, the center channel C and preferably also for the LFE channel, as shown in FIG.
- Both the combination of transient detector 12, synthesis generator 14 and signal substitute 16, ie the high mixer 82, are fed by a decoder 84.
- the decoder 84 is configured to receive and process a bitstream 85 to provide a mono or stereo signal 86 at the output.
- the bit stream may be an MP3 bit stream or an MP3 file, or it may be an AAC file, or may also be a representation of a parametrically encoded multi-channel signal.
- the bitstream 85 could be a left channel, right channel, and center channel parametric representation, including one transmission channel and multiple cues for the second and third channels, this processing being known from BCC multi-channel processing.
- the decoder 84 would be a BCC decoder which not only provides a mono or a stereo signal, but even provides a 3-channel signal, but which still does not include data on the two surround channels Ls, Rs.
- the examination signal is thus in this case a mono signal, a stereo signal or even a multimedia signal. tikanal signal, but does not include their own speaker signals for the surround channels Ls, Rs.
- the examination signal or the ambient signal derived from a sum of left and right channel In the other case z.
- the surrounding signal for the left surround channel is calculated, and from the right channel, the surround signal for the right channel is calculated.
- the method according to the invention can be implemented in hardware or in software.
- the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmable computer system such that the method is performed.
- the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer.
- the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Stereophonic System (AREA)
Abstract
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP11182960.2A EP2402942B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
EP11182965.1A EP2402943B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US74471806P | 2006-04-12 | 2006-04-12 | |
DE102006017280A DE102006017280A1 (de) | 2006-04-12 | 2006-04-12 | Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals |
PCT/EP2007/000791 WO2007118533A1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé pour la génération d'un signal d'ambiance |
Related Child Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP11182960.2A Division EP2402942B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
EP11182965.1A Division EP2402943B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
EP11182960.2 Division-Into | 2011-09-27 | ||
EP11182965.1 Division-Into | 2011-09-27 |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2005421A1 true EP2005421A1 (fr) | 2008-12-24 |
EP2005421B1 EP2005421B1 (fr) | 2013-06-26 |
Family
ID=38514551
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07703145.8A Active EP2005421B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé pour la génération d'un signal d'ambiance |
EP11182960.2A Active EP2402942B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
EP11182965.1A Active EP2402943B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP11182960.2A Active EP2402942B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
EP11182965.1A Active EP2402943B1 (fr) | 2006-04-12 | 2007-01-30 | Dispositif et procédé de production d'un signal d'environnement |
Country Status (11)
Country | Link |
---|---|
US (2) | US8577482B2 (fr) |
EP (3) | EP2005421B1 (fr) |
JP (1) | JP4664431B2 (fr) |
CN (1) | CN101421779B (fr) |
CA (1) | CA2643862C (fr) |
DE (1) | DE102006017280A1 (fr) |
ES (1) | ES2604133T3 (fr) |
HK (1) | HK1124951A1 (fr) |
PL (1) | PL2402943T3 (fr) |
WO (1) | WO2007118533A1 (fr) |
ZA (1) | ZA200809604B (fr) |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1964438B1 (fr) * | 2005-12-13 | 2010-02-17 | Nxp B.V. | Dispositif et méthode de traitement d'un flux de données audio |
GB2437337A (en) * | 2006-04-21 | 2007-10-24 | Snell & Wilcox Ltd | Measuring block artefacts in video data using an auto-correlation function |
US8379868B2 (en) * | 2006-05-17 | 2013-02-19 | Creative Technology Ltd | Spatial audio coding based on universal spatial cues |
JP2007316254A (ja) * | 2006-05-24 | 2007-12-06 | Sony Corp | オーディオ信号補間方法及びオーディオ信号補間装置 |
CN101790756B (zh) * | 2007-08-27 | 2012-09-05 | 爱立信电话股份有限公司 | 瞬态检测器以及用于支持音频信号的编码的方法 |
DE102007048973B4 (de) | 2007-10-12 | 2010-11-18 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Vorrichtung und Verfahren zum Erzeugen eines Multikanalsignals mit einer Sprachsignalverarbeitung |
BR122012006265B1 (pt) * | 2008-03-10 | 2024-01-09 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V | Equipamento e método para a manipulação de um sinal de áudio tendo um evento transiente |
EP2272169B1 (fr) * | 2008-03-31 | 2017-09-06 | Creative Technology Ltd. | Décomposition adaptative de signaux audio en composantes primaires et ambiantes |
JP5237463B2 (ja) * | 2008-12-11 | 2013-07-17 | フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー.ファオ | マルチチャンネルオーディオ信号を生成するための装置 |
EP2214165A3 (fr) * | 2009-01-30 | 2010-09-15 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Appareil, procédé et programme informatique pour manipuler un signal audio comportant un événement transitoire |
CN102157151B (zh) * | 2010-02-11 | 2012-10-03 | 华为技术有限公司 | 一种多声道信号编码方法、解码方法、装置和系统 |
JP5307770B2 (ja) * | 2010-07-09 | 2013-10-02 | シャープ株式会社 | 音声信号処理装置、方法、プログラム、及び記録媒体 |
US8489391B2 (en) * | 2010-08-05 | 2013-07-16 | Stmicroelectronics Asia Pacific Pte., Ltd. | Scalable hybrid auto coder for transient detection in advanced audio coding with spectral band replication |
PL3576087T3 (pl) * | 2013-02-05 | 2021-10-25 | Telefonaktiebolaget Lm Ericsson (Publ) | Ukrywanie klatki utraconej sygnału audio |
DE102014214143B4 (de) * | 2014-03-14 | 2015-12-31 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Vorrichtung und Verfahren zum Verarbeiten eines Signals im Frequenzbereich |
US9820073B1 (en) | 2017-05-10 | 2017-11-14 | Tls Corp. | Extracting a common signal from multiple audio signals |
CN112731519B (zh) * | 2019-10-28 | 2024-06-25 | 中国石油天然气集团有限公司 | 确定颤动时间区间的方法和装置 |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4076969A (en) * | 1975-04-07 | 1978-02-28 | Singer & Singer | Impulse noise reduction system |
US4819269A (en) * | 1987-07-21 | 1989-04-04 | Hughes Aircraft Company | Extended imaging split mode loudspeaker system |
JPH0715800A (ja) * | 1993-06-21 | 1995-01-17 | Toshiba Corp | サラウンド回路 |
US5610986A (en) * | 1994-03-07 | 1997-03-11 | Miles; Michael T. | Linear-matrix audio-imaging system and image analyzer |
US5886276A (en) | 1997-01-16 | 1999-03-23 | The Board Of Trustees Of The Leland Stanford Junior University | System and method for multiresolution scalable audio signal encoding |
US7231060B2 (en) | 1997-08-26 | 2007-06-12 | Color Kinetics Incorporated | Systems and methods of generating control signals |
US6928169B1 (en) * | 1998-12-24 | 2005-08-09 | Bose Corporation | Audio signal processing |
EP2299735B1 (fr) * | 2000-07-19 | 2014-04-23 | Koninklijke Philips N.V. | Convertisseur stéréo multicanaux pour dériver un signal central stéréo d'ambiophonie et/ou audio |
AU2003281128A1 (en) * | 2002-07-16 | 2004-02-02 | Koninklijke Philips Electronics N.V. | Audio coding |
SG108862A1 (en) * | 2002-07-24 | 2005-02-28 | St Microelectronics Asia | Method and system for parametric characterization of transient audio signals |
US7949522B2 (en) * | 2003-02-21 | 2011-05-24 | Qnx Software Systems Co. | System for suppressing rain noise |
US7353169B1 (en) * | 2003-06-24 | 2008-04-01 | Creative Technology Ltd. | Transient detection and modification in audio signals |
CN1849649A (zh) * | 2003-09-09 | 2006-10-18 | 皇家飞利浦电子股份有限公司 | 瞬态音频信号分量的编码 |
US7970144B1 (en) * | 2003-12-17 | 2011-06-28 | Creative Technology Ltd | Extracting and modifying a panned source for enhancement and upmix of audio signals |
SE0400997D0 (sv) * | 2004-04-16 | 2004-04-16 | Cooding Technologies Sweden Ab | Efficient coding of multi-channel audio |
US7876909B2 (en) * | 2004-07-13 | 2011-01-25 | Waves Audio Ltd. | Efficient filter for artificial ambience |
-
2006
- 2006-04-12 DE DE102006017280A patent/DE102006017280A1/de not_active Withdrawn
-
2007
- 2007-01-30 WO PCT/EP2007/000791 patent/WO2007118533A1/fr active Application Filing
- 2007-01-30 CA CA2643862A patent/CA2643862C/fr active Active
- 2007-01-30 EP EP07703145.8A patent/EP2005421B1/fr active Active
- 2007-01-30 EP EP11182960.2A patent/EP2402942B1/fr active Active
- 2007-01-30 JP JP2009504579A patent/JP4664431B2/ja active Active
- 2007-01-30 ES ES11182965.1T patent/ES2604133T3/es active Active
- 2007-01-30 PL PL11182965T patent/PL2402943T3/pl unknown
- 2007-01-30 CN CN2007800129627A patent/CN101421779B/zh active Active
- 2007-01-30 EP EP11182965.1A patent/EP2402943B1/fr active Active
- 2007-04-12 US US11/734,620 patent/US8577482B2/en active Active
-
2008
- 2008-11-11 ZA ZA200809604A patent/ZA200809604B/xx unknown
-
2009
- 2009-04-23 HK HK09103745.3A patent/HK1124951A1/xx unknown
-
2012
- 2012-04-09 US US13/442,649 patent/US9326085B2/en active Active
Non-Patent Citations (1)
Title |
---|
See references of WO2007118533A1 * |
Also Published As
Publication number | Publication date |
---|---|
CA2643862A1 (fr) | 2007-10-25 |
US20120195434A1 (en) | 2012-08-02 |
EP2402943A3 (fr) | 2012-06-20 |
JP4664431B2 (ja) | 2011-04-06 |
EP2402943A2 (fr) | 2012-01-04 |
EP2402943B1 (fr) | 2016-08-24 |
ZA200809604B (en) | 2010-03-31 |
EP2005421B1 (fr) | 2013-06-26 |
US9326085B2 (en) | 2016-04-26 |
EP2402942A3 (fr) | 2012-05-23 |
ES2604133T3 (es) | 2017-03-03 |
CA2643862C (fr) | 2014-12-16 |
DE102006017280A1 (de) | 2007-10-18 |
CN101421779B (zh) | 2013-04-17 |
EP2402942B1 (fr) | 2016-06-01 |
WO2007118533A1 (fr) | 2007-10-25 |
JP2009533910A (ja) | 2009-09-17 |
US20070242833A1 (en) | 2007-10-18 |
US8577482B2 (en) | 2013-11-05 |
EP2402942A2 (fr) | 2012-01-04 |
HK1124951A1 (en) | 2009-07-24 |
CN101421779A (zh) | 2009-04-29 |
PL2402943T3 (pl) | 2017-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2402943B1 (fr) | Dispositif et procédé de production d'un signal d'environnement | |
EP2206113B1 (fr) | Dispositif et procédé permettant de générer un signal multicanal par traitement d'un signal vocal | |
DE102006050068B4 (de) | Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals aus einem Audiosignal, Vorrichtung und Verfahren zum Ableiten eines Mehrkanal-Audiosignals aus einem Audiosignal und Computerprogramm | |
DE602004002390T2 (de) | Audiocodierung | |
DE69933659T2 (de) | Verfahren und system zur räumlichen kodierung mit niedriger bitrate | |
DE69731677T2 (de) | Verbessertes Kombinationsstereokodierverfahren mit zeitlicher Hüllkurvenformgebung | |
EP1763870B1 (fr) | Production d'un signal multicanal code, et decodage d'un signal multicanal code | |
EP1854334B1 (fr) | Dispositif et procede de production d'un signal stereo code d'un morceau audio ou d'un flux de donnees audio | |
DE69214523T2 (de) | Dekodierer für variable anzahl von kanaldarstellungen mehrdimensionaler schallfelder | |
EP1145227B1 (fr) | Procede et dispositif pour masquer une erreur dans un signal audio code, et procede et dispositif de decodage d'un signal audio code | |
EP1864279B1 (fr) | Dispositif et procede pour produire un flux de donnees et pour produire une representation multicanaux | |
DE602005005186T2 (de) | Verfahren und system zur schallquellen-trennung | |
DE60206390T2 (de) | Effiziente und skalierbare parametrische stereocodierung für anwendungen mit niedriger bitrate | |
DE69432012T2 (de) | Wahrnehmungsgebundene Kodierung von Audiosignalen | |
DE69827775T2 (de) | Tonkanalsmischung | |
WO2008125322A1 (fr) | Génération de signaux décorrélés | |
DE10148351B4 (de) | Verfahren und Vorrichtung zur Auswahl eines Klangalgorithmus | |
DE4335739A1 (de) | Verfahren zur Steuerung des Signal-/Rausch-Abstandes bei rauschbehafteten Tonaufnahmen | |
EP1719128A1 (fr) | Dispositif et procede d'ecriture sur un cd audio, et cd audio | |
DE202004003000U1 (de) | Vorrichtung zum Beschreiben einer Audio-CD und Audio-CD |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20080909 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1124951 Country of ref document: HK |
|
17Q | First examination report despatched |
Effective date: 20090917 |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 619006 Country of ref document: AT Kind code of ref document: T Effective date: 20130715 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502007011951 Country of ref document: DE Effective date: 20130822 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130927 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130926 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20130626 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1124951 Country of ref document: HK |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130619 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131026 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131028 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131007 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
26N | No opposition filed |
Effective date: 20140327 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502007011951 Country of ref document: DE Effective date: 20140327 |
|
BERE | Be: lapsed |
Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FORDERUNG DER ANGEWAN Effective date: 20140131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140130 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140131 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140131 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140130 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140131 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 619006 Country of ref document: AT Kind code of ref document: T Effective date: 20140130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140130 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130626 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20070130 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 12 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230512 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240119 Year of fee payment: 18 Ref country code: GB Payment date: 20240124 Year of fee payment: 18 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240124 Year of fee payment: 18 |