EP2499843A1 - Method for dubbing microphone signals of a sound recording having a plurality of microphones - Google Patents

Method for dubbing microphone signals of a sound recording having a plurality of microphones

Info

Publication number
EP2499843A1
EP2499843A1 EP10779267A EP10779267A EP2499843A1 EP 2499843 A1 EP2499843 A1 EP 2499843A1 EP 10779267 A EP10779267 A EP 10779267A EP 10779267 A EP10779267 A EP 10779267A EP 2499843 A1 EP2499843 A1 EP 2499843A1
Authority
EP
European Patent Office
Prior art keywords
spectral values
signal
microphone
prioritized
imag
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP10779267A
Other languages
German (de)
French (fr)
Other versions
EP2499843B1 (en
Inventor
Jens Groh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institut fuer Rundfunktechnik GmbH
Original Assignee
Institut fuer Rundfunktechnik GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institut fuer Rundfunktechnik GmbH filed Critical Institut fuer Rundfunktechnik GmbH
Publication of EP2499843A1 publication Critical patent/EP2499843A1/en
Application granted granted Critical
Publication of EP2499843B1 publication Critical patent/EP2499843B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/02—Arrangements for generating broadcast information; Arrangements for generating broadcast-related information with a direct linking to broadcast information or to broadcast space-time; Arrangements for simultaneous generation of broadcast information and broadcast-related information
    • H04H60/04—Studio equipment; Interconnection of studios

Definitions

  • the invention relates to a method according to the preamble of claim 1.
  • a method is known from WO 2004/084 185 AI. It is known ("Handbuch der Tonstudiotechnik” by Michael Dickreiter et al., ISBN 978-3598117657) to capture an extensive acoustic scene in the production of audio recordings for music conserves, films, radio broadcasts, sound archives, computer games, multimedia presentations or Internet presences , Pages 211-212, 230-235, 265-266, 439, 479) to use multiple microphones instead of just a single microphone.
  • the term "multi-microphone sound recording” is generally used.
  • An extended acoustic scene may be, for example, a concert hall with an orchestra of a variety of musical instruments.
  • each individual musical instrument is recorded with a single, closely positioned microphone and also positions further microphones at a greater distance to capture the overall acoustic image, including the reverberation in the concert hall and the audience noises (in particular applause).
  • an extended acoustic scene is a drum kit consisting of several percussion instruments recorded in the recording studio.
  • a microphone is positioned in close proximity in front of the individual percussion instruments and an additional microphone is mounted above the percussionist.
  • Such multimicrophone sound recordings make it possible to record as many acoustic and sound properties of the details as possible as well as of the overall picture of the scenery in high quality and to make them aesthetically pleasing to design.
  • Each microphone signal of the plurality of microphones is usually recorded as multi-track recording. In the subsequent mixing of the microphone signals further creative work is done. In special cases it is also possible to mix "live" immediately and record only the result of the mixdown.
  • the design goals of the mix are usually a balanced ratio of the volumes of all sound sources, a natural sound and a realistic spatial impression of the overall acoustic image.
  • FIG. 1 shows by way of example a single summation in the signal path of a conventional sound mixer or digital sound system.
  • a series connection of summations in the summer ("bus") in the signal path of a conventional sound mixer or digital sound system is exemplified in FIG.
  • 210 is an n + 1th addition-based summation level
  • multi-microphone sound recordings contain at least two microphone signals due to the unavoidable multipath propagation of sound portions of sound that come from the sound of one and the same sound source. Since these sound components arrive at the microphones as a result of the different sound paths with different transit times, comb filter effects, which are audible as sound changes and run counter to the intended naturalness of the sound, are produced in conventional mixer technology in the summer. In conventional mixing techniques, such sound variations due to comb filter effects can be reduced by adjustable gain and optionally adjustable delay of the recorded microphone signals. However, such a reduction is only possible to a limited extent if a multi-path Sound propagation of more than a single sound source is present. In any case, however, a considerable adjustment effort on the mixer or digital sound system for finding the best compromise is required.
  • the prior DE 10 2008 056 704 describes downmixing for the generation of a two-channel audio format from a multichannel (e.g., five-channel) audio format that mimics phantom sound sources.
  • two input signals are summed, wherein a weighting of the spectral coefficients of one of the two input signals to be summed takes place with a correction factor; that input signal which is weighted by the correction factor is prioritized over the other input signal.
  • the determination of the correction factor described in DE 10 2008 056 704 means that in cases where the amplitude of the prioritized signal is low compared to that of the non-prioritized signal, disturbing background noises can be heard. The probability of occurrence of such disturbances is low, but not influenced.
  • the object of the invention is largely to compensate for the sound changes resulting from the mixing of multi-microphone sound recordings as a result of multipath propagation of sound components.
  • Figure 3 is a general block diagram of an arrangement for carrying out the method according to the invention.
  • Figure 4 is a similar block diagram as in Figure 3, but with the difference that the first summation is extended by a number of other summation stages;
  • FIG. 5 shows a block diagram of a first summing stage provided in FIGS. 3 and 4, and
  • FIG. 6 shows a block diagram of a further summation stage provided in FIG.
  • the reference symbols have the following meanings:
  • an n + 1th summation level 411 spectral values of an n + 1-th sum signal
  • FIG. 3 shows a general block diagram of an arrangement for carrying out the method according to the invention.
  • a first microphone signal 100 and a second microphone signal 101 are each supplied to an associated blocking and spectral transformation unit 320.
  • the supplied microphone signals 100 and 101 are first divided into blocks of time-overlapping signal sections, whereupon the formed blocks undergo Fourier transformation. This results in the spectral values 300 of the first microphone signal 100 and the spectral values 301 of the second microphone signal 101 at the outputs of the blocks 320.
  • the spectral values 300 and 301 are then fed to a first summation stage 310, which converts the spectral values 311 of a first summing stage from the spectral values 300 and 301 Summed signal generated.
  • the spectral values 311 also form the spectral values 399 of a result signal, which are first subjected to an inverse Fourier transformation in a unit 330. The inverse spectral values thus formed are then combined to form blocks. The resulting blocks of time-overlapping signal portions are accumulated into the result signal 199.
  • the block diagram illustrated in FIG. 4 has a similar structure to the block diagram in FIG. 3, but with the essential difference that the spectral values 399 do not simultaneously represent the spectral values 311. Rather, in FIG. 4 between the spectral values 311 and the spectral values 399, a series connection of one or more identical modules 700 is inserted from a respective block formation and spectral transformation unit 320 and an n + 1 th summation stage 410. From the assembly 700 is in Fig. 4 for Simplified only a single assembly 700 shown in the block diagram, which will be described below, where the count index n is the consecutive numbering.
  • the mentioned series connection of assemblies 700 should be understood as meaning that at the beginning of the series connection the spectral values 400 simultaneously form the spectral values of the first sum signal 311 and at the end of the series connection the spectral values 411 also form the spectral values of the result signal 399. In all other sections of the series connection, the spectral values 411 of a summing stage 410 simultaneously form the spectral values 400 of the subsequent summation stage 410.
  • Each block formation and spectral transformation unit 320 of a series-connected module 700 is supplied with an n + 2-th microphone signal 201 in which it is divided into blocks of is divided into temporally overlapping signal sections.
  • the formed blocks of time-overlapping signal sections are Fourier-transformed, resulting in the spectral values 401 of the n + 2-th microphone signal.
  • the spectral values 400 of the n-th sum signal and the spectral values 401 of the n + 2-th microphone signal are then supplied to the n + 1-th summation stage 410, which generates from them the spectral values 411 of the n + 1 th sum signal.
  • FIG. 5 illustrates the details of the first summation stage 310.
  • the spectral values 300 of the first microphone signal 100 and the spectral values 301 of the second microphone signal 101 become an allocation unit
  • the spectral values A (k) of the signal to be prioritized become
  • the spectral values 300 and the spectral values B (k) of the signal 502 which is not to be prioritized are assigned to the spectral values 301.
  • the choice of Priormaschineszuowski determines the spatial impression of the overall acoustic image and is made according to the design requirements.
  • a typical possibility is the signals of those microphones that are intended for recording the overall acoustic image (so-called main microphones) or according to the invention sum signals formed associated with the prioritized signal path and assign the signals of those microphones that are positioned close to the sound sources (so-called support microphones) the non-prioritized signal path.
  • the associated spectral values A (k) of the signal 501 to be prioritized and spectral values B (k) of the signal 502 that is not to be prioritized are then fed to a correction factor value m (k) calculating unit 510 which uses the spectral values A (k) and B (k) the correction factor values m (k) are calculated as output signal 51 1 as follows: Either the correction factor m (k) is calculated as follows:
  • eA (k) Real (A (k)) ⁇ Real (A (k)) + Imag (A (k)) ⁇ Imag (A (k))
  • eA (k) Real (A (k)) ⁇ Real (A (k)) + Imag (A (k)) ⁇ Imag (A (k))
  • eB (k) Real (B (k)) ⁇ Real (B (k)) + Imag (B (k)) ⁇ Imag (B (k))
  • m (k) is the k-th correction factor
  • a (k) is the k-th spectral value of the signal to be prioritized
  • B (k) is the k-th spectral value of the signal which is not to be prioritized
  • the degree L is chosen so that experience has shown that no background noises are perceived. Typically, the degree L is on the order of 0.5. The greater the degree L, the lower the probability of the disturbances, but this also partially reduces the compensation of sound changes determined by the setting of D.
  • the spectral values A (k) of the signal 501 to be prioritized are additionally supplied to a multiplier 520, while the spectral values B (k) of the signal 502 which is not to be prioritized are additionally supplied to an adder 530.
  • the multiplier 520 is supplied with the correction factor values m (k) of the output signal 511 to the calculation unit 510 where they are multiplied by the spectral values A (k) 501 complex (real part and imaginary part).
  • the result values of the multiplier 520 are supplied to the adder 530 where they are added complexly (after real part and imaginary part) with the spectral values B (k) of the non-prioritizing signal 502. This results in the spectral values 311 of the first summation signal of the first summation stage 310.
  • the decisive factor for the prioritization is thus the multiplication of the correction factor m (k) with exactly one of the two summands of the addition performed in the adder 530.
  • the entire signal path of this summand is "prioritized" from the microphone signal input to the adder 530.
  • FIG. 6 shows the details of the n + 1-th summation stage 410.
  • the n + 1-th summation stage 410 is similar in construction to the first summation stage 310, but with the difference that here the allocation unit 500 displays the spectral values 400 of the n-th sum signal and the spectral values 401 of the n + 2-th microphone signal are supplied, and further that the result values of the adder 530 form the spectral values 411 of the n + 1-th sum signal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

The invention relates to dubbing multi-microphone sound recordings, wherein in order to largely compensate for changes in sound due to a multi-path propagation of sound components, the spectral values of overlapping time windows of sampling values are formed from each of a first microphone signal (100) and a second microphone signal (101). The spectral values (300) of the first microphone signal (100) are distributed across the spectral values (301) of the second microphone signal (101) in a first summation stage (310), forming spectral values (311) of a first summation signal, wherein a dynamic correction of the spectral values (300, 301) is performed for one of the two microphone signals (100, 101). Spectral values (399) of a result signal are formed from the spectral values (311) of the first summation signal, and are subjected to an inverse Fourier transform and block consolidation.

Description

VERFAHREN ZUM ABMISCHEN VON MIKROFONSIGNALEN EINER TONAUFNAHME MIT MEHREREN MIKROFONEN  METHOD FOR MIXING MICROPHONE SIGNALS FROM A MULTIPLE MICROPHONE TONE RECORD
BESCHREIBUNG DESCRIPTION
Die Erfindung bezieht sich auf ein Verfahren gemäß dem Oberbegriff des Patentanspruchs 1. Ein derartiges Verfahren ist aus der WO 2004/084 185 AI bekannt. Um bei der Herstellung von Tonaufnahmen für Musikkonserven, Filme, Rundfunksendungen, Schallarchive, Computerspiele, Multimedia-Präsentationen oder Internet-Präsenzen eine ausgedehnte akustische Szenerie zu erfassen, ist es bekannt ("Handbuch der Tonstudiotechnik" von Michael Dickreiter et al, ISBN 978- 3598117657, Seiten 211-212, 230-235, 265-266, 439, 479), mehrere Mikrofone anstelle nur eines einzelnen Mikrofons zu verwenden. Hierfür wird allgemein der Ausdruck "Multimikrofon-Tonaufnahme" gebraucht. Eine ausgedehnte akustische Szenerie kann zum Beispiel ein Konzertsaal mit einem Orchester aus einer Vielzahl von Musikinstrumenten sein. Für die Erfassung der klanglichen Details nimmt man hier jedes einzelne Musikinstrument mit jeweils einem einzelnen, nahe positionierten Mikrofon auf und positioniert zusätzlich für die Erfassung des akustischen Gesamtbildes einschließlich des Nachhalls im Konzertsaal und der Publikumsgeräusche (insbesondere Beifall) weitere Mikrofone in größerer Entfernung.  The invention relates to a method according to the preamble of claim 1. Such a method is known from WO 2004/084 185 AI. It is known ("Handbuch der Tonstudiotechnik" by Michael Dickreiter et al., ISBN 978-3598117657) to capture an extensive acoustic scene in the production of audio recordings for music conserves, films, radio broadcasts, sound archives, computer games, multimedia presentations or Internet presences , Pages 211-212, 230-235, 265-266, 439, 479) to use multiple microphones instead of just a single microphone. For this purpose, the term "multi-microphone sound recording" is generally used. An extended acoustic scene may be, for example, a concert hall with an orchestra of a variety of musical instruments. To record the tonal details, each individual musical instrument is recorded with a single, closely positioned microphone and also positions further microphones at a greater distance to capture the overall acoustic image, including the reverberation in the concert hall and the audience noises (in particular applause).
Ein anderes Beispiel für eine ausgedehnte akustische Szenerie ist ein aus mehreren Schlaginstrumenten bestehendes Schlagzeug, das im Tonstudio aufgenommen wird. Bei der "Multimikrofon-Tonaufnahme" wird in diesem Falle vor den einzelnen Schlaginstrumenten jeweils ein Mikrofon nahe positioniert und es wird ein zusätzliches Mikrofon oberhalb des Schlagzeugers angebracht.  Another example of an extended acoustic scene is a drum kit consisting of several percussion instruments recorded in the recording studio. In the case of the "multi-microphone sound recording", in each case a microphone is positioned in close proximity in front of the individual percussion instruments and an additional microphone is mounted above the percussionist.
Derartige Multimikrofon-Tonaufnahmen ermöglichen es, dass möglichst viele akustische und klangliche Eigenschaften sowohl der Details als auch des Gesamtbildes der Szenerie in hoher Qualität erfasst und ästhetisch befriedigend gestaltbar gemacht werden. Jedes Mikrofonsignal der Vielzahl von Mikrofonen wird in der Regel als Vielspuraufnahme aufgezeichnet. Bei der nachfolgenden Abmischung der Mikrofonsignale erfolgt die weitere gestalterische Arbeit. In Sonderfällen kann auch unmittelbar "live" abgemischt und nur das Ergebnis der Abmischung aufgezeichnet werden. Die gestalterischen Ziele der Abmischung sind in der Regel ein ausgewogenes Verhältnis der Lautstärken aller Schallquellen, ein natürlicher Klang und ein wirklichkeitsnaher räumlicher Eindruck des akustischen Gesamtbildes. Such multimicrophone sound recordings make it possible to record as many acoustic and sound properties of the details as possible as well as of the overall picture of the scenery in high quality and to make them aesthetically pleasing to design. Each microphone signal of the plurality of microphones is usually recorded as multi-track recording. In the subsequent mixing of the microphone signals further creative work is done. In special cases it is also possible to mix "live" immediately and record only the result of the mixdown. The design goals of the mix are usually a balanced ratio of the volumes of all sound sources, a natural sound and a realistic spatial impression of the overall acoustic image.
Bei der herkömmliche Abmischungstechnik in einem Tonmischpult oder in der Mischfunktion von digitalen Tonschnittsystemen erfolgt eine Summierung der zugeführten Mikrofonsignale, ausgeführt von einem Summierer ("Bus"), der eine technische Realisierung einer gewöhnlichen mathematischen Addition ist. In Figur 1 ist beispielhaft eine einzelne Summation im Signalweg eines herkömmlichen Tonmischpults oder digitalen Tonschnittsystems dargestellt. Eine Hintereinanderschaltung von Summationen im Summierer ("Bus") im Signalweg eines herkömmlichen Tonmischpults oder digitalen Tonschnittsystems ist in Figur 2 beispielhaft veranschaulicht. In den Figuren 1 und 2 bedeuten die BezugszeichenIn the conventional mixing technique in a sound mixing console or in the mixing function of digital sound editing systems, summation of the supplied microphone signals is carried out by a summer ("bus"), which is a technical realization of ordinary mathematical addition. FIG. 1 shows by way of example a single summation in the signal path of a conventional sound mixer or digital sound system. A series connection of summations in the summer ("bus") in the signal path of a conventional sound mixer or digital sound system is exemplified in FIG. In Figures 1 and 2, the reference numerals
100 ein erstes Mikrofonsignal 100 a first microphone signal
101 ein zweites Mikro fonsignal  101, a second microphone signal
110 eine auf Addition basierende Summierungsstufe 110 is an addition-based summation stage
111 ein Summensignal 111 a sum signal
199 ein Ergebnissignal  199 a result signal
200 ein n -tes Summensignal  200 an n-th sum signal
201 ein n+2 -tes Mikrofonsignal  201 a n + 2 -th microphone signal
210 eine n+1 -te, auf Addition basierende Summierungsstufe 210 is an n + 1th addition-based summation level
211 ein n+1 -tes Summensignal 211 a n + 1-th sum signal
Bei Multimikrofon-Tonaufnahmen enthalten infolge der unvermeidlichen Mehrwegeausbreitung des Schalls mindestens zwei Mikrofonsignale Schallanteile, die vom Schall ein und derselben Schallquelle herrühren. Da diese Schallanteile infolge der unterschiedlichen Schallwege mit unterschiedlicher Laufzeit an den Mikrofonen eintreffen, entstehen bei herkömmlicher Abmischungstechnik im Summierer Kammfiltereffekte, die als Klangveränderungen hörbar sind und der angestrebten Natürlichkeit des Klanges zuwiderlaufen. Bei herkömmlicher Abmischungstechnik können derartige Klangveränderungen infolge von Kammfiltereffekten durch eine einstellbare Verstärkung und gegebenenfalls eine einstellbare Verzögerung der aufgezeichneten Mikrofonsignale verringert werden. Eine solche Verringerung ist indessen nur in eingeschränktem Maße möglich, wenn eine Mehrwege- Schallausbreitung von mehr als nur einer einzigen Schallquelle vorliegt. In jedem Falle ist aber ein erheblicher Einstellaufwand am Mischpult bzw. digitalen Tonschnittsystem für das Auffinden des besten Kompromisses erforderlich. In multi-microphone sound recordings contain at least two microphone signals due to the unavoidable multipath propagation of sound portions of sound that come from the sound of one and the same sound source. Since these sound components arrive at the microphones as a result of the different sound paths with different transit times, comb filter effects, which are audible as sound changes and run counter to the intended naturalness of the sound, are produced in conventional mixer technology in the summer. In conventional mixing techniques, such sound variations due to comb filter effects can be reduced by adjustable gain and optionally adjustable delay of the recorded microphone signals. However, such a reduction is only possible to a limited extent if a multi-path Sound propagation of more than a single sound source is present. In any case, however, a considerable adjustment effort on the mixer or digital sound system for finding the best compromise is required.
In der älteren DE 10 2008 056 704 ist eine Abwärtsmischung (sogenanntes "Downmixing") für die Erzeugung eines zweikanaligen Tonformates aus einem mehrkanaligen (z.B. fünfkanaligen) Tonformat beschrieben, mit dem Phantomschallquellen abgebildet werden. Hierbei werden jeweils zwei Eingangssignale summiert, wobei eine Gewichtung der spektralen Koeffizienten eines der beiden zu summierenden Eingangssignale mit einem Korrekturfaktor erfolgt; dasjenige Eingangssignal, welches mit dem Korrekturfaktor gewichtet wird, ist gegenüber dem anderen Eingangssignal priorisiert. Die in der DE 10 2008 056 704 beschriebene Bestimmung des Korrekturfaktors führt jedoch dazu, dass in Fällen, wo die Amplitude des priorisierten Signals gegenüber der des nicht-priorisierten Signals gering ist, störende Nebengeräusche hörbar werden können. Die Wahrscheinlichkeit des Auftretens von solchen Störungen ist zwar gering, aber nicht beeinflussbar.  The prior DE 10 2008 056 704 describes downmixing for the generation of a two-channel audio format from a multichannel (e.g., five-channel) audio format that mimics phantom sound sources. In each case, two input signals are summed, wherein a weighting of the spectral coefficients of one of the two input signals to be summed takes place with a correction factor; that input signal which is weighted by the correction factor is prioritized over the other input signal. However, the determination of the correction factor described in DE 10 2008 056 704 means that in cases where the amplitude of the prioritized signal is low compared to that of the non-prioritized signal, disturbing background noises can be heard. The probability of occurrence of such disturbances is low, but not influenced.
Aus der WO 2004/084 185 AI ist es bei einem Verfahren zum Abmischen von Mikrofonsignalen einer Tonaufnahme mit mehren Mikrofonen bekannt, von einem ersten Mikrofonsignal und einem zweiten Mikrofonsignal jeweils die Spektralwerte überlappender Zeitfenster von Abtastwerten zu bilden. Die Spektralwerte des ersten Mikrofonsignals werden auf die Spektralwerte des zweiten Mikrofonsignals in einer ersten Summierungsstufe unter Bildung von Spektralwerten eines ersten Summensignals verteilt, wobei eine dynamische Korrektur der Spektralwerte eines der beiden Mikrofonsignale erfolgt. Aus den Spektralwerten des ersten Summensignals werden Spektralwerte eines Ergebnissignals gebildet, die einer inversen Fourier- Transformation und Blockzusammenführung unterworfen werden. Für jeden Block von Abtastwerten können auf diese Weise individuelle Korrekturfaktoren bestimmt werden. Die dynamische Korrektur durch eine signalabhängige Gewichtung spektraler Koeffizienten anstelle einer gewöhnlichen Addition vermindert unerwünschte Kammfiltereffekte bei der Multimikrofon-Tonabmischung, die in den Summiergliedern des Tonmischpultes oder Tonschnittsystems durch gewöhnliche Additionen entstehen. Indessen sind auch bei diesem Verfahren störende Nebengeräusche hörbar, falls die Amplitude des priorisierten Signals gegenüber der des nicht-priorisierten Signals gering ist. From WO 2004/084 185 A1 it is known in a method for mixing microphone signals of a sound recording with a plurality of microphones to form in each case the spectral values of overlapping time windows of sampled values from a first microphone signal and a second microphone signal. The spectral values of the first microphone signal are distributed to the spectral values of the second microphone signal in a first summation stage to form spectral values of a first sum signal, with a dynamic correction of the spectral values of one of the two microphone signals. From the spectral values of the first sum signal, spectral values of a result signal are formed, which are subjected to inverse Fourier transformation and block merging. For each block of samples, individual correction factors can be determined in this way. The dynamic correction by signal-dependent weighting of spectral coefficients, rather than ordinary addition, reduces undesirable comb filter effects in multimicrophone tone-mixing that arise in the summators of the sound mixer or tone-cutting system by ordinary additions. Meanwhile, even in this method disturbing background noises are audible if the amplitude of the prioritized signal compared to the non-prioritized signal low is.
Die Aufgabe der Erfindung besteht darin, die beim Abmischen von Multimikrofon- Tonaufnahmen infolge einer Mehrwegeausbreitung von Schallanteilen entstehenden Klangveränderungen weitgehend zu kompensieren.  The object of the invention is largely to compensate for the sound changes resulting from the mixing of multi-microphone sound recordings as a result of multipath propagation of sound components.
Die Lösung dieser Aufgabe ergibt sich aus den Merkmalen des Patentanspruchs 1. The solution to this problem arises from the features of claim 1.
Vorteilhafte Ausgestaltungen und Weiterbildungen des erfindungsgemäßen Verfahrens sind in den Unteransprüchen angegeben.  Advantageous embodiments and further developments of the method according to the invention are specified in the subclaims.
Die Erfindung wird anhand der in den Figuren 3 bis 6 gezeigten Ausführungsbeispiele erläutert. Es zeigt  The invention will be explained with reference to the embodiments shown in Figures 3 to 6. It shows
Figur 3 ein generelles Blockschaltbild einer Anordnung zur Durchführung des erfindungsgemäßen Verfahrens; Figure 3 is a general block diagram of an arrangement for carrying out the method according to the invention;
Figur 4 ein ähnliches Blockschaltbild wie in Fig. 3, jedoch mit dem Unterschied, dass die erste Summierungsstufe um eine Anzahl von weiteren Summierungsstufen erweitert ist;  Figure 4 is a similar block diagram as in Figure 3, but with the difference that the first summation is extended by a number of other summation stages;
Figur 5 ein Blockschaltbild einer in den Figuren 3 und 4 vorgesehenen ersten Summierungsstufe, und FIG. 5 shows a block diagram of a first summing stage provided in FIGS. 3 and 4, and
Figur 6 ein Blockschaltbild einer in Figur 4 vorgesehenen weiteren Summierungsstufe. In den Figuren 3 bis 6 haben die Bezugszeichen folgende Bedeutungen:  FIG. 6 shows a block diagram of a further summation stage provided in FIG. In FIGS. 3 to 6, the reference symbols have the following meanings:
100 ein erstes Mikrofonsignal 100 a first microphone signal
101 ein zweites Mikrofonsignal 101, a second microphone signal
199 ein Ergebnissignal 199 a result signal
201 ein n+2 -tes Mikrofonsignal 201 a n + 2 -th microphone signal
300 Spektralwerte des ersten Mikrofonsignals  300 spectral values of the first microphone signal
301 Spektralwerte des zweiten Mikrofonsignals  301 spectral values of the second microphone signal
310 eine erste Summierungsstufe 310 a first summation level
311 Spektralwerte eines ersten Summensignals  311 spectral values of a first sum signal
320 eine Blockbildungs- und Spektraltransformationseinheit 320 a blocking and spectral transformation unit
330 eine inverse Spektraltransformations- und Blockzusammen-führungseinheit 399 Spektralwerte eines Ergebnissignals 330 an inverse spectral transformation and block merging unit 399 spectral values of a result signal
400 Spektralwerte eines n -ten Summensignals 400 spectral values of an n-th sum signal
401 Spektralwerte eines n+2 -ten Mikrofonsignals  401 spectral values of a n + 2-th microphone signal
410 eine n+1 -te Summierungsstufe 411 Spektralwerte eines n+1 -ten Summensignals 410, an n + 1th summation level 411 spectral values of an n + 1-th sum signal
500 Zuordnungseinheit  500 allocation unit
501 Spektralwerte A(k) des zu priorisierenden Signals  501 spectral values A (k) of the signal to be prioritized
502 Spektralwerte B(k) des nicht zu priorisierenden Signals  502 Spectral values B (k) of the signal which is not to be prioritized
510 Berechnungseinheit für Korrekturfaktorwerte 510 Calculation unit for correction factor values
511 Korrekturfaktorwerte m(k)  511 correction factor values m (k)
520 Multiplizierer- Addierer-Einheit 520 multiplier-adder unit
700 eine n -te Baugruppe bestehend aus der Einheit 320 und der n+1 -ten Summierungsstufe 410.  700 an n-th module consisting of the unit 320 and the n + 1-th summation stage 410th
Figur 3 zeigt ein generelles Blockschaltbild einer Anordnung zur Durchführung des erfindungsgemäßen Verfahrens. Ein erstes Mikrofonsignal 100 und ein zweites Mikrofonsignal 101 werden je einer zugeordneten Blockbildungs- und Spektraltransformationseinheit 320 zugeführt. In den Einheiten 320 werden die zugeführten Mikrofonsignale 100 und 101 zunächst in Blöcke von zeitlich überlappenden Signalabschnitten unterteilt, worauf die gebildeten Blöcke einer Fourier- Transformation unterzogen werden. Hieraus ergeben sich die Spektralwerte 300 des ersten Mikrofonsignals 100 beziehungsweise die Spektralwerte 301 des zweiten Mikrofonsignals 101 an den Ausgängen der Blöcke 320. Die Spektralwerte 300 und 301 werden anschließend einer ersten Summierungsstufe 310 zugeführt, welche aus den Spektralwerten 300 und 301 die Spektralwerte 311 eines ersten Summensignals erzeugt. Die Spektralwerte 311 bilden zugleich die Spektralwerte 399 eines Ergebnissignals, welche in einer Einheit 330 zuerst einer inversen Fourier-Transformation unterzogen werden. Die so gebildeten inversen Spektralwerte werden anschließend zu Blöcken zusammengeführt werden. Die daraus entstandenen Blöcke von zeitlich überlappenden Signalabschnitten werden zu dem Ergebnissignal 199 akkumuliert. FIG. 3 shows a general block diagram of an arrangement for carrying out the method according to the invention. A first microphone signal 100 and a second microphone signal 101 are each supplied to an associated blocking and spectral transformation unit 320. In the units 320, the supplied microphone signals 100 and 101 are first divided into blocks of time-overlapping signal sections, whereupon the formed blocks undergo Fourier transformation. This results in the spectral values 300 of the first microphone signal 100 and the spectral values 301 of the second microphone signal 101 at the outputs of the blocks 320. The spectral values 300 and 301 are then fed to a first summation stage 310, which converts the spectral values 311 of a first summing stage from the spectral values 300 and 301 Summed signal generated. The spectral values 311 also form the spectral values 399 of a result signal, which are first subjected to an inverse Fourier transformation in a unit 330. The inverse spectral values thus formed are then combined to form blocks. The resulting blocks of time-overlapping signal portions are accumulated into the result signal 199.
Das in Figur 4 veranschaulichte Blockschaltbild ist ähnlich aufgebaut wie das Blockschaltbild in Figur 3, jedoch mit dem wesentlichen Unterschied, dass die Spektralwerte 399 nicht zugleich die Spektralwerte 311 darstellen. Vielmehr ist in Fig. 4 zwischen den Spektralwerten 311 und den Spektralwerten 399 eine Hintereinanderschaltung von einer oder mehreren gleichen Baugruppen 700 aus je einer Blockbildungs- und Spektraltransformationseinheit 320 und einer n+1 -ten Summierungsstufe 410 eingefügt. Von der Baugruppe 700 ist in Fig. 4 zur Vereinfachung nur eine einzige Baugruppe 700 im Blockschaltbild dargestellt, die nachfolgend beschrieben wird, wobei der Zählindex n der fortlaufenden Nummerierung dient. Die erwähnte Hintereinanderschaltung von Baugruppen 700 ist so zu verstehen, dass am Anfang der Hintereinanderschaltung die Spektralwerte 400 zugleich die Spektralwerte des ersten Summensignals 311 bilden und am Ende der Hintereinanderschaltung die Spektralwerte 411 zugleich die Spektralwerte des Ergebnissignals 399 bilden. Bei allen anderen Abschnitten der Hintereinanderschaltung bilden die Spektralwerte 411 einer Summierungsstufe 410 zugleich die Spektralwerte 400 der nachfolgenden Summierungsstufe 410. Jeder Blockbildungs- und Spektraltransformationseinheit 320 einer Baugruppe 700 der Hintereinanderschaltung wird ein n+2 -tes Mikrofonsignal 201 zugeführt, in der es in Blöcke von zeitlich überlappenden Signalabschnitten unterteilt wird. Die gebildeten Blöcke von zeitlich überlappenden Signalabschnitten werden Fourier-transformiert, woraus sich die Spektralwerte 401 des n+2 -ten Mikrofonsignals ergeben. Die Spektralwerte 400 des n - ten Summensignals und die Spektralwerte 401 des n+2 -ten Mikrofonsignals werden dann der n+1 -ten Summierungsstufe 410 zugeführt, welche aus ihnen die Spektralwerte 411 des n+1 -ten Summensignals erzeugt. The block diagram illustrated in FIG. 4 has a similar structure to the block diagram in FIG. 3, but with the essential difference that the spectral values 399 do not simultaneously represent the spectral values 311. Rather, in FIG. 4 between the spectral values 311 and the spectral values 399, a series connection of one or more identical modules 700 is inserted from a respective block formation and spectral transformation unit 320 and an n + 1 th summation stage 410. From the assembly 700 is in Fig. 4 for Simplified only a single assembly 700 shown in the block diagram, which will be described below, where the count index n is the consecutive numbering. The mentioned series connection of assemblies 700 should be understood as meaning that at the beginning of the series connection the spectral values 400 simultaneously form the spectral values of the first sum signal 311 and at the end of the series connection the spectral values 411 also form the spectral values of the result signal 399. In all other sections of the series connection, the spectral values 411 of a summing stage 410 simultaneously form the spectral values 400 of the subsequent summation stage 410. Each block formation and spectral transformation unit 320 of a series-connected module 700 is supplied with an n + 2-th microphone signal 201 in which it is divided into blocks of is divided into temporally overlapping signal sections. The formed blocks of time-overlapping signal sections are Fourier-transformed, resulting in the spectral values 401 of the n + 2-th microphone signal. The spectral values 400 of the n-th sum signal and the spectral values 401 of the n + 2-th microphone signal are then supplied to the n + 1-th summation stage 410, which generates from them the spectral values 411 of the n + 1 th sum signal.
Figur 5 stellt die Details der ersten Summierungsstufe 310 dar. In der Summierungsstufe 310 werden die Spektralwerte 300 des ersten Mikrofonsignals 100 und die Spektralwerte 301 des zweiten Mikrofonsignals 101 einer Zuordnungseinheit FIG. 5 illustrates the details of the first summation stage 310. In the summation stage 310, the spectral values 300 of the first microphone signal 100 and the spectral values 301 of the second microphone signal 101 become an allocation unit
500 zugeführt, in der je nach getroffener Wahl des Herstellers oder eines Benutzers eine Priorisierung der Ausgangssignale 501, 502 der Einheit 500 erfolgt. Zwei alternative Zuordnungen sind möglich: Bei Priorisierung des Ausgangssignals 501 werden die Spektralwerte A(k) des zu priorisierenden Signals 501 den Spektralwerten 301 und die Spektralwerte B(k) des nicht zu priorisierenden Signals 502 den Spektralwerten 300 zugeordnet. Alternativ werden die Spektralwerte A(k) des zu priorisierenden Signals500, in which, depending on the choice made by the manufacturer or a user, a prioritization of the output signals 501, 502 of the unit 500 takes place. Two alternative assignments are possible: Prioritizing the output signal 501, the spectral values A (k) of the signal 501 to be prioritized are assigned to the spectral values 301 and the spectral values B (k) of the signal 502 to be prioritized to the spectral values 300. Alternatively, the spectral values A (k) of the signal to be prioritized become
501 den Spektralwerten 300 und die Spektralwerte B(k) des nicht zu priorisierenden Signals 502 den Spektralwerten 301 zugeordnet. Die Wahl der Priorisierungszuordnung bestimmt den räumlichen Eindruck des akustischen Gesamtbildes und wird entsprechend den gestalterischen Anforderungen getroffen. Eine typische Möglichkeit ist, die Signale derjenigen Mikrofone, die zur Erfassung des akustischen Gesamtbildes bestimmt sind (so genannte Hauptmikrofone) beziehungsweise die erfindungsgemäß gebildeten Summensignale dem priorisierten Signalweg zuzuordnen und die Signale derjenigen Mikrofone, die nahe an den Schallquellen positioniert sind (so genannte Stützmikrofone) dem nicht priorisierten Signalweg zuzuordnen. Die zugeordneten Spektralwerte A(k) des zu priorisierenden Signals 501 und Spektralwerte B(k) des nicht zu priorisierenden Signals 502 werden dann einer Berechnungseinheit 510 für Korrekturfaktorwerte m(k) zugeführt, welche aus den Spektralwerten A(k) und B(k) die Korrekturfaktorwerte m(k) als Ausgangssignal 51 1 wie folgt berechnet: Entweder wird der Korrekturfaktor m(k) wie folgt berechnet: 501 the spectral values 300 and the spectral values B (k) of the signal 502 which is not to be prioritized are assigned to the spectral values 301. The choice of Priorisierungszuordnung determines the spatial impression of the overall acoustic image and is made according to the design requirements. A typical possibility is the signals of those microphones that are intended for recording the overall acoustic image (so-called main microphones) or according to the invention sum signals formed associated with the prioritized signal path and assign the signals of those microphones that are positioned close to the sound sources (so-called support microphones) the non-prioritized signal path. The associated spectral values A (k) of the signal 501 to be prioritized and spectral values B (k) of the signal 502 that is not to be prioritized are then fed to a correction factor value m (k) calculating unit 510 which uses the spectral values A (k) and B (k) the correction factor values m (k) are calculated as output signal 51 1 as follows: Either the correction factor m (k) is calculated as follows:
eA(k) = Real(A(k)) · Real(A(k)) + Imag(A(k)) · Imag(A(k))  eA (k) = Real (A (k)) · Real (A (k)) + Imag (A (k)) · Imag (A (k))
x(k) = Real(A(k)) · Real(B(k)) + Imag(A(k)) · Imag(B(k))  x (k) = Real (A (k)) · Real (B (k)) + Imag (A (k)) · Imag (B (k))
w(k) = D · x(k)/eA(k)  w (k) = Dx (k) / eA (k)
m(k) = (w(k) 2 + l) iV2) - w(k) m (k) = (w (k) 2 + 1) iV2) - w (k)
oder der Korrekturfaktor m(k) wird wie folgt berechnet: or the correction factor m (k) is calculated as follows:
eA(k) = Real(A(k)) · Real(A(k)) + Imag(A(k)) · Imag(A(k))  eA (k) = Real (A (k)) · Real (A (k)) + Imag (A (k)) · Imag (A (k))
eB(k) = Real(B(k)) · Real(B(k)) + Imag(B(k)) · Imag(B(k))  eB (k) = Real (B (k)) · Real (B (k)) + Imag (B (k)) · Imag (B (k))
x(k) = Real(A(k)) · Real(B(k)) + Imag(A(k)) · Imag(B(k))  x (k) = Real (A (k)) · Real (B (k)) + Imag (A (k)) · Imag (B (k))
w(k) = D · x(k)/(eA(k) + L · eB(k))  w (k) = D × (k) / (eA (k) + L × eB (k))
m(k) = (w(k) 2 + l) iV2) - w(k) m (k) = (w (k) 2 + 1) iV2) - w (k)
wobei in which
m(k) der k -te Korrekturfaktor  m (k) is the k-th correction factor
A(k) der k -te Spektralwert des zu priorisierenden Signals  A (k) is the k-th spectral value of the signal to be prioritized
B(k) der k -te Spektralwert des nicht zu priorisierenden Signals  B (k) is the k-th spectral value of the signal which is not to be prioritized
D der Grad der Kompensation  D the degree of compensation
L der Grad der Begrenzung der Kompensation  L is the degree of limitation of the compensation
bedeuten. mean.
Der Grad D der Kompensation ist ein Zahlenwert, der bestimmt, in welchem Maße die durch Kammfiltereffekte verursachten Klangveränderungen ausgeglichen werden. Er wird je nach den gestalterischen Anforderungen und der gewünschten klanglichen Wirkung gewählt und liegt vorteilhafterweise im Bereich von 0 bis 1. Ist D=0, so entspricht der Klang genau dem der konventionellen Abmischung. Ist D=l , so ergibt sich eine vollständige Entfernung der Kammfilterwirkung. Werte für D zwischen 0 und 1 ergeben entsprechend eine klangliche Wirkung zwischen derjenigen bei D=0 und derjenigen bei D=l . Degree of compensation is a numerical value that determines the extent to which the sound effects caused by comb filter effects are compensated. It is chosen according to the design requirements and the desired tonal effect and is advantageously in the range of 0 to 1. If D = 0, the sound is exactly the same as the conventional mix. If D = 1, this results in a complete removal of the comb filter effect. Values for D between 0 and 1 accordingly give a sound effect between those at D = 0 and those at D = l.
Der Grad L der Begrenzung der Kompensation ist ein Zahlenwert, der bestimmt, in welchem Maße die Wahrscheinlichkeit des Auftretens von störend wahrnehmbaren Nebengeräuschen verringert wird. Diese Wahrscheinlichkeit ist gegeben, wenn die Amplitude des zu priorisierenden Mikrofonsignals gegenüber der des nicht zu priorisierenden Mikrofonsignals gering ist. Es gilt L>=0. Ist L=0, so ergibt sich keine Verringerung der Wahrscheinlichkeit der störenden Nebengeräusche. Der Grad L wird so gewählt, dass erfahrungsgemäß gerade keine Nebengeräusche mehr wahrgenommen werden. Typischerweise liegt der Grad L in der Größenordnung von 0,5. Je größer der Grad L ist, umso geringer wird die Wahrscheinlichkeit der Störungen, jedoch verringert sich damit auch teilweise der durch die Einstellung von D bestimmte Ausgleich von Klangveränderungen.  The degree L of the limitation of the compensation is a numerical value which determines to what extent the probability of the occurrence of disturbing perceptible background noises is reduced. This probability is given if the amplitude of the microphone signal to be prioritized is small compared to that of the microphone signal which is not to be prioritized. It is L> = 0. If L = 0, there is no reduction in the probability of disturbing background noises. The degree L is chosen so that experience has shown that no background noises are perceived. Typically, the degree L is on the order of 0.5. The greater the degree L, the lower the probability of the disturbances, but this also partially reduces the compensation of sound changes determined by the setting of D.
Die Spektralwerte A(k) des zu priorisierenden Signals 501 werden zusätzlich einem Multiplizierer 520 zugeführt, während die Spektralwerte B(k) des nicht zu priorisierenden Signals 502 zusätzlich einem Addierer 530 zugeführt werden. Außerdem werden dem Multiplizierer 520 die Korrekturfaktorwerte m(k) des Ausgangssignals 511 der Berechnungseinheit 510 zugeführt, wo sie mit den Spektralwerten A(k) 501 komplex (nach Realteil und Imaginärteil) multipliziert werden. Die Ergebniswerte des Multiplizierers 520 werden dem Addierer 530 zugeführt, wo sie mit den Spektralwerten B(k) des nicht zu priorisierenden Signals 502 komplex (nach Realteil und Imaginärteil) addiert werden. Hieraus ergeben sich die Spektralwerte 311 des ersten Summensignals der ersten Summierungsstufe 310.  The spectral values A (k) of the signal 501 to be prioritized are additionally supplied to a multiplier 520, while the spectral values B (k) of the signal 502 which is not to be prioritized are additionally supplied to an adder 530. In addition, the multiplier 520 is supplied with the correction factor values m (k) of the output signal 511 to the calculation unit 510 where they are multiplied by the spectral values A (k) 501 complex (real part and imaginary part). The result values of the multiplier 520 are supplied to the adder 530 where they are added complexly (after real part and imaginary part) with the spectral values B (k) of the non-prioritizing signal 502. This results in the spectral values 311 of the first summation signal of the first summation stage 310.
Das Entscheidende für die Priorisierung ist somit die Multiplikation des Korrekturfaktors m(k) mit genau einem der beiden Summanden der im Addierer 530 durchgeführten Addition. Damit wird der gesamte Signalpfad dieses Summanden vom Mikrofonsignaleingang bis zum Addierer 530 "priorisiert".  The decisive factor for the prioritization is thus the multiplication of the correction factor m (k) with exactly one of the two summands of the addition performed in the adder 530. Thus, the entire signal path of this summand is "prioritized" from the microphone signal input to the adder 530.
Figur 6 stellt die Details der n+1 -ten Summierungsstufe 410 dar. Die n+1 -te Summierungsstufe 410 gleicht in ihrem Aufbau der ersten Summierungsstufe 310, jedoch mit dem Unterschied, dass hier der Zuordnungseinheit 500 die Spektralwerte 400 des n-ten Summensignals und die Spektralwerte 401 des n+2 -ten Mikrofonsignals zugeführt werden, ferner, dass die Ergebniswerte des Addierers 530 die Spektralwerte 411 des n+1 -ten Summensignals bilden.  FIG. 6 shows the details of the n + 1-th summation stage 410. The n + 1-th summation stage 410 is similar in construction to the first summation stage 310, but with the difference that here the allocation unit 500 displays the spectral values 400 of the n-th sum signal and the spectral values 401 of the n + 2-th microphone signal are supplied, and further that the result values of the adder 530 form the spectral values 411 of the n + 1-th sum signal.

Claims

PATENT ANSPRÜCHE PATENT CLAIMS
1. Verfahren zum Abmischen von Mikrofonsignalen einer Tonaufnahme mit mehreren Mikrofonen (Multimikrofon-Tonaufnahmen), wobei eine Mehrwegeausbreitung von Schallanteilen gegeben ist, bei dem  1. A method for mixing microphone signals of a sound recording with multiple microphones (multi-microphone sound recordings), wherein a multipath propagation of sound components is given, in which
ein erstes Mikrofonsignal (100) und ein zweites Mikrofonsignal (101) jeweils einer Bildung von Blöcken von Abtastwerten und einer Fourier-Transformation unterzogen werden, wobei die Spektralwerte (300, 301) des jeweiligen Mikrofonsignals (100, 101) gebildet werden,  a first microphone signal (100) and a second microphone signal (101) are each subjected to the formation of blocks of samples and a Fourier transformation, the spectral values (300, 301) of the respective microphone signal (100, 101) being formed,
die Spektralwerte (300) des ersten Mikrofonsignals (100) auf die Spektralwerte the spectral values (300) of the first microphone signal (100) to the spectral values
(301) des zweiten Mikrofonsignals (101) in einer ersten Summierungsstufe (310) unter Bildung von Spektralwerten (311) eines ersten Summensignals verteilt werden, wobei eine dynamische Korrektur der Spektralwerte (300, 301) eines der beiden(301) of the second microphone signal (101) are distributed in a first summation stage (310) to form spectral values (311) of a first sum signal, wherein a dynamic correction of the spectral values (300, 301) of one of the two
Mikrofonsignale (100, 101) erfolgt, Microphone signals (100, 101) takes place,
aus den Spektralwerten (311) des ersten Summensignals Spektralwerte (399) eines from the spectral values (311) of the first sum signal spectral values (399) of a
Ergebnissignals gebildet werden, und Result signal are formed, and
- die Spektralwerte (399) des Ergebnissignals einer inversen Fourier-Transformation und einer Zusammenführung von Blöcken von Abtastwerten unterzogen werden, wobei das Ergebnissignal (199) gebildet wird, the spectral values (399) of the result signal are subjected to an inverse Fourier transformation and a combination of blocks of samples, whereby the result signal (199) is formed,
dadurch gekennzeichnet, dass zur Bildung der Spektralwerte (311) des erstencharacterized in that for forming the spectral values (311) of the first
Summensignals von den Spektralwerten (300) des ersten Mikrofonsignals (100) und den Spektralwerten (301) des zweiten Mikrofonsignals (101) die Spektralwerte (300,Sum signal of the spectral values (300) of the first microphone signal (100) and the spectral values (301) of the second microphone signal (101) the spectral values (300,
301) eines der beiden Signale ausgewählt werden, 301) of one of the two signals are selected
werden, welches gegenüber dem anderen Signal zu priorisieren ist,  which is to be prioritized over the other signal,
dass die Spektralwerte (A(k)) des zu priorisierenden Signals mit jeweils zugehörigenin that the spectral values (A (k)) of the signal to be prioritized are in each case associated with
Korrekturfaktoren m(k) multipliziert werden, und dass die Spektralwerte (B(k)) des nicht zu priorisierenden Signals und die korrigierten Spektralwerte m(k) · A(k) des zu priorisierenden Signals unter Bildung von Spektralwerten eines Ergebnissignals (399) addiert werden. Correction factors m (k) are multiplied, and that the spectral values (B (k)) of the signal not to be prioritized and the corrected spectral values m (k) · A (k) of the signal to be prioritized add together to form spectral values of a result signal (399) become.
2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass die Korrekturfaktoren m(k) folgendermaßen berechnet werden:  2. Method according to claim 1, characterized in that the correction factors m (k) are calculated as follows:
eA(k) = Real(A(k)) · Real(A(k)) + Imag(A(k)) · Imag(A(k))  eA (k) = Real (A (k)) · Real (A (k)) + Imag (A (k)) · Imag (A (k))
x(k) = Real(A(k)) · Real(B(k)) + Imag(A(k)) · Imag(B(k)) w(k) = D · x(k)/eA(k) x (k) = Real (A (k)) · Real (B (k)) + Imag (A (k)) · Imag (B (k)) w (k) = Dx (k) / eA (k)
m(k) = (w(k) 2 + l) iV2) - w(k) m (k) = (w (k) 2 + 1) iV2) - w (k)
oder folgendermaßen berechnet werden: or calculated as follows:
eA(k) = Real(A(k)) · Real(A(k)) + Imag(A(k)) · Imag(A(k))  eA (k) = Real (A (k)) · Real (A (k)) + Imag (A (k)) · Imag (A (k))
eB(k) = Real(B(k)) · Real(B(k)) + Imag(B(k)) · Imag(B(k))  eB (k) = Real (B (k)) · Real (B (k)) + Imag (B (k)) · Imag (B (k))
x(k) = Real(A(k)) · Real(B(k)) + Imag(A(k)) · Imag(B(k))  x (k) = Real (A (k)) · Real (B (k)) + Imag (A (k)) · Imag (B (k))
w(k) = D · x(k)/(eA(k) + L · eB(k))  w (k) = D × (k) / (eA (k) + L × eB (k))
m(k) = (w(k) 2 + l) iV2) - w(k) m (k) = (w (k) 2 + 1) iV2) - w (k)
und and
m(k) der k -te Korrekturfaktor  m (k) is the k-th correction factor
und and
A(k) der k -te Spektralwert des zu priorisierenden Signals  A (k) is the k-th spectral value of the signal to be prioritized
und and
B(k) der k -te Spektralwert des nicht zu priorisierenden Signals  B (k) is the k-th spectral value of the signal which is not to be prioritized
und and
D der Grad der Kompensation  D the degree of compensation
und and
L der Grad der Begrenzung der Kompensation  L is the degree of limitation of the compensation
bedeuten. mean.
3. Verfahren nach Anspruch 1 oder 2, dadurch gekennzeichnet, 3. The method according to claim 1 or 2, characterized
dass die erste Summierungsstufe (310) um eine Anzahl N von weiteren Summierungsstufen (410) erweitert wird, the first summation stage (310) is extended by a number N of further summation stages (410),
dass jeweils in der n+1 -ten Summierungsstufe (410) ein n+2 -tes Mikrofonsignal (201) einer Bildung von Blöcken von Abtastwerten und einer Fourier-Transformation unterzogen wird, wobei die Spektralwerte (401) des n+2 -ten Mikrofonsignals (201) gebildet werden, dass jeweils in der n+1 -ten Summierungsstufe (410) die Spektralwerte (400) des n -ten Summensignals auf die Spektralwerte (401) des n+2 -ten Mikrofonsignals (201) unter Bildung der Spektralwerte (411) eines n+1 -ten Summensignals verteilt werden, wobei eine dynamische Korrektur entweder der Spektralwerte (400) des n -ten Summensignals oder der Spektralwerte (401) des n+2 - ten Mikrofonsignals (201) erfolgt, dass jeweils in der n+1 -ten Summierungsstufe (410) von den Spektralwerten (400) des n-ten Summensignals und den Spektralwerten (401) - Il des n+2 -ten Mikrofonsignals (201) die Spektralwerte (400, 401) eines der beiden Signale ausgewählt werden, welches gegenüber dem anderen der beiden Signale zu priorisieren ist, in each of the n + 1-th summation stage (410) an n + 2-th microphone signal (201) is subjected to the formation of blocks of samples and a Fourier transformation, the spectral values (401) of the n + 2-th microphone signal (201), that in each of the n + 1-th summation stage (410) the spectral values (400) of the n-th sum signal to the spectral values (401) of the n + 2-th microphone signal (201) to form the spectral values ( 411) of an n + 1-th sum signal, wherein a dynamic correction of either the spectral values (400) of the n-th sum signal or the spectral values (401) of the n + 2-th microphone signal (201) takes place, that in the n +1 -th summation stage (410) of the spectral values (400) of the n-th sum signal and the spectral values (401) The spectral values (400, 401) of one of the two signals, which is to be prioritized with respect to the other of the two signals, are selected for the n + 2-th microphone signal (201),
wobei in which
n = [1 ... N] die laufende Nummer der Summierungsstufe  n = [1 ... N] the sequential number of the summation stage
und and
N die Anzahl der erweiternden Summierungsstufen  N is the number of expanding summation levels
bedeuten. mean.
4. Verfahren nach Anspruch 2 oder 3, dadurch gekennzeichnet, dass der Grad D der Kompensation ein Zahlenwert ist, der bestimmt, in welchem Maße die durch 4. The method according to claim 2 or 3, characterized in that the degree D of the compensation is a numerical value which determines to what extent the by
Kammfiltereffekte verursachten Klangveränderungen ausgeglichen werden, wobei der Wert von D je nach den gestalterischen Anforderungen und der gewünschten klanglichen Wirkung gewählt wird. Comb filter effects caused sound changes are compensated, the value of D is selected depending on the design requirements and the desired tonal effect.
5. Verfahren nach Anspruch 4, dadurch gekennzeichnet, dass der Wert für den Grad D im Bereich von 0 bis 1 liegt, wobei für D=0 der Klang genau dem der konventionellen Abmischung entspricht und für D=l sich eine vollständige Entfernung der Kammfilterwirkung ergibt.  5. The method according to claim 4, characterized in that the value for the degree D is in the range of 0 to 1, wherein for D = 0, the sound corresponds exactly to the conventional blend and for D = l results in a complete removal of the comb filter effect ,
6. Verfahren nach einem der Ansprüche 2 oder 3, dadurch gekennzeichnet, dass der Grad L der Begrenzung der Kompensation ein Zahlenwert ist, der bestimmt, in welchem Maße die Wahrscheinlichkeit des Auftretens von störend wahrnehmbaren Nebengeräuschen verringert wird, wobei diese Wahrscheinlichkeit gegeben ist, wenn die Amplitude des zu priorisierenden Mikrofonsignals gegenüber der des nicht zu priorisierenden Mikrofonsignals gering ist.  6. Method according to one of claims 2 or 3, characterized in that the degree L of the limitation of the compensation is a numerical value which determines to what extent the probability of the occurrence of disturbing perceptible background noises is reduced, this probability being given the amplitude of the microphone signal to be prioritized is small compared to that of the microphone signal which is not to be prioritized.
7. Verfahren nach Anspruch 6, dadurch gekennzeichnet, dass der Grad L der Begrenzung der Kompensation größer oder gleich Null ist, wobei für L=0 sich keine 7. The method according to claim 6, characterized in that the degree L of the limitation of the compensation is greater than or equal to zero, wherein for L = 0 no
Verringerung der Wahrscheinlichkeit der störenden Nebengeräusche ergibt und der Grad L so gewählt wird, dass erfahrungsgemäß gerade keine Nebengeräusche mehr wahrgenommen werden. Reduction of the probability of the disturbing noise results and the degree L is chosen so that experience has shown that no noise is perceived.
8. Verfahren nach Anspruch 2, 6 oder 7, dadurch gekennzeichnet, dass der Grad L der Begrenzung der Kompensation in der Größenordnung von 0,5 liegt.  8. The method of claim 2, 6 or 7, characterized in that the degree L of the limitation of the compensation is in the order of 0.5.
EP10779267.3A 2009-11-12 2010-11-02 Method for mixing microphone signals of a recording using multiple microphones Active EP2499843B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE200910052992 DE102009052992B3 (en) 2009-11-12 2009-11-12 Method for mixing microphone signals of a multi-microphone sound recording
PCT/EP2010/066657 WO2011057922A1 (en) 2009-11-12 2010-11-02 Method for dubbing microphone signals of a sound recording having a plurality of microphones

Publications (2)

Publication Number Publication Date
EP2499843A1 true EP2499843A1 (en) 2012-09-19
EP2499843B1 EP2499843B1 (en) 2016-07-13

Family

ID=43571276

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10779267.3A Active EP2499843B1 (en) 2009-11-12 2010-11-02 Method for mixing microphone signals of a recording using multiple microphones

Country Status (8)

Country Link
US (1) US9049531B2 (en)
EP (1) EP2499843B1 (en)
JP (1) JP5812440B2 (en)
KR (1) KR101759976B1 (en)
CN (1) CN102687535B (en)
DE (1) DE102009052992B3 (en)
TW (1) TWI492640B (en)
WO (1) WO2011057922A1 (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ITTO20110890A1 (en) 2011-10-05 2013-04-06 Inst Rundfunktechnik Gmbh INTERPOLATIONSSCHALTUNG ZUM INTERPOLIEREN EINES ERSTEN UND ZWEITEN MIKROFONSIGNALS.
ITTO20120067A1 (en) 2012-01-26 2013-07-27 Inst Rundfunktechnik Gmbh METHOD AND APPARATUS FOR CONVERSION OF A MULTI-CHANNEL AUDIO SIGNAL INTO TWO-CHANNEL AUDIO SIGNAL.
ITTO20120274A1 (en) * 2012-03-27 2013-09-28 Inst Rundfunktechnik Gmbh DEVICE FOR MISSING AT LEAST TWO AUDIO SIGNALS.
ITTO20130028A1 (en) 2013-01-11 2014-07-12 Inst Rundfunktechnik Gmbh MIKROFONANORDNUNG MIT VERBESSERTER RICHTCHARAKTERISTIK
WO2015173422A1 (en) 2014-05-15 2015-11-19 Stormingswiss Sàrl Method and apparatus for generating an upmix from a downmix without residuals
IT201700040732A1 (en) * 2017-04-12 2018-10-12 Inst Rundfunktechnik Gmbh VERFAHREN UND VORRICHTUNG ZUM MISCHEN VON N INFORMATIONSSIGNALEN
EP3963902A4 (en) * 2019-09-24 2022-07-13 Samsung Electronics Co., Ltd. Methods and systems for recording mixed audio signal and reproducing directional audio
CN114449434B (en) * 2022-04-07 2022-08-16 北京荣耀终端有限公司 Microphone calibration method and electronic equipment

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5228093A (en) * 1991-10-24 1993-07-13 Agnello Anthony M Method for mixing source audio signals and an audio signal mixing system
US6154552A (en) * 1997-05-15 2000-11-28 Planning Systems Inc. Hybrid adaptive beamformer
JP4163294B2 (en) * 1998-07-31 2008-10-08 株式会社東芝 Noise suppression processing apparatus and noise suppression processing method
JP3745227B2 (en) 1998-11-16 2006-02-15 ザ・ボード・オブ・トラスティーズ・オブ・ザ・ユニバーシティ・オブ・イリノイ Binaural signal processing technology
EP1081985A3 (en) * 1999-09-01 2006-03-22 Northrop Grumman Corporation Microphone array processing system for noisy multipath environments
US6668062B1 (en) * 2000-05-09 2003-12-23 Gn Resound As FFT-based technique for adaptive directionality of dual microphones
EP1356706A2 (en) * 2000-09-29 2003-10-29 Knowles Electronics, LLC Second order microphone array
GB2375698A (en) * 2001-02-07 2002-11-20 Canon Kk Audio signal processing apparatus
US7315623B2 (en) * 2001-12-04 2008-01-01 Harman Becker Automotive Systems Gmbh Method for supressing surrounding noise in a hands-free device and hands-free device
JP4286637B2 (en) * 2002-11-18 2009-07-01 パナソニック株式会社 Microphone device and playback device
EP1606797B1 (en) * 2003-03-17 2010-11-03 Koninklijke Philips Electronics N.V. Processing of multi-channel signals
DE102004005998B3 (en) * 2004-02-06 2005-05-25 Ruwisch, Dietmar, Dr. Separating sound signals involves Fourier transformation, inverse transformation using filter function dependent on angle of incidence with maximum at preferred angle and combined with frequency spectrum by multiplication
KR101079066B1 (en) 2004-03-01 2011-11-02 돌비 레버러토리즈 라이쎈싱 코오포레이션 Multichannel audio coding
US8275147B2 (en) * 2004-05-05 2012-09-25 Deka Products Limited Partnership Selective shaping of communication signals
US20060147063A1 (en) * 2004-12-22 2006-07-06 Broadcom Corporation Echo cancellation in telephones with multiple microphones
JP4896449B2 (en) * 2005-06-29 2012-03-14 株式会社東芝 Acoustic signal processing method, apparatus and program
DE102006027673A1 (en) 2006-06-14 2007-12-20 Friedrich-Alexander-Universität Erlangen-Nürnberg Signal isolator, method for determining output signals based on microphone signals and computer program
JP4455614B2 (en) * 2007-06-13 2010-04-21 株式会社東芝 Acoustic signal processing method and apparatus
JP2009069181A (en) * 2007-09-10 2009-04-02 Sharp Corp Sound field correction apparatus
KR101434200B1 (en) * 2007-10-01 2014-08-26 삼성전자주식회사 Method and apparatus for identifying sound source from mixed sound
DE102008056704B4 (en) 2008-11-11 2010-11-04 Institut für Rundfunktechnik GmbH Method for generating a backwards compatible sound format

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2011057922A1 *

Also Published As

Publication number Publication date
JP5812440B2 (en) 2015-11-11
EP2499843B1 (en) 2016-07-13
JP2013511178A (en) 2013-03-28
US20120237055A1 (en) 2012-09-20
WO2011057922A1 (en) 2011-05-19
KR101759976B1 (en) 2017-07-20
DE102009052992B3 (en) 2011-03-17
CN102687535A (en) 2012-09-19
TWI492640B (en) 2015-07-11
US9049531B2 (en) 2015-06-02
TW201129115A (en) 2011-08-16
KR20120095971A (en) 2012-08-29
CN102687535B (en) 2015-09-23

Similar Documents

Publication Publication Date Title
DE102009052992B3 (en) Method for mixing microphone signals of a multi-microphone sound recording
EP1844627B1 (en) Device and method for simulating an electromagnetic field synthesis system
DE4134130C2 (en) Device for expanding and balancing sound fields
EP2206113B1 (en) Device and method for generating a multi-channel signal using voice signal processing
DE69827775T2 (en) TONKANALSMISCHUNG
EP1013141B1 (en) Method and device for reproducing a stereophonic audiosignal
EP2402942B1 (en) Method and device for creating an environmental signal
DE4136022C2 (en) Device for expanding and symmetrizing sound fields
DE102010030534A1 (en) Device for changing an audio scene and device for generating a directional function
DE4102080A1 (en) SOUND EFFECT DEVICE
DE19632734A1 (en) Method and device for generating a multi-tone signal from a mono signal
EP0808076B1 (en) Surround sound system
DE102012224454A1 (en) Generation of 3D audio signals
DE10148351B4 (en) Method and device for selecting a sound algorithm
DE102012025016B3 (en) Method for determining at least two individual signals from at least two output signals
DE19900961A1 (en) Method and device for reproducing multi-channel sound signals
EP1518441B1 (en) Device and method for suppressing a feedback
DE10125229A1 (en) Audio system with virtual loudspeakers that are positioned using processor so that listener is at sweet spot
DE102010015630B3 (en) Method for generating a backwards compatible sound format
DE102006059764B4 (en) signal processing
DE4102078C2 (en) Sound effects device for creating reverberation effects
EP1900250B1 (en) Electroacoustic method
DE102020108958A1 (en) Method for presenting a first audio signal while a second audio signal is being presented
DE102020100131A1 (en) Method for processing a multi-channel audio signal
DE102019124285A1 (en) DECORRELATION OF INPUT SIGNALS

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120607

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20130308

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160216

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 813129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160715

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502010012024

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: KATZAROV S.A., CH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160713

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161013

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161114

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161014

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502010012024

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161013

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

26N No opposition filed

Effective date: 20170418

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161102

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 813129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161102

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20161130

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN, MORITZ, DR., DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20101102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20221123

Year of fee payment: 13

Ref country code: FR

Payment date: 20221118

Year of fee payment: 13

Ref country code: DE

Payment date: 20221121

Year of fee payment: 13