EP3105753A1 - Method for the synthetic generation of a digital audio signal - Google Patents
Method for the synthetic generation of a digital audio signalInfo
- Publication number
- EP3105753A1 EP3105753A1 EP15707298.4A EP15707298A EP3105753A1 EP 3105753 A1 EP3105753 A1 EP 3105753A1 EP 15707298 A EP15707298 A EP 15707298A EP 3105753 A1 EP3105753 A1 EP 3105753A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- waveform
- control points
- segment
- values
- amplitude
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000005236 sound signal Effects 0.000 title claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 23
- 230000006870 function Effects 0.000 claims description 51
- 230000008859 change Effects 0.000 claims description 33
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 9
- 230000001419 dependent effect Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 abstract description 14
- 230000009466 transformation Effects 0.000 abstract description 12
- 230000003595 spectral effect Effects 0.000 abstract description 11
- 230000000737 periodic effect Effects 0.000 abstract description 4
- 230000002269 spontaneous effect Effects 0.000 abstract 1
- 230000015572 biosynthetic process Effects 0.000 description 40
- 238000003786 synthesis reaction Methods 0.000 description 39
- 239000011295 pitch Substances 0.000 description 25
- 230000004048 modification Effects 0.000 description 16
- 238000012986 modification Methods 0.000 description 16
- 230000005428 wave function Effects 0.000 description 16
- 230000000694 effects Effects 0.000 description 13
- 230000036961 partial effect Effects 0.000 description 11
- 238000013459 approach Methods 0.000 description 10
- 230000008901 benefit Effects 0.000 description 9
- 238000012800 visualization Methods 0.000 description 9
- 230000002123 temporal effect Effects 0.000 description 8
- 239000000654 additive Substances 0.000 description 6
- 230000000996 additive effect Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 6
- 241001074639 Eucalyptus albens Species 0.000 description 5
- 230000006399 behavior Effects 0.000 description 5
- 238000000354 decomposition reaction Methods 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 239000012634 fragment Substances 0.000 description 3
- 238000007794 visualization technique Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004040 coloring Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000003179 granulation Effects 0.000 description 2
- 238000005469 granulation Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000010363 phase shift Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 208000010201 Exanthema Diseases 0.000 description 1
- 239000004927 clay Substances 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 235000013766 direct food additive Nutrition 0.000 description 1
- 235000012489 doughnuts Nutrition 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 201000005884 exanthem Diseases 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 238000005755 formation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 206010037844 rash Diseases 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
- G10H7/08—Instruments in which the tones are synthesised from a data store, e.g. computer organs by calculating functions or polynomial approximations to evaluate amplitudes at successive sample points of a tone waveform
- G10H7/10—Instruments in which the tones are synthesised from a data store, e.g. computer organs by calculating functions or polynomial approximations to evaluate amplitudes at successive sample points of a tone waveform using coefficients or parameters stored in a memory, e.g. Fourier coefficients
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
- G10H7/08—Instruments in which the tones are synthesised from a data store, e.g. computer organs by calculating functions or polynomial approximations to evaluate amplitudes at successive sample points of a tone waveform
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/02—Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos
- G10H1/04—Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos by additional modulation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
- G10H7/02—Instruments in which the tones are synthesised from a data store, e.g. computer organs in which amplitudes at successive sample points of a tone waveform are stored in one or more memories
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/131—Mathematical functions for musical analysis, processing, synthesis or composition
- G10H2250/165—Polynomials, i.e. musical processing based on the use of polynomials, e.g. distortion function for tube amplifier emulation, filter coefficient calculation, polynomial approximations of waveforms, physical modeling equation solutions
- G10H2250/205—Third order polynomials, occurring, e.g. in vacuum tube distortion modeling
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/541—Details of musical waveform synthesis, i.e. audio waveshape processing from individual wavetable samples, independently of their origin or of the sound they represent
- G10H2250/551—Waveform approximation, e.g. piecewise approximation of sinusoidal or complex waveforms
- G10H2250/565—Polynomial waveform approximation, i.e. using polynomials of third order or higher
Definitions
- the invention relates to a method for the synthetic generation of a digital audio signal by using recurrently calculated amplitude values of a waveform.
- the synthetic generation of digital audio signals is used in the context of sound synthesis for the electronic generation of sounds.
- digital audio signals are used directly after their generation directly for sound synthesis or alternatively deposited for later use in a memory unit.
- the digital audio signal is then used, for example by means of a digital-to-analog converter, to control a loudspeaker unit or the like.
- the sound synthesis is usually used for the purpose of generating sounds that do not occur in nature. Another application concerns the imitation of natural sounds or natural instruments such as piano, guitar or the like. In addition, the sound synthesis also allows the deliberate or random alienation of natural sounds, for example, by their superposition and editing with electronic effects, for creative music design.
- the periodic sampling of a suitably chosen or predetermined waveform or an algorithmically generated signal is usually performed, which as a result a value stream per channel ("result waveform") with a fixed sample rate and a fixed Value resolution (so-called bit depth), whereby one value per channel per output rate step is made available.
- the frequency of the repetitions of the result waveform determines the pitch
- oscillator In order to provide the waveform provided as the basis for the sampling, a so-called oscillator is usually used, which can be modified according to the desired adaptation in terms of pitch, timbre or other musical effects as needed.
- oscillators generate waveforms with many partials, so that subsequent processors and typically a low-pass filter can attenuate or amplify those partials according to the sound designer's wishes (“Subtractive Synthesis.")
- Subtractive Synthesis. In the context of physical modeling, for example, the oscillator assumes the role of
- the oscillator is a sine wave of a particular frequency, amplitude, and phase.Additional synthesis generates complex waveforms by adding many such simple oscillators. Further and a mixture of these tone generation techniques are typically used in the context of a synthesizer.
- the invention has for its object to provide a method for the synthetic generation of a digital audio signal, which allows the user a particularly simple and intuitive access to the change and creative transformation of the sample underlying waveform.
- This object is achieved according to the invention by using recurrently calculated amplitude or magnitude values of a waveform which are determined by a period-phase or frequency-dependent, by approximation or interpolation between a number of amplitude period phase, magnitude frequency or phase
- the control points are variable in their parameter values and / or other attributes by respectively associated control signals, and wherein the approximation of the control points determined by the currently available control signals is used for the calculation of the amplitude or magnitude values becomes.
- the waveform used for generating the audio signal is thus generated on the basis of user-adjustable control points and their amplitude or magnitude values calculated recurring, the control points in the form of nodes specify the approximate course of the wave function and the actual wave function suitable interpolation or approximation between these control points or interpolation points is generated.
- the amplitude, magnitude or phase values of the wave lenfunktion, which are used for the generation of the audio signal are thereby continuously updated in the manner of a cyclic or periodic scanning and in particular as needed (ie, for example, by a user or by control signals caused change in the parameters or attributes of one or more of the control points) new calculated. This allows influencing the wave function and thus of the audio signal generated therefrom by the user in real time, ie during the generation, the audio signal can be changed directly by the user.
- the amplitude, magnitude or phase values of the control points on the one hand and the period phase or frequency values of the control points on the other hand of individually assigned control signals independently and alternatively or additionally particularly preferably also other attributes of the control points each of individually assigned control signals independent of the Amplitude or magnitude values or phase values and / or the period phase or frequency values of the control points are changed, so that the user has individualized modification options for individual parameters of the control points via a corresponding influencing of the control signals.
- This allows a particularly high number of possible variations with regard to the tonal properties of the generated audio signal.
- an attribute of a control point for example, mark this as an anchor point.
- all other control points that lie between this anchor and a next anchor are influenced by the control signal present to the anchor control point, so that the relations of the control point parameters or attributes, after changing the anchor control point, between the anchor control points relative be preserved to the initial state.
- control points and their attributes are in particular intended to specify the course of the waveform approximately, so that by resorting to only a manageable number of parameters (namely, the parameters defining the control points in the manner of coordinates) the coarse course of the waveform can be defined.
- This allows the definition of a continuous waveform based on a comparatively small number of parameters or attributes, which can be rendered in arbitrary resolution during a sampling cycle by evaluation on the basis of the current value of the control signals.
- a time-domain waveform may be processed for purposes of bandlimitation either by oversampling or by Fourier transformation together with band-limited inverse transformation.
- the filter waveform is evaluated at a high sampling rate and multiplicatively distributed, for example, by means of trapezoidal integration to the resulting frequency bands of the magnitudes.
- phase offset which, however, is distributed additive.
- the constructive filter and the constructive phase offset can also be applied without prior transformation of a constructively evaluated time-domain waveform in the sense of a direct additive synthesis.
- the filter is also distributed additive and not multiplicatively.
- the use in real time is achieved by repeating this evaluation process at regular intervals (possibly in the sample rate) taking into account the current state of the control signals and the pitch to be output.
- a band limitation of the pitch to be outputted is automatic if the spectrum, which consists of fewer pitches with increasing pitch, is "oversampled” effectively by increasing "zero-padding” as a result of the inverse transformation.
- Pitch changes between the updates Recovery cycles only apply to audio rate modulation of the pitch and can create artifacts. To minimize this, if necessary, the pitch changing speed can be reduced by, for example, a low-pass filter.
- the approximation of the waveform by the control points can be done preferably by the waveform by interpolation of the control points (ie determination of that waveform that passes under observance of any predetermined boundary conditions, the control points) or approximation of the control points (ie determination of that waveform, which adheres to a given optimization criterion as close as possible to the individual control points) is determined.
- the waveform is particularly preferably composed in segments by linear combination of basic functions, with two adjacent control points specifying the edges and thus the boundary conditions for each segment.
- the waveform is composed of a number of within a period phase - or, if the processing in the frequency domain, within a frequency band - successive, each defined by a segment-specific linear combination of a number of basis functions wave segments, the wave segments at their segment edges over each one of the control points are connected to the respective adjacent wave segment, and wherein - in the case of interpolation - those linear coefficients are determined for each wave segment with which the respective wave segment at its segment edges in the context of the local control point respectively specifiable, variable amplitude or mag and during the recurrent calculation of the values respectively updated linear coefficients for the respective ge wave segment determined and used for the calculation of the values.
- the waveform is thus segmentally composed of a number of time-sequential wave segments, within each wave segment the corresponding "sub-waveform" is constructively or additively generated by linearly superimposing a number of basic wave functions, using standardized, preferably stored in a library or the like
- Sets of basic wave functions such as, for example, the particularly preferred cubic polynomials, amber polynomials, b-spline basis functions or the like
- the respective segment can thus either based on the linear coefficients used in the constructive superimposition of the basis function or on the constructive overlays of Basisfunktion- and control point products are clearly defined and determined in the case of the approximation so that the description and definition of comparatively unconventional or complex waveforms can be clearly defined ff on a limited number of parameters, and the generation and thus also later modulation or modification of the waveform by editing these parameter sets is made possible.
- the variation of the waveform in real time is possible, so that new creative
- the linear coefficients of the basic functions used for the constructive superimposition are particularly preferably selected such that in the respective wave segment a desired (output) partial wave is reproduced largely exactly or at least approximately approximated.
- the output sub-wave which can be specified by a user or predefined or also selected from waveforms stored in a library, is thus decomposed into a superposition from the basis functions.
- the linear coefficients are preferably selected in such a way that the amplitude of the waveform in the respective segment is dependent on its segment. mend change prescribable amplitude margin values. By a change made by the user of these amplitude edge values, a change of the partial waveform in the respective segment can thus be made, since this is defined by the boundary conditions which can be predetermined via the amplitude boundary values.
- a particularly preferred, even more advanced possibility for influencing the change of the waveform by the user is achievable by the linear coefficients are selected for each shaft segment in an advantageous development such that the respective shaft segment at its segment edges each having a predeterminable, changeable edge slope.
- the amplitude values at the segment edges, but also their edge slopes are given as boundary conditions for the decomposition of the partial waveform of the respective segment into the basic functions.
- Amplitude value of the neighboring segment passes, this can also be made available for easy editing by the user by providing as a variable parameter, the angle between the two on the edge of the segment meeting gradients.
- the linear coefficients of the basis functions are advantageously given by a node vector, and in the case of a non-variable node vector the basic functions are not changeable.
- the control points influence the course of the wave segment by overlapping it with the basis functions.
- the approximation guarantees certain properties as well as the degree of continuity of the resulting waveform.
- discontinuities can also be specified.
- both approaches can be carried out in the same way and represent only two different approaches by means of which the values between the control points can be determined.
- function sets such as Bezier functions, B-splines, NURBS or the like may be provided.
- basic functions are polynomial-based functions; it is thus preferably a polynomial decomposition of the waveform segments.
- third-order polynomials or cubic polynomials are used as basic functions.
- alignment with polynomials of the third degree or cubic polynomials can generate a functional behavior in a particularly simple manner, for which only the boundary conditions of the respective segment (amplitude margins, edge slopes in the case of interpolation or continuity in the case of the Ap- proximation) without the need for further parameterization must be resorted to.
- the waveform is composed of a number of wave segments consecutive within a period phase or frequency band, each defined by a segment specific linear combination of a number of basis functions and, in the case of approximation control points, the wave segments at their segment edges near each one of the control points are connected to the respectively adjacent wave segment, and wherein during the recurrent calculation of the values, the linear combination of the current control points is respectively based on the assigned basis functions - or in the case of interpolation the linear combination of the linear coefficients derived from the current control points
- a graphical user interface is particularly preferred for the modification of the waveform and thus also for the generation of the digital audio signal.
- the "constructive synthesis" embodied as explained above makes the waveform and the effects of its modification transparent to the user in a particularly preferred embodiment by displaying the result, ie the waveform, on a suitable display means such as a screen or display, because the user, without the detour of algorithms, directly specifies the resulting waveform and its temporal change, eliminating the limitations of possible resulting waveforms given by algorithms, resulting in a copy of the universe of all imaginable waveforms that can be mapped by the basic functions used ,
- an associated input device via which the control points and their attributes can be changed.
- touch points the user can thus directly access the amplitude. denominations in the edge regions of the segment and on their gradients or possibly also other attributes of the control points.
- the amplitude edge values and / or the edge slopes or the parameters and attributes of the control points are particularly preferably changed over time according to a modulation function stored in a memory unit. This allows the user to specify the temporal development with which the waveform and / or its segments change their state.
- the modulation function is in turn generated by use of recurrently calculated amplitude values of a waveform formed by a period-phase or frequency-dependent, by approximation or interpolation between a number of control points formed by amplitude-period phase or amplitude-frequency value pairs Amplitude curve over a predetermined interval is determined, the control points in their parameter values and / or other attributes are variable by respectively associated control signals, and wherein the calculation of the amplitude values is based on the interpolation or approximation of the control points determined by the currently available control signals.
- the concept of constructive synthesis is also applied to the modulation function per se.
- the linear coefficients are recalculated for the wave segment limited by this.
- This is particularly advantageous in the case of the preferred use of cubic polynomials or third-degree polynomials as basic functions, since these can be unambiguously defined by the parameters "boundary value" and "edge slope".
- polynomials higher than the third degree it is also possible to change coefficients without influencing the edge slope and / or the amplitude edge value.
- further parameters are available for describing and defining the wave segment, for example the curve curvature at the segment edge.
- the boundary conditions are advantageously chosen so that the curve inevitably receives a predetermined cross-segment degree of continuity and a change in the control points thus entails a change in the curve while retaining this degree of continuity.
- Other attributes may, for example, describe a weighting of individual control points with which the degree of continuity of the curve can be influenced in the vicinity of the corresponding control points, up to the extreme at which the curve, as in the interpolation, actually passes through the control points. In all cases, after changing a control point or its attributes, at least the wave segments affected by this control point are re-evaluated.
- the number of segments into which the waveform is divided is preferably selected as needed.
- the number of segments can be changed by inserting or eliminating segment boundaries or control points.
- those sections of the waveform can be combined into a common segment in which uniform or common changes or modifications by the user are provided.
- the number of segments is also a user definable and / or variable by the user parameters.
- the "constructive synthesis” basically describes the waveform as a system that can basically take any form, unlike algorithmic approaches, the waveform becomes superpositioned Basic functions clearly and explicitly defined on the basis of the linear coefficients and control points. A particular advantage is that the result can be changed without the limitations of the algorithm. Effects such as a filter on an output wave can be directly "constructed” and altered in an unusual way.
- a temporal change of the construction and thus the resulting waveform can be achieved by changing the control or touch points over time.
- the motion possibilities include shifting the control points and changing the weight of the control points.
- the C2 continuity guaranteed with equal weighting is a desirable property, whereby the continuity in the vicinity of a control point can be varied by weighting to the near discontinuity.
- a complex waveform may sometimes require a complicated structure of control points and / or their changes.
- the splitting of the desired waveform into individual components, which are constructively generated and added later, makes it possible to simplify such a structure.
- a constructive waveform can be used to alter its own control points and / or attributes, or those of another constructive waveform.
- a constructive and modifiable waveform can be used to map input values to output values, e.g. in a waveshaper. Furthermore, a constructive waveform can also be used as a changeable kernel of a convolution, etc.
- a signal present as a value stream can be converted by (auto) vectorization into a control-point-based form, which in an approximation of the signal results.
- constructive sampling explicitly provides the result in the form of basis functions and their control elements, which in turn can be subjected to further modification and adaptation.
- the waveforms that are created in the constructive synthesis can be output directly in the time domain as a signal. Alternatively, these waveforms can also be interpreted as signals in the frequency domain.
- this signal can be converted back into the time domain. It also makes sense to apply a constructive waveform scaling to an existing magnitude spectrum. This gives the functionality of a filter. The same applies to the phase offset.
- wave elements are preferably constituted by a "constructively synthesized" waveform as described above which is preferably composed of piecewise polynomial functions, such wave elements may be part of a waveform or stand alone as a waveshaper, control signal shaper, spectral filter , Spectral phase offset, or convolution kernel.
- the next higher layer represents components which encapsulate structural wave elements together with certain parameters and interaction possibilities.
- the next higher layer represents waveforms that consist of several components mixed together. These components, as described above, preferably in the nature of standardized components wave elements in superposition of the basic functions decomposed form, from which complex waveforms can be assembled in a particularly simple manner. These waveforms are used as an oscillator for generating the audio signal and as a modulator for generating control signals.
- the parameters of the components and the control points of the wave elements can be influenced by control signals and by external control signals.
- the wave elements as well as all the aforementioned parameters can be specified separately for each channel. Control points can be inserted, removed, moved, rotated, angled or changed in other parameters or attributes.
- constructive synthesis is the so-called “constructive sampling”. Namely, on the basis of the method of generating changing waveforms by means of changeable convolution point-based interpolation / approximation, it may be provided to go backwards from the result. In this case, an automatic image of an already existing, time-varying waveform would have to be derived as an approximation by algorithms.
- the determined control points are then preferably displayed in a display unit or a display and can be processed from there regularly.
- the automatically generated control points can be revised or the underlying (auto) vectorization algorithm can be re-parameterized in order to reproduce the measured result as precisely as possible.
- the result can be selectively changed in subsequent phases in order to generate new timbres based on the audio experience heard.
- the combination of sampling and constructive synthesis results in a novel tool for sound design.
- the time domain maps the waveform analogous to the motion of the loudspeaker diaphragm. That is, it shows the rash of the membrane (negative or positive) in relation to time - this illustration describes the appearance of the waveform.
- DFT Discrete Fourier Transform
- additive synthesis one can transform a signal from the frequency domain into the time domain, which then can directly control the loudspeaker diaphragm again.
- the DFT needs a number x of magnitude and phase values, each magnitude value representing a frequency at a multiple of the pitch to be output and each phase value representing a phase shift (relative to 0 degrees of a cosine). Since the constructive waveform is in continuous form, it can be sampled at random intervals, so that any pitch can be mapped, which can be represented by an even number of samples. For pitches which do not contain an even number of samples, it is then preferable to use oversampling, interpolation or windowing.
- the next higher 2-power number of samples can be calculated, and thus an optimized Fast Fourier Transform (FFT) algorithm can be used.
- FFT Fast Fourier Transform
- the transformation into the time domain is then performed with a high-resolution FFT (or DFT), increasingly by zero-padding, depending on the pitch to be output, which has the effect that the resulting waveforms are increasingly over-sampled for increasing pitches and thus readout without aliasing effects is possible.
- a constructive time-domain waveform can first be transformed into the frequency domain by means of the FFT described above, and then the resulting magnitudes and phases can be modified using further constructional waveforms before the above-described inverse transformation. This then reflects the common practice of, for example, a subtractive synthesizer in which the oscillator (time domain waveform) is modified by a filter (frequency domain waveforms).
- a particularly preferred embodiment of the mentioned concept is the generation of the waveform or waveform segments by additive or constructive superimposition of individual components, which in turn are preferably generated according to the aforementioned concept of linear combination generation from suitable basic functions.
- these can be equipped in a further advantageous embodiment additionally with parameters that allow to set individual components in relation to other components.
- said components are preferably provided in the nature of standardized constituents or building blocks from which the waveforms can be assembled.
- the components thus themselves constitute building blocks or wave elements for the synthesis of the waveform, and in turn are composed of the basic functions in the manner of a superposition.
- the composite of the components waveforms are then preferably used as an oscillator for generating the audio signal and as a modulator for generating control signals.
- the components mentioned are expediently chosen so that results can be formed in a simple manner with you, which typically occur in synthetically generated digital audio signals.
- each signal-generating component (except for the noise component) having an independent frequency, phase and amplitude.
- the frequency is always relative to a fundamental frequency, which is indicated by a control signal from the user.
- wave elements of the components it is particularly preferred to have ready access for easier access (preferably in the manner of a library):
- the oscillator component is intended as a changing constructive wave element, which can be considered as a fundamental oscillator.
- a frequency offset, phase offset and amplitude parameters which can be changed by means of control signals.
- the frequency offset is to be seen here relative to the fundamental frequency to be output.
- several such components occur in the generation of a complex clay.
- This component is preferably in turn of a constructive waveform, which, however, indicates a frequency change of one or more other oscillator or staircase components (target wave elements) that is dependent on the phase of the fundamental frequency.
- a frequency multiplier which multiplies the frequency of the target waveform depending on the fundamental frequency allows the repetitive target wave element to be longitudinally varied over time. As a result, this process is similar to frequency modulation.
- a sweep of the frequency multiplier allows phase synchronization effects to be realized. Since these frequency changes are used in constructive wave generation, any of these effects are band limited, as long as the Fourier transform approach is followed as described above.
- an envelope component Like the displacement components, this component again consists of a constructive waveform that scales the amplitude of the target wave components depending on the phase of the fundamental frequency.
- an envelope can be created that allows frequency modulation of the target wave element phase-synchronized to the fundamental frequency without causing discontinuities at the edges of the fundamental frequency.
- this fundamental wave element is not based on a constructive waveform - it is specified by a step-editor which allows to render shapes with hard edges efficiently.
- a variable 1-pole low-pass filter that can be used to adjust the degree of sharpness that is to be brought into the signal by the hard edges.
- This component produces white or pink noise, giving the opportunity to add a signal without pitch (without a repeating pattern). It also preferably has a sample and hold functionality, which makes it possible to query new noise values only at certain time intervals. This is particularly interesting with regard to the modulator context. Lastly, this component also has a 2-pole low-pass filter with which, e.g. the sample and hold transitions can be softened.
- This component again consists of a constructive waveform, which, however, is applied in the frequency domain. In addition to the control point parameters and their attributes, it has no other parameters. Similar to the shift component and the envelope component, it can be applied to the Oscillator components and the stair components are applied.
- the target wave components are transformed into the frequency domain based on the currently desired pitch, and the filter component waveform, which is preferably pitch-independent, is multiplicatively distributed by preferably trapezoidal integration to the resulting magnitude spectrum.
- This component behaves analogously to the spectral filter component, with the difference that its waveform is additively distributed to the phase spectrum of the target waveform (s).
- This component preferably contains a parameter for the amplitude scaling and / or a parameter for the specification of the so-called clipping behavior of the final mixed-together signal.
- the result of the component summation is the waveform.
- the output of each component can in turn be used to modulate the control points and parameters in the other components.
- this modulation is preferably carried out in audio rates and allows common methods such as FM, AM, also between the components, etc.
- the phases of the components can be synchronized with each other and thus enable e.g. Phase Sync.
- Such a component-based synthesis concept permits a novel, particularly advantageous concept, which is likewise considered to be inventive in its own right, for the visualization of audio signals, which is explained in more detail below.
- Common visualization methods build on the heard result waveform, that is on the audio signal as a whole, with no differentiation between individual components of the signal flow can be made.
- Common visualization methods usually only allow the detection of frequencies, for example with the aid of Fourier transformation, as well as the detection of transients.
- a representation derived from the individual components of the signal flow allows one to "construct" a visualization, taking into account the underlying signal flow of the generated sound.
- a consistent audio stream can be varied for the listener / viewer solely by the visual stimulus, which emphasizes various processes in the composition of the audio stream, resulting in a more precise understanding of the sound of sound design as well as in the context of sound experience is beneficial.
- Such a “constructive visualizer” is not to be understood as an external instrument placed on an audio stream, but as a tool integrated in the audio stream, which allows “guided hearing”.
- the inexperienced ear hears a tone less differentiated than a trained, in addition, the ear can focus on certain areas in a sound. Through the visual stimulus one can now draw attention to details in the sound and "focus” the focusing of all viewers / listeners on a certain element.
- the "constructive visualizer” implements the above two points: First, individual components within the signal flow are tapped, preferably by using the interpolation or approximation method used in the synthesis of the audio signal Visualization a complex, changing and colored toned geometry based on the tapped signal flows.
- the starting points are a three-dimensional space and a definable set of vertices.
- Each vertex has an X, Y, and Z coordinate (dimensions) and is associated with a red, green, and blue value (color channels).
- signals from the signal flow which determine the values of the selected dimension or the color channel can be tapped or determined on the basis of the present parameters. Instead of a tapped or projected signal, either a linear course or a constant value can be specified for this.
- the mesh created from the vertex has the theoretical possibility of accepting any kind of three-dimensional shape and coloring.
- the individual dimensions / color channels can be scaled independently of one another and the entire mesh can be rotated and moved in three-dimensional space around each axis. This scaling, rotation and displacement can also be temporally changed via control signals or control signals.
- the preferred goal is to visualize the change in the tapped and / or projected value streams over time, irrespective of their fundamental frequencies, so that one can work with an entire period in one step and thus determine independently of the frequency.
- the information of the frequency of the signal in the components is preferably provided.
- the frequency information is not explicitly available, it can also be determined analytically.
- a two-dimensional buffer is advantageously used for each dimension and for each color channel - one dimension for the period of the value stream and the second dimension for the time ,
- mapping of the tapped signal to a dimension / color channel buffer can be carried out in the following ways:
- the tapped signal is written in the first row of the buffer. All other rows are moved backwards and the last row is omitted. This can be realized efficiently in the form of a circular buffer.
- the tapped signal is written to the nth row of the buffer, incrementing n every time. If n exceeds the number of rows, it is reset to zero.
- a separate signal is tapped and the values are generated by bilinear interpolation.
- vertex and fragment shaders which are provided by the OpenGL standard.
- Vertex and fragment shaders can access two-dimensional data sources (textures). This possibility is used to create the geometry.
- a dedicated texture is created for each dimension and for each color channel. Each texture has the same size (# x values * # y values) and this also determines the number of available vertices.
- the textures are mapped over the vertex so that each texel (a point in the texture) identifies a vertex.
- the vertex shader can now read a position for each vertex based on the corresponding texel from the texture of each dimension.
- a dimension is linear, e.g. tubular structures are generated.
- the advantages achieved by the invention are in particular that creates by the constructive synthesis by resorting to the segmental decomposition of the waveform in a linear combination of basic function special transparency and handling with respect to the nature of the waveform.
- changes and influences on the waveform also taking into account their future course or behavior, created, which are not possible with previous methods.
- the constructive synthesis now provided can be understood as a "whitebox” method, in which the user has full transparency with regard to the resulting waveform and explicitly specifies its shaping and temporal change.
- the user also acts as a "gray box” creator by defining parameters within the whitebox that are available outside the whitebox.
- the user-defined behavior of the whitebox can be controlled via parameters, as in conventional synthesis methods.
- the user has the option of viewing the whitebox and can understand and change the internal processes in the process of creating the sound and its parameters.
- the advantage of this approach is boundless creative freedom regarding the definition of the waveform whose temporal change. Results can be created which are not intended or even possible with the parameters of the black box method.
- FIG. 1 a synthesizer for synthesizing a digital audio signal
- FIG. 2-9 each show a sequence of a display unit of the synthesizer of FIG. 1 and edited there waveform.
- the synthesizer 1 comprises a central processing unit 2, in particular a computer, in which the processing of a so-called oscillator or a waveform can be performed, which can be modified as required in terms of pitch, timbre or other musical effects as needed.
- the oscillator or waveform is generated from the construction of piecewise basis functions and control points in the context of the system (components, temporal changes, etc) stored as a data set in the memory 4. Since the control points can be continuously evaluated, the resulting construct can also be continuously evaluated. Therefore, it is possible to sample the underlying construct at any frequency and thus generate any pitches.
- the sampling takes place with a constant sample rate, and the sampled values are stored, possibly after spectral band limitation and further processing, with a constant bit depth in a memory 4 and / or output directly as a digital audio signal, which is in a downstream digital Analog converter 10 is converted into an analog audio signal.
- the analog audio signal is then used to drive a downstream speaker unit 12 and supplied to this.
- the synthesizer 1 is specifically designed to allow the user a particularly simple and intuitive access to change and creatively transform the waveform underlying the sample.
- the central unit 2 is a processing unit 20, so in particular an editor assigned, via which a modification or processing of the read in the central unit oscillator or present in the central unit 2 waveform is possible.
- a display unit 22 To the central unit 2 is also a display unit 22, so in particular a screen or a display connected, via which the processing of the present waveform is displayed directly and made comprehensible for the user.
- the processing unit 20 is designed as a separate unit from the display unit. Alternatively, however, it can also be integrated into the display unit 22 in a particularly preferred embodiment, in particular by a design as a touch screen.
- the waveform for processing in the central processing unit 2 is provided in a particularly processing-friendly manner.
- the waveform is divided into a number of time-sequential wave segments so that the overall waveform is obtainable by combining the temporal wave segments (or frequency-domain frequency-domain processing) successive wave segments.
- Each wave segment is simulated in the manner of a mathematical decomposition by a segment-specific linear combination of a number of basis functions and control points, wherein in the embodiment as basic functions, the particularly preferred cubic polynomials, or in other words polynomial functions third order, are used.
- the linear coefficients for each wave segment are selected in the interpolation-based embodiment such that the respective wave segment at its segment edges each predetermined, changeable
- the number of segments in this decomposition of the waveform can be specified by the user and also modified. In particular, it may be considered whether and to what extent there should be or should be sections within the waveform which should be characterized by a specific characteristic or behavior; it may be expedient for the user to associate such individualized sections within the waveform each with its own wave segment, so that a targeted and selective modification of the respective section is made possible.
- Examples of such modifiable waveforms are shown in the form of sequences of screenshots or snapshots of the display unit 22 in FIGS. 2-9.
- the respective waveform shown as an amplitude line 30 comprises the wave segments 32, which merge at their segment edges 34 at so-called touch points 36 into the respectively adjacent wave segment 32.
- the linear coefficients for each shaft segment 32 are selected in a particularly preferred embodiment such that the respective shaft segment 32 has at its segment edges 34 each have a predetermined, variable edge slope.
- the amplitude boundary values and the edge slopes are directly changeable via the processing unit 20, thus in particular via the touch screen, by selecting the grab points 36 and entering the corresponding values via a context-related menu or a context-related editor.
- the amplitude edge values and the edge slopes can be changed in time in addition to the immediate change by the user according to a stored in the memory 4 modulation function.
- a periodic change of the respective parameters in the manner of an oscillation or a linear change in the sense of a continuous Enlargement of the respective parameter or any other changes may be provided.
- the respective modulation function is in turn composed of a number of temporally successive, each defined by a segment-specific linear combination of a number of basis functions and control points wave segments, wherein selected in the exemplary embodiment, the linear coefficients for each wave segment be that the respective wave segment at its segment edges each have predeterminable, variable amplitude edge values and / or edge slopes. After a change in an amplitude margin value and / or a boundary gradient, the linear coefficients for the limited wave segment are recalculated.
- the waveform present in the form of the amplitude line 30 is designed with respect to the x or period axis in the manner of a symmetrical configuration and comprises two wave segments 32 which are connected at their common segment edge 34 via the control or touch point 36 and merge into each other.
- the amplitude line 30 is calculated in each wave segment 32 in each of the temporally successive wave segments 32, from which the waveform is composed, based on a segment-specific cubic polynomial, ie a segment-specific linear combination of a number of polynomials used as basis functions Display unit shown.
- each wave segment 32 of this corresponding proportion of the waveform is mathematically defined and based on a comparatively small number of four coefficients (ie the linear coefficients for the polynomials to the third order). With these, the respective wave segment 32 can be described for the current state, but if necessary also extrapolated into the future.
- the corresponding "sub-waveform" is constructively or additively generated by linearly superposing a number of polynomials provided as base wave functions.
- the linear coefficients for each shaft segment 32 are selected such that the amplitude line 30 in the respective shaft segment 32 at its segment edges 34 each have predeterminable, variable amplitude edge values.
- Am by the in FIG. 2c control point or touch point 36 defined transition point between the adjacent wave segments 32 are selected in the example shown suitable for a steady transition between the adjacent wave segments 32 suitable.
- control or touch point 36 can be moved with the aid of the processing unit 20, or an automatic shift can be specified by means of control menus using a context menu. Accordingly, the linear coefficients of the cubic polynomials in the wave segments 32 are recalculated and determined to correctly reflect the modified design.
- the thus modified waveform is then provided by its mathematical definition of the constructive synthesis for the tone-generating sample.
- FIG. 2 is a modification of the wave function by shifting the control or touch point 36 in the x-direction, corresponding to the time axis of the wave function.
- FIG. 2b shows the wave function after the shift of the control or touch point 36 to the left
- FIG. 2c shows the right.
- Such a shift in the x direction also means that the boundary between the wave segments 32 shifts correspondingly, ie that in terms of time each one of the wave segments 32 after shifting occupies a correspondingly larger proportion of the time interval of the wave function as a whole.
- FIG. 3 shows a modification of the wave function by shifting the control or touch point 36 in the y-direction, corresponding to the amplitude of the wave function.
- FIG. 3b shows the wave function after the shift of the control or touch point 36 upward, FIG.
- such displacement in the y-direction essentially means a corresponding change in the amplitude of the wave function as a whole.
- the linear coefficients of the cubic polynomials in the wave segments 32 are, on the one hand, chosen such that the respective wave segment 32 has at its segment edges 34 the respectively definable amplitude boundary values which may be variable via the control or touch points 36.
- the linear coefficients of the cubic polynomials for each shaft segment 32 but also selected such that the respective shaft segment 32 has at its segment edges 34 each have a predetermined edge slope. This is individually variable by the user, which in the embodiment by appropriate configuration of the editor based on a rotation of the respective control oravaddlings 36, in its entirety or for each segment edge 34 is independent, allows.
- FIG. 4 An example of such a rotation of the control or touch point 36, in which on the edge of the segment 34 between the shaft segments 32 on both sides of the edge slopes are changed accordingly, is in the sequence shown in FIG. 4.
- the sequence gem FIG. Fig. 5 shows an example that the edge slopes on both sides of the segment edge 34 between the wave segments 32 are changed separately from each other. Such a separate change of the edge slopes results in a change in the angle in the control or touch point 36.
- the number of control or touch points 36 and thus the number of wave segments 32, from which the wave function is composed, is also changeable by the user.
- An example of adding or removing Control or touch points 36 and, consequently, the modification of the number of wave segments 32 is gem. FIG. 6 shown.
- modulations of a wave function as such can be edited and changed in an analogous manner. Examples of this are in the sequences gem.
- FIG. FIG. 7 Amplitude modulation of a constructive waveform by shifting a control or touch point 36 of another constructive waveform
- FIG. 8 frequency modulation of a constructive waveform by rotation of a control or touch point 36 of another constructive waveform.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP14154741.4A EP2905774A1 (en) | 2014-02-11 | 2014-02-11 | Method for synthesszing a digital audio signal |
PCT/EP2015/052630 WO2015121194A1 (en) | 2014-02-11 | 2015-02-09 | Method for the synthetic generation of a digital audio signal |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3105753A1 true EP3105753A1 (en) | 2016-12-21 |
EP3105753B1 EP3105753B1 (en) | 2018-01-31 |
Family
ID=50072983
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14154741.4A Withdrawn EP2905774A1 (en) | 2014-02-11 | 2014-02-11 | Method for synthesszing a digital audio signal |
EP15707298.4A Active EP3105753B1 (en) | 2014-02-11 | 2015-02-09 | Method for synthesizing a digital audio signal |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14154741.4A Withdrawn EP2905774A1 (en) | 2014-02-11 | 2014-02-11 | Method for synthesszing a digital audio signal |
Country Status (6)
Country | Link |
---|---|
US (1) | US9741329B2 (en) |
EP (2) | EP2905774A1 (en) |
DK (1) | DK3105753T3 (en) |
ES (1) | ES2667492T3 (en) |
NO (1) | NO3105753T3 (en) |
WO (1) | WO2015121194A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2905774A1 (en) * | 2014-02-11 | 2015-08-12 | JoboMusic GmbH | Method for synthesszing a digital audio signal |
US10565973B2 (en) * | 2018-06-06 | 2020-02-18 | Home Box Office, Inc. | Audio waveform display using mapping function |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4453440A (en) * | 1980-11-28 | 1984-06-12 | Casio Computer Co., Ltd. | Envelope control system for electronic musical instrument |
US5340938A (en) * | 1990-04-23 | 1994-08-23 | Casio Computer Co., Ltd. | Tone generation apparatus with selective assignment of one of tone generation processing modes to tone generation channels |
GB9318524D0 (en) * | 1993-09-07 | 1993-10-20 | Ethymonics Ltd | Tone generator |
US6124542A (en) * | 1999-07-08 | 2000-09-26 | Ati International Srl | Wavefunction sound sampling synthesis |
US8247677B2 (en) * | 2010-06-17 | 2012-08-21 | Ludwig Lester F | Multi-channel data sonification system with partitioned timbre spaces and modulation techniques |
EP2905774A1 (en) * | 2014-02-11 | 2015-08-12 | JoboMusic GmbH | Method for synthesszing a digital audio signal |
-
2014
- 2014-02-11 EP EP14154741.4A patent/EP2905774A1/en not_active Withdrawn
-
2015
- 2015-02-09 EP EP15707298.4A patent/EP3105753B1/en active Active
- 2015-02-09 DK DK15707298.4T patent/DK3105753T3/en active
- 2015-02-09 ES ES15707298.4T patent/ES2667492T3/en active Active
- 2015-02-09 WO PCT/EP2015/052630 patent/WO2015121194A1/en active Application Filing
- 2015-02-09 US US15/117,898 patent/US9741329B2/en active Active
- 2015-02-09 NO NO15707298A patent/NO3105753T3/no unknown
Also Published As
Publication number | Publication date |
---|---|
NO3105753T3 (en) | 2018-06-30 |
DK3105753T3 (en) | 2018-05-07 |
US9741329B2 (en) | 2017-08-22 |
EP3105753B1 (en) | 2018-01-31 |
ES2667492T3 (en) | 2018-05-11 |
EP2905774A1 (en) | 2015-08-12 |
WO2015121194A1 (en) | 2015-08-20 |
US20170011727A1 (en) | 2017-01-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1878308B1 (en) | Device and method for generation and processing of sound effects in spatial audio reproduction systems using a graphical user interface | |
EP2189010B1 (en) | Apparatus and method for determining a component signal with great accuracy | |
US10037186B2 (en) | Multi-channel data sonification employing data-modulated sound timbre classes | |
DE3936693C2 (en) | Device for generating digital audio signals | |
DE2926090C2 (en) | Tone generation process | |
DE19726271C2 (en) | Method and device for simulating machine noises | |
DE102010030534A1 (en) | Device for changing an audio scene and device for generating a directional function | |
WO2006089683A1 (en) | Device and method for simulating an electromagnetic field synthesis system | |
DE69629934T2 (en) | REVERSED TRANSFORM NARROW / BROADBAND TONSYNTHESIS | |
US8037413B2 (en) | Brush tool for audio editing | |
DE60024157T2 (en) | Device and method for entering a style of a presentation | |
EP3105753B1 (en) | Method for synthesizing a digital audio signal | |
DE102011082310A1 (en) | Apparatus, method and electroacoustic system for reverberation time extension | |
US7880748B1 (en) | Audio view using 3-dimensional plot | |
DE2513127A1 (en) | METHOD OF ARTIFICIAL GENERATION OF A MUSICAL SOUND | |
DE3604686C2 (en) | ||
DE4190031B4 (en) | Data compression of decaying musical instrument sounds for a digital scanning system | |
EP1239453B1 (en) | Method and apparatus for generating sound signals | |
DE60130654T2 (en) | METHOD FOR PROCESSING A SIGNAL | |
EP1033035B1 (en) | Unit and method for transforming and displaying acoustic signals | |
DE10302448B4 (en) | Method for synchronized change of the pitch and length of an audio signal | |
EP2228798B1 (en) | Method and device for editing an audio signal | |
DE112020007385T5 (en) | An arbitrary waveform generator with advanced command control | |
DE4425767A1 (en) | Reproducing signals at altered speed | |
DE10145360A1 (en) | Method for converting/recording sound-related data identifies a sound event within this data to generate characteristic parameters by using groups of rules in order to represent moving three-dimensional objects in a space. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20160908 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10H 7/08 20060101AFI20170920BHEP Ipc: G10H 7/02 20060101ALI20170920BHEP Ipc: G10H 1/04 20060101ALI20170920BHEP Ipc: G10H 7/10 20060101ALI20170920BHEP |
|
INTG | Intention to grant announced |
Effective date: 20171017 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: BOCHMANN, JOHANNES |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 968020 Country of ref document: AT Kind code of ref document: T Effective date: 20180215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502015002983 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 4 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20180504 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2667492 Country of ref document: ES Kind code of ref document: T3 Effective date: 20180511 |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: LU Payment date: 20180423 Year of fee payment: 4 |
|
REG | Reference to a national code |
Ref country code: NO Ref legal event code: T2 Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: MC Payment date: 20180419 Year of fee payment: 4 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180430 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180501 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180531 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502015002983 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20181102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190228 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190209 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IE Payment date: 20200219 Year of fee payment: 6 Ref country code: DK Payment date: 20200220 Year of fee payment: 6 Ref country code: NL Payment date: 20200220 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180131 Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20150209 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NO Payment date: 20210218 Year of fee payment: 7 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: EBP Effective date: 20210228 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MM Effective date: 20210301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210228 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210209 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FI Payment date: 20220215 Year of fee payment: 8 Ref country code: AT Payment date: 20220215 Year of fee payment: 8 |
|
REG | Reference to a national code |
Ref country code: NO Ref legal event code: MMEP |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220228 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230217 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SE Payment date: 20230220 Year of fee payment: 9 Ref country code: IT Payment date: 20230228 Year of fee payment: 9 Ref country code: BE Payment date: 20230220 Year of fee payment: 9 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230527 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 968020 Country of ref document: AT Kind code of ref document: T Effective date: 20230209 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230209 Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230209 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20240319 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240228 Year of fee payment: 10 Ref country code: CH Payment date: 20240301 Year of fee payment: 10 Ref country code: GB Payment date: 20240222 Year of fee payment: 10 |