EP3384688B1 - Aufeinanderfolgende dekompositionen von audiofiltern - Google Patents
Aufeinanderfolgende dekompositionen von audiofiltern Download PDFInfo
- Publication number
- EP3384688B1 EP3384688B1 EP16815620.6A EP16815620A EP3384688B1 EP 3384688 B1 EP3384688 B1 EP 3384688B1 EP 16815620 A EP16815620 A EP 16815620A EP 3384688 B1 EP3384688 B1 EP 3384688B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- filters
- individual
- coefficients
- transformed
- weighting coefficients
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000354 decomposition reaction Methods 0.000 title description 14
- 230000006870 function Effects 0.000 claims description 56
- 238000012546 transfer Methods 0.000 claims description 47
- 230000000877 morphologic effect Effects 0.000 claims description 43
- 238000000034 method Methods 0.000 claims description 40
- 239000011159 matrix material Substances 0.000 claims description 27
- 230000009466 transformation Effects 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 8
- 238000001228 spectrum Methods 0.000 claims description 7
- 238000012952 Resampling Methods 0.000 claims description 6
- 230000002123 temporal effect Effects 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 230000010339 dilation Effects 0.000 claims 1
- 210000000624 ear auricle Anatomy 0.000 claims 1
- 238000005259 measurement Methods 0.000 description 19
- 210000003128 head Anatomy 0.000 description 7
- 230000033458 reproduction Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 210000000613 ear canal Anatomy 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000000844 transformation Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000011068 loading method Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 206010011878 Deafness Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 239000002775 capsule Substances 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000007917 intracranial administration Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/007—Two-channel systems in which the audio signals are in digital form
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/307—Frequency adjustment, e.g. tone control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
- H04S7/304—For headphones
Definitions
- the present invention relates to the field of the restitution of sound data.
- telecommunications terminals in particular mobile, for which it is envisaged a sound reproduction with a stereophonic listening system (a headset for example) allowing the listener to position the sound sources in space.
- a stereophonic listening system a headset for example
- the invention uses invariant and stationary linear systems that can be characterized by a set of filters depending on a direction between the sound source and one of the listener's auditory canals.
- This set of filters represents the directivity of the system. Filters can be represented in their time (as an impulse response) or frequency (as a transfer function) form.
- an individual or an artificial head with a microphone at the entrance to each auditory canal are particular cases of such an invariant and stationary linear system.
- the system can be characterized by its transfer functions, specific to each individual.
- the transfer functions define the spatial hearing characteristics of the individual by taking into account in particular the reflections linked to his morphology.
- the transfer functions are classically called transfer functions of the HRTF type for “ Head Related Transfer Function ”, when the filters are given in the frequency domain, and HRIR for “ Head Related Impulse Response ”, when the filters are given in the time domain. . It is possible to go from one representation to another by a Fourier transform.
- HRTF transfer functions are therefore a set of complex values. It is possible to return to real values by taking their respective moduli: we thus obtain the moduli of the HRTF.
- the invention can be generalized to the directivities of systems having different shapes and / or numbers of sensors (for example a mobile telephone with 3 microphones). Without prejudicing the generalization of the invention to any linear system that can be characterized by ORTFs, and in order to facilitate understanding of the invention, the particular case of DTF transfer functions is considered hereinafter. In fact, it is possible to pass from the DTF transfer functions to the HRTF transfer functions by calculating minimum phase filters associated with the DTF transfer functions and by adding a delay to them modeling the propagation delays between the capsules (inter-aural delay by a human ). The personalization of these delays is obtained by other techniques well known and not described here.
- HRTF-type transfer functions One technique using HRTF-type transfer functions is binaural synthesis. This technique is based on the use of so-called “binaural” filters, which reproduce the acoustic transfer functions between the sound source (s) and the auditory canals of the listener. These filters are used to simulate auditory localization cues that allow a listener to locate sound sources in a real listening situation.
- the techniques linked to binaural synthesis are therefore based on a pair of binaural signals which feeds a reproduction system.
- the two binaural signals can be obtained by signal processing, by filtering a monophonic signal by binaural filters which reproduce the properties of the acoustic propagation between the source placed at a given position and each of the listener's ear canals.
- Binaural synthesis can be used for different reproductions such as, for example, a reproduction by means of a headset with two earphones, or by means of two loudspeakers.
- the goal is reconstruction a sound field at the level of the listener's ears practically identical to that induced by real sources in space.
- Binaural filters take into account all the acoustic phenomena which modify the acoustic waves in their path between the source and the auditory canals of the listener. Acoustic phenomena include in particular diffraction from the listener's head and reflections on the auditory pinna and upper torso of the user.
- a quality binaural synthesis therefore relies on binaural filters which best reproduce the acoustic coding that the listener's body naturally produces, taking into account the individual specificities of his morphology.
- the binaural filters represent the acoustic transfer functions or HRTF type transfer functions which model the transformations generated by the listener's torso, head and pinna on the acoustic signal coming from a sound source.
- HRTF transfer functions carry the acoustic imprint of the morphology of the individual on which they were measured.
- the HRTF transfer functions are obtained during a measurement phase.
- a selection of directions which cover more or less finely the whole of the space surrounding the listener is fixed.
- the left and right HRTF transfer functions are measured using microphones inserted at the entrance to the listener's ear canals.
- a listener-centered sphere is defined in this way.
- the measurement must be carried out in an anechoic chamber, or an anechoic chamber, so that only the reflections and acoustic phenomena linked to the listener are taken into account.
- M directions we obtain, for a given listener, a database of 2M HRTF type transfer functions (because two right and left auditory channels) representing, for each auditory canal, each of the positions of the sources. .
- Some individuals thus spend long hours in the laboratory in order to have the acoustic signature associated with their physiognomy analyzed in detail, as well as their ability to perceive sound space in three dimensions. These individuals then benefit from binaural listening shaped from the analysis results, offering comfort and a high quality sound impression.
- a first approach consists in calculating the filters on the basis of the acquisition of the listener's morphology and in particular of his flag.
- I Wen Zhang et al. thus propose to decompose an HRTF transfer function on a basis of independent functions (" Efficient Continuous HRTF Model Using Data Independent Basis Functions: Experimentally Guided Approach ", IEEE Transactions on Audio, Speech and Language Processing, vol. 17, no. 4, May 1, 2009, pages 819-829 ).
- Personalization can also be based on the transformation of non-individual HRTF transfer functions extracted from a database including the morphologies associated with HRTF transfer functions ( "Individualization of spectral indices for binaural synthesis: research and exploitation of interindividual similarities for the adaptation or reconstruction of HRTF", Guillon, P, PhD Thesis, University of Maine, Le Mans, France, 2009 ; see also the patent FR 2958825 ).
- the transformation of the HRTF transfer functions to adapt them to a given individual is then controlled by the comparison of the morphologies of the original flag from the database and the target flag of the given individual. This comparison is based on a technique of matching the three-dimensional meshes of the pavilions. Another method consists in using morphological parameters to create or deform a three-dimensional mesh, which will then be used for a detailed calculation and a numerical simulation of the HRTF transfer functions of the individual, by finite elements of border for example. It is also possible, from the morphological parameters of a given individual, to search in a database a third individual with similar morphological parameters.
- One method of acquiring pinna morphology is to use a three-dimensional scan, but this method is sometimes problematic as it requires both specific hardware and implementation.
- the first approach consists in studying the capacity of the auditors to appropriate generic HRTF transfer functions which are not initially adapted to them.
- the second approach suggests learning by a computer of the reactions of a user participating in an interactive game or responding to an interactive questionnaire. The computer iteratively reconstructs the set of HRTF transfer functions suitable for the user from the observation of his localization performance and / or his responses.
- the present invention improves the situation.
- a first aspect of the invention relates to a method for processing individualized data representative of the directivity of an individualized audio system, according to claim 1.
- the successive decomposition in a first base of N independent components common to all the individuals of the first set, then in a second base of P independent components advantageously makes it possible to compress the stored data.
- the numbers N and P of independent components can be chosen as a function of criteria linked to the size of the data stored and to the precision desired for the filter sets.
- the second basis of P independent components can be a basis of spherical harmonics of order P and the second set of weighting coefficients is a set of spherical coefficients.
- the storage of the sets of filters in the form of morphological data advantageously makes it possible to easily apply transformations in order to adapt the second set of weighting coefficients of an individual from the initial set.
- the initial set can thus be used as a starting point for a quick and non-binding determination of sets of filters for users other than the users of the initial set.
- the transformation can comprise at least the application of a rotation matrix to the set of spherical coefficients associated with the selected individual.
- the method comprises the application of an inverse Fourier transform to the new set of filters prior to the temporal resampling.
- the morphological data relate at least to the auditory pinna of the user.
- the morphological data having the most influence on the filter set associated with an individual are taken into account when determining a new set for a new individual.
- the filters can be transfer functions in the frequency domain (or the modules of these transfer functions), each independent component can be a function having a non-zero spectrum in a frequency band. given, and the given frequency bands may be distinct.
- independent components can be expressed in logarithmic scale of frequencies.
- the modules of the set of filters can be deconvolved by a spatial average of the modules of the set of filters and the N independent components can be determined from the deconvolved modules.
- This embodiment makes it possible to reduce the variance of the filters by eliminating the part common to all the filters and makes it possible to work on real values rather than complex (DTF).
- a second aspect of the invention relates to a computer program product comprising program code instructions recorded on a medium readable by a computer, for the execution of the steps of the method according to the first aspect of the invention.
- a third aspect relates to a device for processing individualized data representative of the directivity of an audio system, according to claim 10.
- the figure 1 is a diagram illustrating the general steps of a data processing method according to one embodiment of the invention.
- a personalized set of filters is obtained.
- the initial set of individuals is a restricted set of individuals for which the solutions of the state of the art could be applied in order to obtain a set of filters personalized for each of the individuals.
- each individual was tested in an anechoic chamber in order to obtain at least one personalized set of filters.
- two sets of custom filters are obtained for each individual, one for each ear canal.
- the sets of filters of the initial set of individuals are stored in a step 102, for example in a memory of a device implementing the method according to the invention.
- Filter sets can be expressed as the coefficients of a matrix.
- HRTF transfer functions in the frequency domain is considered without limitation as sets of filters.
- N independent components common to the sets of filters obtained are determined.
- the decomposition into independent components disclosed in the document “Independent conduct analysis”, Stone JV, 2004, John Wiley & Sons can be applied to the modules of the filters of a set (HRTF transfer functions), the modules being optionally deconvolved (frequency division) by the spatial mean of the set of filters. Such an operation is equivalent to removing from the HRTF transfer functions the frequency components common to all the filters.
- Such deconvolved modules are called DTF hereafter. Modules can be optionally smoothed in order to keep only the perceptually relevant frequency variations.
- any HRTF (or DTF) transfer function modulus of the initial set of individuals can be reconstructed by a linear combination of independent components weighted by weighting coefficients, as shown in Figure figure 2 .
- a first matrix 200 of coefficients w i, j , i varying between 1 and M (2 * M being the total number of directions measured, M filters corresponding to one of the two ears of the listener) and j varying between 1 and N represents the weighting coefficients obtained after decomposition of the filters corresponding to one of the ears of a set on a basis formed of the N independent components.
- a second matrix 201 of coefficients c n, f , with n varying between 1 and N and f varying between 1 and F represents the coefficients of the N independent components, each row corresponding to one of the N independent components.
- a third matrix 202 represents a set of filters (the deconvolved modules of the HRTF transfer functions in the preceding example) for an individual, for an ear, obtained in step 101, and comprises coefficients d m, f , m varying between 1 and M and f varying between 1 and F.
- Each row m of the third matrix 202 represents a filter for a given direction of space, and each column corresponds to a frequency (or a band of frequencies more precisely), translating thus the spectrum of HRTF transfer functions.
- the HRTF transfer function modules can be in logarithmic or linear scale, abscissa or ordinate, which results in four distinct configurations (linear, linear), (logarithmic, linear), (linear, logarithmic) and (logarithmic, logarithmic).
- a logarithmic scale on the abscissa amounts to resampling the spectrum of a transfer function (a row of the matrix 202) with a logarithmic and non-linear frequency step, which more precisely reflects the perceptual functioning of the human ear. (more sensitive in high frequencies than low frequencies).
- a logarithmic scale on the ordinate amounts to considering 20 * log 10 (abs (HRTF)), abs (HRTF) representing the modules of the HRTF transfer functions.
- each row of the second matrix 201 represents an independent component, each coefficient of the row corresponding to the energy of the independent component in a given frequency band.
- the first matrix 200 depends on the azimuth and the elevation (on the ordinate) and on the weights assigned to each independent component (on the abscissa).
- the set of coefficients w m, n for a given column n represents, for an individual, the directivity for an independent component for the component n.
- Each index m corresponding to a measurement for a direction (azimuth (m), elevation (m)).
- the first matrix 200 is determined in a step 104, by decomposing each of the sets of filters obtained in step 101, into the base formed from the N independent components.
- the coefficients of a column of the first matrix 200 represent the values of the weights for an independent component for the different measurement directions. They thus represent a figure of spatial directivity.
- the figure 4 illustrates such figures of spatial directivity for eight individuals of the initial set of individuals, according to one embodiment of the invention. We see on the figure 4 that the spatial directivities look the same from one individual to another and that rotations can be applied to bring these spatial directivities closer together.
- the figure 4 presents in particular the weighting coefficients of the first matrix 200, for each individual, applied to the third independent component (third row of the second matrix 201) for eight different individuals. These are therefore the respective third columns of the first matrices 200 for the eight individuals.
- the columns are re-cut by the same elevation and represented in a three-dimensional way.
- the abscissa corresponds to the azimuth expressed in degrees, and the ordinate corresponds to the elevation in degrees.
- the third dimension is represented by color variations (in shades of gray on the figure 4 ).
- the shades of gray represent the values of the weighting coefficients.
- the figure 4 can thus be interpreted as a set of directivity figures for the third independent components of eight individuals of the initial set.
- each set of weighting coefficients (each first matrix 200 of an individual from the initial set ) in a base of P independent functions in the mathematical sense, for example in a base of spherical harmonics of order P-1, in order to obtain a set of spherical coefficients.
- the choice of the base of spherical harmonics allows the easy application of rotations to the sets of spherical coefficients in order to recalculate a new set of spherical coefficients following a rotation of the measurement reference frame, which is not the case with a basis of independent components in two dimensions.
- the determination of a set of spherical coefficients amounts to carrying out a spatial Fourier transform of the directivities (of a first matrix 200 therefore).
- each set of spherical coefficients obtained in association with an identifier of the individual to which it corresponds.
- the decomposition into spherical harmonics thus makes it possible to fully characterize a set of filters corresponding to the directivity of the ear canal of one of the individuals by means of spherical coefficients cw ic, p which are of dimension P * N, where P-1 is the order of the decomposition into spherical harmonics and N the number of independent components.
- N and P can thus be chosen as a function of a compromise between the level of compression and storage constraints, in order to ensure that the complexity of the HRTF transfer functions is reduced after successive decompositions.
- a second advantage arising from the successive application of a decomposition on a basis of N independent components then on a basis of spherical harmonics is linked to the customization of the transfer functions HRTF or DTF.
- the steps 101 to 106 detailed previously have been applied to an initial set of individuals, the set comprising a limited number of individuals (around fifty for example) due to the complexity linked to the acquisition of the functions of HRTF transfer to step 101.
- the sets of spherical coefficients determined for this restricted number of individuals can also be used to rapidly determine a set of filters for a new individual, not belonging to the initial set.
- the method according to the invention can comprise obtaining current morphological data of a new individual.
- a transformation which can include a simple rotation defined by three axes of rotation ( ⁇ , ⁇ , ⁇ ).
- a homothety To can be applied.
- the transformation parameters can be obtained by comparison between two three-dimensional meshes of two individuals, and more generally by comparison between data morphological data of the individuals of the initial set and the current morphological data of the new individual.
- the parameters ⁇ , ⁇ , ⁇ and ⁇ can also depend on a factor f representing a frequency band or a set of frequency bands.
- morphological data of the individuals of the first set can also be obtained in step 101 described above and then stored in step 102. These morphological data can describe the geometry of the linear system whose directivity is characterized by the set of associated filters.
- No restriction is attached to the means used to obtain the morphological data of the individuals of the initial set as well as the current morphological data.
- they can be obtained by direct measurements on the individual, from photographs or even using the three-dimensional scanner of the Kinect TM type for example.
- the morphological data related to the flag of the individual can be particularly taken into account in determining the transformation parameters. Indeed, the bell is the most influencing factor in the information of HRTF filter sets.
- the current morphological data is compared with the set of morphological data of the individuals of the initial set, with a view to selecting, at a step 109, an individual from the initial set. For example, the individual of the initial set having the parameters closest to the current parameters is selected.
- the morphological data to be stored and compared as being 3D meshes of the pavilions, one can search in the base for the 3D mesh which, after a rotation and a homothety, will be closest to the current 3D mesh. No restriction is attached to the criterion used to characterize the proximity of the morphological parameters.
- a transformation to be applied to the set of spherical coefficients associated with the selected individual is determined from the current morphological data.
- the transformation is determined by determining the first parameters that allow data to be passed current morphological data to the morphological data of the individual selected from the initial set. In the example above, the values of the rotation found in the previous step are used. From these first parameters, are deduced the transformation parameters making it possible to transform the set of filters of the selected individual into a new set of filters.
- such a method amounts to determining a transformation model and its parameters on the sets of filters characterizing the directivities of the systems from a signal point of view, another transformation model and its parameters describing the geometries, shapes or morphologies of the systems, and also to determine a function to match these two models.
- the transformation is then applied to the set of spherical coefficients associated with the selected individual in order to obtain a set of spherical coefficients transformed in a step 111.
- the transformed set of spherical coefficients is stored in association with an identifier of the new individual at a step 112.
- the figure 5 represents device 500 according to one embodiment of the invention.
- the device 500 comprises a random access memory 503 and a processor 502 for storing instructions allowing the implementation of steps 101 to 112 of the method described above with reference to figure 1 .
- the device also comprises a database 504 for storing data intended to be kept after the application of the method, in particular the sets of spherical coefficients, the independent components, and optionally the base of spherical harmonics.
- the device 500 further comprises an input interface 501 intended to receive the sets of filters of the initial set of individuals, and optionally the morphological parameters of the individuals of the initial set and the current morphological parameters.
- the device 500 further comprises an output interface 505 for the transmission of data resulting from the application of the method according to the invention. For example, the output interface can transmit the modified filter set or the transformed spherical coefficient set obtained for the new user.
- the present invention is not limited to the embodiments described above by way of examples; it extends to other variants, included within the scope of the protection defined by the following claims.
- the present invention makes it possible to improve the quality of immersive audio rendering in binaural systems, since it makes it possible to easily obtain a set of filters personalized for an individual from morphological data, without requiring long and expensive measurements on each of the individuals.
- the invention thus applies to communications services including audio conferencing and content distribution services or applications (music, films, games, user interfaces, etc.).
- the present invention allows the compression of the sets of filters (HRTF or DTF for example), which facilitates the storage, exchange or loading thereof.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
Claims (10)
- Verfahren zur Verarbeitung von individualisierten Daten, die für die Richtwirkung eines individualisierten Audiosystems repräsentativ sind, wobei das Verfahren die folgenden Schritte aufweist:- Erhalten (101) für jedes Individuum eines Anfangssatzes von Individuen von mindestens einem personalisierten Satz von binauralen Filtern;- Bestimmen (103) von N unabhängigen Komponenten, die den erhaltenen Filtersätzen gemeinsam sind;- Zerlegen (104) von jedem der erhalten Sätze in einer ersten Basis, die aus den N unabhängigen Komponenten gebildet wird, um für jeden Filtersatz einen ersten Satz von Gewichtungskoeffizienten zu erhalten;- Zerlegen (105) von jedem ersten Satz von Gewichtungskoeffizienten in einer zweiten Basis aus P unabhängigen Komponenten, um einen zweiten Satz von Gewichtungskoeffizienten zu erhalten, wobei die zweite Basis aus P unabhängigen Komponenten eine Basis aus sphärischen Oberwellen der Ordnung P-1 ist und der zweite Satz von Gewichtungskoeffizienten ein Satz von sphärischen Koeffizienten ist;- Speichern (106) von jedem zweiten Satz von Gewichtungskoeffizienten, der in Verbindung mit einer Kennung des Individuums aus dem Anfangssatz von Individuen erhalten wurde;wobei jedes Individuum des Anfangssatzes von Individuen darüber hinaus einem Satz von morphologischen Daten zugeordnet ist, wobei das Verfahren darüber hinaus die folgenden Schritte aufweist:- Erhalten (107) von aktuellen morphologischen Daten eines neuen Individuums;- Auswählen (109) eines Individuums aus dem Anfangssatz durch Vergleich zwischen den aktuellen morphologischen Daten und den Sätzen von morphologischen Daten der Individuen des Anfangssatzes;- Anwenden (111) einer Transformation auf den zweiten Satz von Gewichtungskoeffizienten, der dem ausgewählten Individuum zugeordnet ist, um einen transformierten zweiten Satz von Gewichtungskoeffizienten zu erhalten, wobei der transformierte zweite Satz von Gewichtungskoeffizienten ein Satz von transformierten sphärischen Koeffizienten ist, wobei die Transformation aus den aktuellen morphologischen Daten bestimmt wird, um einen modifizierten Satz von Filtern zu erhalten;- Speichern (112) des zweiten transformierten Satzes von Gewichtungskoeffizienten in Verbindung mit einer Kennung des neuen Individuums.
- Verfahren nach Anspruch 1, wobei die Transformation mindestens das Anwenden einer Rotationsmatrix auf den Satz von sphärischen Koeffizienten aufweist, die dem ausgewählten Individuum zugeordnet sind.
- Verfahren nach Anspruch 2, wobei das Verfahren darüber hinaus die folgenden Schritte aufweist, um den modifizierten Satz von Filtern aus dem transformierten Satz von sphärischen Koeffizienten zu erhalten:- Anwenden einer Homothetie auf N unabhängige Komponenten durch frequenzielle Ausdehnung, wobei die Homothetie aus den aktuellen morphologischen Daten bestimmt wird, um N transformierte unabhängige Komponenten zu erhalten;- Multiplizieren des transformierten Satzes von sphärischen Koeffizienten mit einer Matrix, die aus den N transformierten unabhängigen Komponenten gebildet wird, um den modifizierten Satz von Filtern in Verbindung mit der Kennung des neuen Individuums zu erhalten.
- Verfahren nach Anspruch 2, wobei das Verfahren darüber hinaus die folgenden Schritte aufweist, um den modifizierten Satz von Filtern aus dem Satz von transformierten sphärischen Koeffizienten zu erhalten:- Multiplizieren des transformierten Satzes von sphärischen Koeffizienten mit einer Matrix, die aus den N unabhängigen Komponenten gebildet wird, um einen neuen Satz von Filtern zu erhalten;- Anwenden einer Homothetie durch zeitliches Neuabtasten des neuen Satzes von Filtern, um den modifizierten Satz von Filtern in Verbindung mit der Kennung des neuen Individuums zu erhalten.
- Verfahren nach Anspruch 4, wobei, wenn sich der neue Satz von Filtern im Frequenzbereich befindet, das Verfahren das Anwenden einer inversen FourierTransformation auf den neuen Satz von Filtern vor dem zeitlichen Neuabtasten aufweist.
- Verfahren nach einem der vorhergehenden Ansprüche, wobei die morphologischen Daten sich mindestens auf die Hörmuschel des Benutzers beziehen.
- Verfahren nach einem der vorhergehenden Ansprüche, wobei die binauralen Filter Übertragungsfunktionen im Frequenzbereich sind, wobei jede unabhängige Komponente eine Funktion ist, die ein Nicht-Null-Spektrum in einem gegebenen Frequenzband aufweist, und wobei die gegebenen Frequenzbänder unterschiedlich sind.
- Verfahren nach Anspruch 7, wobei die unabhängigen Komponenten in einer logarithmischen Frequenzskala ausgedrückt werden.
- Computerprogrammprodukt, das Programmcodeanweisungen aufweist, die auf einem computerlesbaren Medium gespeichert sind, um die Schritte des Verfahrens nach einem der Ansprüche 1 bis 8 auszuführen.
- Vorrichtung zur Verarbeitung von individualisierten Daten, die für die Richtwirkung eines Audiosystems repräsentativ sind, wobei die Vorrichtung (500) einen Prozessor aufweist, um:- über eine Eingangsschnittstelle (501) der Vorrichtung für jedes Individuum eines Anfangssatzes von Individuen mindestens einen personalisierten Satz von binauralen Filtern zu erhalten;- N unabhängige Komponenten, die den erhaltenen Filtersätzen gemeinsam sind, zu bestimmen;- jeden der erhaltenen Sätze in einer ersten Basis, die aus den N unabhängigen Komponenten gebildet wird, zu zerlegen, um für jeden Filtersatz einen ersten Satz von Gewichtungskoeffizienten zu erhalten;- jeden ersten Satz von Gewichtungskoeffizienten in einer zweiten Basis aus P unabhängigen Komponenten zu zerlegen, um einen zweiten Satz von Gewichtungskoeffizienten zu erhalten, wobei die zweite Basis aus P unabhängigen Komponenten eine Basis aus sphärischen Oberwellen der Ordnung P-1 ist und der zweite Satz von Gewichtungskoeffizienten ein Satz von sphärischen Koeffizienten ist;- in einem Speicher (504) der Vorrichtung jeden zweiten Satz von Gewichtungskoeffizienten zu speichern, der in Verbindung mit einer Kennung des Individuums aus dem Anfangssatz von Individuen erhalten wurde;wobei jedes Individuum des Anfangssatzes von Individuen darüber hinaus einem Satz von morphologischen Daten zugeordnet ist, wobei der Prozessor darüber hinaus konfiguriert ist, um:- aktuelle morphologische Daten eines neuen Individuums zu erhalten;- ein Individuum aus dem aus dem Anfangssatz durch Vergleich zwischen den aktuellen morphologischen Daten und den Sätzen von morphologischen Daten der Individuen des Anfangssatzes auszuwählen;- eine Transformation auf den Satz von Gewichtungskoeffizienten anzuwenden, die dem ausgewählten Individuum zugeordnet sind, um einen zweiten transformierten Satz von Gewichtungskoeffizienten zu erhalten, wobei der zweite transformierte Satz von Gewichtungskoeffizienten ein transformierter Satz von sphärischen Koeffizienten ist, wobei die Transformation aus den aktuellen morphologischen Daten bestimmt wird, um einen modifizierten Satz von Filtern zu erhalten; und- den zweiten transformierten Satz von Gewichtungskoeffizienten in Verbindung mit einer Kennung des neuen Individuums zu speichern.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1561637A FR3044459A1 (fr) | 2015-12-01 | 2015-12-01 | Decompositions successives de filtres audio |
PCT/FR2016/053153 WO2017093666A1 (fr) | 2015-12-01 | 2016-11-30 | Décompositions successives de filtres audio |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3384688A1 EP3384688A1 (de) | 2018-10-10 |
EP3384688B1 true EP3384688B1 (de) | 2021-02-17 |
Family
ID=55542812
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16815620.6A Active EP3384688B1 (de) | 2015-12-01 | 2016-11-30 | Aufeinanderfolgende dekompositionen von audiofiltern |
Country Status (4)
Country | Link |
---|---|
US (1) | US10555105B2 (de) |
EP (1) | EP3384688B1 (de) |
FR (1) | FR3044459A1 (de) |
WO (1) | WO2017093666A1 (de) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115715470A (zh) | 2019-12-30 | 2023-02-24 | 卡姆希尔公司 | 用于提供空间化声场的方法 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5659619A (en) * | 1994-05-11 | 1997-08-19 | Aureal Semiconductor, Inc. | Three-dimensional virtual audio display employing reduced complexity imaging filters |
FR2958825B1 (fr) * | 2010-04-12 | 2016-04-01 | Arkamys | Procede de selection de filtres hrtf perceptivement optimale dans une base de donnees a partir de parametres morphologiques |
-
2015
- 2015-12-01 FR FR1561637A patent/FR3044459A1/fr active Pending
-
2016
- 2016-11-30 EP EP16815620.6A patent/EP3384688B1/de active Active
- 2016-11-30 US US15/780,948 patent/US10555105B2/en active Active
- 2016-11-30 WO PCT/FR2016/053153 patent/WO2017093666A1/fr active Application Filing
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
WO2017093666A1 (fr) | 2017-06-08 |
EP3384688A1 (de) | 2018-10-10 |
US10555105B2 (en) | 2020-02-04 |
US20180288554A1 (en) | 2018-10-04 |
FR3044459A1 (fr) | 2017-06-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3348079B1 (de) | Verfahren und system zur entwicklung einer an ein individuum angepassten kopfbezogenen übertragungsfunktion | |
EP1836876B1 (de) | Verfahren und vorrichtung zur individualisierung von hrtfs durch modellierung | |
EP2898707B1 (de) | Optimierte kalibrierung eines klangwiedergabesystems mit mehreren lautsprechern | |
EP1992198B1 (de) | Optimierung des binauralen raumklangeffektes durch mehrkanalkodierung | |
EP1563485B1 (de) | Verfahren zur verarbeitung von audiodateien und erfassungsvorrichtung zur anwendung davon | |
EP1946612B1 (de) | Hrtfs-individualisierung durch modellierung mit finiten elementen gekoppelt mit einem korrekturmodell | |
EP1999998B1 (de) | Verfahren zur binauralen synthese unter berücksichtigung eines raumeffekts | |
EP2258119B1 (de) | Verfahren und vorrichtung zur bestimmung von übertragungsfunktionen vom typ hrtf | |
EP2042001B1 (de) | Binaurale spatialisierung kompressionsverschlüsselter tondaten | |
EP2901718B1 (de) | Verfahren und vorrichtung zur wiedergabe eines audiosignals | |
EP1479266B1 (de) | Verfahren und vorrichtung zur steuerung einer anordnung zur wiedergabe eines schallfeldes | |
EP2005420A1 (de) | Einrichtung und verfahren zur codierung durch hauptkomponentenanalyse eines mehrkanaligen audiosignals | |
EP1586220B1 (de) | Verfahren und einrichtung zur steuerung einer wiedergabeeinheitdurch verwendung eines mehrkanalsignals | |
EP3475943B1 (de) | Verfahren zur umwandlung und stereophonen codierung eines dreidimensionalen audiosignals | |
EP3384688B1 (de) | Aufeinanderfolgende dekompositionen von audiofiltern | |
EP3025514B1 (de) | Klangverräumlichung mit raumwirkung | |
FR3065137A1 (fr) | Procede de spatialisation sonore | |
EP3449643B1 (de) | Verfahren und system zum senden eines 360°-audiosignals | |
Duraiswami et al. | Capturing and recreating auditory virtual reality | |
FR3073659A1 (fr) | Modelisation d'ensemble de fonctions de transferts acoustiques propre a un individu, carte son tridimensionnel et systeme de reproduction sonore tridimensionnelle | |
FR2782228A1 (fr) | Dispositif de simulation sonore et procede pour realiser un tel dispositif |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180525 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190326 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: ORANGE |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20200923 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602016052724 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1363097 Country of ref document: AT Kind code of ref document: T Effective date: 20210315 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20210217 |
|
RAP4 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: ORANGE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210518 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210617 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210517 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210517 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1363097 Country of ref document: AT Kind code of ref document: T Effective date: 20210217 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602016052724 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20211118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20211130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20161130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231019 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231019 Year of fee payment: 8 Ref country code: DE Payment date: 20231019 Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210217 |