EP1946612A1 - Hrtfs individualisation by a finite element modelling coupled with a revise model - Google Patents
Hrtfs individualisation by a finite element modelling coupled with a revise modelInfo
- Publication number
- EP1946612A1 EP1946612A1 EP06820237A EP06820237A EP1946612A1 EP 1946612 A1 EP1946612 A1 EP 1946612A1 EP 06820237 A EP06820237 A EP 06820237A EP 06820237 A EP06820237 A EP 06820237A EP 1946612 A1 EP1946612 A1 EP 1946612A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- hrtfs
- individual
- directions
- model
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000877 morphologic effect Effects 0.000 claims abstract description 53
- 230000006870 function Effects 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000005259 measurement Methods 0.000 claims abstract description 36
- 238000012546 transfer Methods 0.000 claims abstract description 26
- 238000013528 artificial neural network Methods 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 15
- 238000009434 installation Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 abstract description 6
- 210000003128 head Anatomy 0.000 description 13
- 210000002569 neuron Anatomy 0.000 description 13
- 238000010200 validation analysis Methods 0.000 description 11
- 210000005069 ears Anatomy 0.000 description 9
- 230000015572 biosynthetic process Effects 0.000 description 7
- 238000003786 synthesis reaction Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 7
- 238000013178 mathematical model Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000013179 statistical model Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 206010011878 Deafness Diseases 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 206010021403 Illusion Diseases 0.000 description 1
- 101710084218 Master replication protein Proteins 0.000 description 1
- 101710112083 Para-Rep C1 Proteins 0.000 description 1
- 101710112078 Para-Rep C2 Proteins 0.000 description 1
- 102100022881 Rab proteins geranylgeranyltransferase component A 1 Human genes 0.000 description 1
- 102100022880 Rab proteins geranylgeranyltransferase component A 2 Human genes 0.000 description 1
- 101710119887 Trans-acting factor B Proteins 0.000 description 1
- 101710119961 Trans-acting factor C Proteins 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 210000000613 ear canal Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 210000002837 heart atrium Anatomy 0.000 description 1
- 238000007917 intracranial administration Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000007620 mathematical function Methods 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 210000003454 tympanic membrane Anatomy 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- the present invention relates to the modeling of individual transfer functions called HRTFs (for "Head Related Transfer Functions"), relating to the hearing of an individual in the three-dimensional space.
- HRTFs for "Head Related Transfer Functions”
- the invention is particularly in the context of telecommunication services offering spatialized sound broadcasting (for example an audio conference between several speakers, a movie trailer).
- spatialized sound broadcasting for example an audio conference between several speakers, a movie trailer.
- the most effective technique for positioning sound sources in space is then binaural synthesis.
- Binaural synthesis is based on the use of so-called "binaural" filters, which reproduce the acoustic transfer functions between the sound source and the listener's ears. These filters are used to simulate auditory location indices, indices that allow a listener to locate sound sources in real listening situations. These filters take into account all the acoustic phenomena (in particular the diffraction by the head, the reflections on the roof of the ear and the top of the torso) which modify the acoustic wave in its path between the source and the ears of the listener. These phenomena vary greatly with the position of the sound source (mainly with its direction) and these variations allow the listener to locate the source in the space.
- Binaural quality synthesis is based on binaural filters that reproduce better the acoustic coding that naturally produces the body of the listener, taking into account the individual specificities of its morphology. When these conditions are not respected, there is a deterioration in performance binaural rendering, which results in particular intracranial perception of sources and confusions forward / backward. The sources at the front are seen at the back and vice versa.
- the binaural techniques described above are applied to the treatment of a 3D sound intended for headset broadcasting to two left and right atria. These techniques aim at reconstructing the sound field at the level of a listener's ears, so that their eardrums perceive a sound field that is virtually identical to that which would have been induced by real sources in 3D space.
- the binaural techniques are based on a pair of binaural signals that respectively feed the two headphones of the headphones. These binaural signals can be obtained in two ways:
- Binaural techniques employing binaural filters define the field of binaural synthesis in an advantageous context of the present invention. Binaural synthesis is based on binaural filters that model the propagation of the acoustic wave between the source and the two ears of the listener. These filters represent acoustic transfer functions called HRTFs that model the transformations generated by the torso, head and horn of the listener on the signal coming from a sound source. At each sound source position is associated a pair of HRTFs (one HRTF for the right ear, one HRTF for the left ear). In addition, HRTFs carry the acoustic fingerprint of the morphology of the individual on which they were measured.
- HRTFs therefore depend not only on the direction of the sound, but also on the individual. They are thus a function of the frequency f, the position ( ⁇ , ⁇ ) of the sound source (where the angle ⁇ represents the azimuth and the angle ⁇ the elevation), of the ear (left or right) and the individual.
- HRTFs are obtained by measurement.
- left and right HRTFs are measured by means of microphones inserted at the entrance of a subject's ear canal. The measurement must be performed in an anechoic chamber (or "deaf room”).
- M directions we obtain, for a given subject, a database of 2M acoustic transfer functions representing each position of the space for each ear.
- the spatialization effect is based on the use of HRTFs which, for optimal performance, must take into account acoustic propagation phenomena between the source and the ears, but also the individual specificities of the morphology of the listener.
- the experimental measurement of HRTFs directly on an individual is, at the moment, the most reliable solution to obtain binaural filters of quality and really individualized (taking into account the individual specificities of the morphology of the individual). It is recalled that it is a matter of measuring the transfer function between a source located at a given position ( ⁇ 1, ⁇ 1) and the two ears of the subject by means of microphones placed at the entrance of the auditory ducts of this person.
- the measurement of HRTFs becomes very difficult, if not impossible, in the context of binaural synthesis applications for the general public.
- the measurement of HRTFs actually poses at least three main problems: •
- the measurement of HRTF itself is difficult to implement because it requires specific equipment.
- the measurement must be performed in an anechoic chamber. It also requires a mechanical device to move and control the measurement speaker to perform measurements for a large number of directions evenly distributed in azimuth and elevation around the listener.
- the measurement procedure as a whole is painful for the subject, because of the constraints imposed on the subject by the measuring system and because of the duration of the measurement.
- a second problem is the need to measure HRTFs in a large number of directions to provide sufficient and homogeneous spatial sampling of the 3D sphere surrounding the listener. Plus the number measured directions, the longer the duration of the measurement, which increases the discomfort of the subject.
- a third problem is the measurement of a particular individual. To offer a binaural performance to any individual implies using its own HRTFs, which must have been measured beforehand, which is generally impossible.
- this document also provides for enriching the morphological data of an individual, at the input of the model, by some HRTFs measured on this individual and in respective respective directions.
- the present invention aims at a method for modeling HRTFs transfer functions specific to an individual, in which there is provided: an initial stage of model constitution in which: a) a first database including a plurality of HRTFs measured in a multiplicity of directions of space and for a plurality of individuals, b) a second database is constituted including own and respective morphological parameters of said plurality of individuals, c) from said morphological parameters of the second database, a finite element modeling is applied to obtain a third database comprising modeled, clean and respective HRTFs of said plurality of individuals, for at least a part of said multiplicity of directions, d) by comparison and learning about the data of the first and third databases, we build a correct model if clean to providing modeled and adjusted HRTFs for said multiplicity of directions, * and a current step of determining HRTFs in said plurality of directions, for any individual, wherein: e) measuring morphological parameters of any individual, and ) Modeled and corrected HRTFs of the individual are obtained
- the present invention intends to take advantage of the advantages of the technique described in document FR-2 851 877, according to which it is possible to model, at least roughly, the HRTFs of an individual for which an appropriate set of parameters has been measured.
- morphological It is typically a finite element modeling, which amounts to estimating, as a function of their original direction, the disturbances that the acoustic waves undergo when they encounter an obstacle corresponding to the bust of the individual.
- FR-2 851 877 proposes to also locate at least the position of an ear on the head of the individual and preferably the shape of the flag of the ear too.
- the quality of the HRTFs thus modeled remained to perfect and the present invention proposes for this purpose to apply a corrective model, setting advantageously, a network of artificial neurons is used, in particular in step d) model constitution of the above method.
- the measurement conditions of the morphological parameters are substantially reproducible at least between the stage of constitution of the model and the current stage carried out on any individual. It is also preferable that the simplified geometric model, as well as the finite element model, be reproducible.
- an installation can be provided for estimating HRTFs transfer functions specific to an individual, comprising:
- a processing unit capable of evaluating the HRTFs of the individual in a multiplicity of spatial directions by applying to the morphological parameters of the individual a finite element modeling and a corrective model based on a learning and advantageously putting in a network of artificial neurons.
- the present invention also aims at such an installation.
- the installation can be equipped with means of shooting, according to at least two different angles (for example of face and profile), the bust at least of an individual to deduce general dimensions of his head, of his torso, or others.
- the cabin may comprise, in a preferred embodiment, a measurement standard so that the shots show, with the bust of the individual, the measurement standard.
- Means shape recognition for example, can then be used to measure the morphological parameters involved in the modeling.
- finite element HRTF modeling is more efficient in some particular directions, in that for these directions finite element HRTFs are closer to the measured HRTFs than in the other directions. regardless of the individual. Thus, at the end of finite element modeling, only those best-modeled HRTFs corresponding to preferred directions can be retained and the comparison made only on these privileged directions. On the other hand, we will conduct learning on all the multiplicity of directions of space.
- the model constitution step from said morphological parameters of the second database and by comparison with the measured HRTFs of the first database, one selects directions privileged space models in which finite element modeling provides modeled HRTFs close to the measured HRTFs in these preferred directions, and
- step c) from said morphological parameters of the second database, finite element modeling is applied to obtain a third database comprising modeled, clean and respective HRTFs of said plurality of individuals, following said privileged directions,
- step d) by comparison and learning on the data of the first and third databases, a corrective model is constructed that is capable of yielding modeled and adjusted HRTFs for the multiplicity of directions.
- the present invention also relates to a computer program product, intended to be stored in a memory of a processing unit or on a removable support adapted to cooperate with a reader of said processing unit, or intended to be transmitted from a server to said processing unit.
- the program includes instructions in the form of computer code for constructing a learning-based model advantageously implementing an artificial neural network, capable of providing HRTFs transfer functions of an individual for a multiplicity of directions, from 'a set of measurements, made on this individual, of morphological parameters of this individual.
- the program then implements, from a first database including a plurality of HRTFs along a plurality of spatial directions and for a plurality of individuals, and a second database containing morphological parameters of these individuals, at least one finite element modeling, followed by a comparison / learning phase.
- the present invention also relates to a second computer program product, intended to be stored in a memory of a processing unit or on a removable support adapted to cooperate with a reader of said processing unit, or intended to be transmitted from a server to said processing unit.
- the program includes instructions in computer code form for implementing a learning-based model advantageously implementing an artificial neural network, which model is capable of providing HRTFs transfer functions of an individual for a multiplicity of directions, from a set of measurements made on this individual, of morphological parameters of this individual.
- the first program described above allows to build the model, while the second program consists of computer instructions representing the model itself.
- FIG. 1 diagrammatically illustrates the main steps of the process within the meaning of the invention
- FIG. 2 diagrammatically illustrates the operating steps of a model implementing an artificial neural network, which can then correspond to a flowchart schematically showing the progress of the second computer program described above,
- FIG. 3 schematically illustrates the steps of construction of the model, which may then correspond to a flowchart schematically showing the progress of the first computer program described above,
- FIG. 4a schematically illustrates the first model constitution step in a method according to the invention
- FIG. 4b schematically illustrates the current step using the model constituted in a process in the sense of the invention
- FIG. 4c schematically illustrates an advantageous embodiment for constituting the aforementioned model
- the interest of the mathematical model lies in the use of input parameters that it is easy to acquire for any individual, bearing in mind, however, that their relationship to the transfer function is not necessarily direct or obvious.
- the mathematical model must in particular be able to extract more or less hidden information in the input parameters in order to deduce the desired transfer function.
- the method of the invention is essentially based on two points:
- the mathematical model of HRTFs is based on a function F for expressing an HRTF from a given number of input parameters.
- the input parameters are grouped into a vector X (Xe ⁇ T me K) which therefore constitutes the input vector of the function F.
- the output vector of the function is an HRTF which is represented by a vector Y (Ye ⁇ ne K).
- this vector Y may consist of frequency coefficients describing the spectrum modulus of the transfer function defined by the HRTF.
- Y may consist of:
- the function F is therefore a function of 9T in SR ".
- the input vector X of the model contains mainly information relating to:
- an HRTF preferably in the form of an azimuth angle ( ⁇ ) and an elevation angle ( ⁇ )
- - and "individual" parameters such as HRTFs estimated from the morphological parameters of the individual and by finite element modeling in all or only a few directions of space, as will be seen later
- these parameters Individuals are intended to provide the model with information relating to the specificities of the individual whose HRTFs are to be calculated.
- the output vector Y of the model consists of coefficients associated with a given representation of an HRTF.
- the vector Y may correspond to the frequency coefficients describing the spectrum modulus of an HRTF, but other representations may be considered (principal component analysis, HR filter, or others).
- the model is applied here for correction and optionally interpolation purposes.
- Morphological parameters such as the dimensions of the head Dim H and / or the torso Dim ⁇ of an individual are measured on this individual (step E10).
- HRTFs estimated HRTF g (0j, 0j) are deduced for all or part of the directions of space (step E12).
- the corrective model based on an artificial neural network is then used (step E13) to calculate HRTF corrected HRTF c (0i, 0j) of this individual in all directions (over 360 °) covering the entire 3D sphere (step E 14 ), and this, by comparison with a first database of real HRTFs measurements of the same individual (denoted HRTF m (0i, 0j)) throughout the 3D sphere (step E15 of Figure 1).
- the previously estimated HRTFs are thus used as input parameters of the correction model of step E13, and the previously measured HRTFs E15 are used as input comparison parameters also of the correction model of step E13.
- modeling based on an artificial neural network consists essentially of:
- the method of the invention is preferably based on statistical learning algorithms and, in a preferred embodiment, on network type algorithms. artificial neurons. These algorithms are briefly presented below.
- Statistical learning algorithms are tools for predicting statistical processes. They have been used successfully for the prediction of processes for which several explanatory variables can be identified. Artificial neural networks define a particular category of these algorithms. The interest of neural networks lies in their ability to capture high-level dependencies, that is, dependencies that involve multiple variables at once. The process prediction takes advantage of the knowledge and exploitation of high-level dependencies. There is a wide variety of application domains of neural networks, especially in financial techniques to predict market fluctuations, in pharmaceuticals, in the banking field for the detection of credit card fraud, in marketing to predict behavior. consumers, or others. Neural networks are often considered as universal predictors, in the sense that they are capable of predicting any data from variables explanatory, since the number of hidden units is sufficient. In other words, they make it possible to model any mathematical function of SR '"in SR", if the number of hidden units is sufficient.
- a neural network consists of three layers: an input layer 10, a hidden layer 11 and an output layer 12.
- the input layer 11 corresponds to the explanatory variables, that is to say the variables of input (the aforementioned vector X), from which the prediction is made, and which will be described in detail later.
- the output layer 12 defines the predicted values (the above-mentioned vector Y).
- a first step 111 consists in calculating linear combinations of the explanatory variables so as to combine the information coming potentially from several variables.
- a second step 112 may consist in applying a non-linear transformation (for example a function of the "hyperbolic tangent" type) to each of the linear combinations in order to obtain the values of the hidden units or neurons that constitute the hidden layer. This nonlinear transformation defines the activation function of the neurons.
- the hidden units are recombined linearly, in step 113, to calculate the value predicted by the neural network.
- a prediction error is thus evaluated on examples from a validation set, which is distinct from the training set. This error defines the validation error. For example, it begins to decrease when increasing the number of hidden layers, reaches a minimum, and then increases when the number of hidden layers becomes too large. The minimum therefore defines an optimal number of hidden layers of the network;
- neural network There are different categories of neural network distinguished by their architecture (type of interconnection between neurons, choice of activation functions, or other) and the learning mode used.
- Neural networks are not used for prediction purposes only. They are also used for classification and / or clustering of data in a perspective of information reduction. Indeed, a network of neurons is able, in a set of data, to identify common characteristics between the elements of this set, to group them according to their resemblance. Each group thus formed is then associated with an element representative of the information contained in the group, called "representative". This representative can then be substituted for the entire group.
- the set of data can thus be described by means of a reduced number of elements, which constitutes a reduction of data.
- Kohonen cards or self-check cards organizers may be neural networks dedicated to this grouping task.
- the grossly estimated HRTFs can be determined from finite element modeling by considering for example simple geometrical shapes for the head, torso, neck, or other of an individual, as described in FIG. document FR-2 851 877, without repeating this description in detail here.
- the method that seemed the most immediate was a uniform selection in which a subset of roughly estimated HRTFs directions was chosen by trying to cover the entire 3D sphere as homogeneously and evenly as possible. This method was based on a regular sampling of the 3D sphere. However, it turned out that the HRTFs did not vary in a uniform way depending on the direction. From this point of view, a uniform selection of HRTFs was not really optimal.
- this grouping technique may consist of: in a first step, to identify the redundancies between the HRTFs of neighboring directions,
- each group is associated with an HRTF which is considered to be the representative of the group.
- This "representative" HRTF is one of the HRTFs of the cluster and is selected as the HRTF minimizing a distance criterion with all the other HRTFs in the group.
- the representative HRTF contains most of the HRTFs information of the group. In the end, all the representative HRTFs thus obtained constitute a compact description of the properties of the HRTFs for the entire 3D sphere.
- the clustering procedure also provides additional information as to the directions associated with the representative HRTFs, this information making it possible to define a selection of HRTFs intended to feed the input of the HRTFs calculation model. This selection is a priori non-uniform, but more efficient, and guarantees a better "representativeness" of the entire 3D sphere.
- the invention uses "artificial neural network” type statistical learning algorithms as a modeling tool for the corrective calculation of HRTFs (for example with a "Multi Layer Perception” neuron network or MLP).
- the input parameters of the neuron network are at least the azimuth angle ( ⁇ 1) and elevation angle ( ⁇ 1) specifying the direction of an HRTF to be calculated, and the HRTFs roughly estimated using the finite element model .
- the output parameters of the model are then the coefficients of the vector describing the HRTF for the direction ( ⁇ 1, ⁇ 1) and for the individual whose HRTFs were estimated by finite element modeling.
- the principle of calculating HRTFs by implementing an artificial neural network (for example of the MLP type) consists of:
- input layer 10 consisting of input parameters then including: o HRTFs roughly estimated and denoted HRTF 9 ( ⁇ , ⁇ j), with i lying between 1 and n, o the directions for which the HRTFs are to be calculated , preferably specified in the form of an elevation angle ( ⁇ j cal ) and an azimuth angle ( ⁇ j cal ), with j being between 1 and N, N possibly being different and in particular greater than n,
- the output layer 12 giving the corrected HRTFs of the individual in the directions ( ⁇ j cal , ⁇ j cal ) specified input
- One or more hidden layers 11 which seek, by adjusting the weight and activation functions of neurons, to better model the relationship between the input layer and the output layer.
- the test phase 23 To carry out these three phases, a database of HRTFs roughly estimated on one or more individuals is initially available. Thus, it will be understood that a prior step of collecting measurements of morphological parameters of several individuals and hence of their roughly estimated HRTFs in all directions of space is implemented. This is how we build the database 20.
- This database 20 is broken down into three distinct sets:
- VALID validation set
- TEST test set
- an input vector X (describing the direction of the HRTF to be calculated and the individual parameters such as the rough estimation of the HRTFs in all or some directions), and an output vector Y (corresponding to the HRTF that must be estimated at best the neural network).
- the validation phase 22 is conducted in conjunction with the learning phase 21. It consists in evaluating the prediction error of the neural network on a validation set (distinct from the training set ), which defines the validation error. During learning, the validation error begins to decrease and then starts to grow again when over-learning occurs. The minimum of the validation error therefore determines the end of the learning.
- test error finally describes the final performance of the neural network.
- the method illustrated by way of example therefore comprises a step a) during which a database 20 is constituted by measuring a plurality of HRTFs in a multiplicity of directions of the space and for a plurality of individuals.
- This measurement step referenced 40 in FIG. 4a consists in collecting the HRTFs measurements in N spatial directions, for M individuals, preferably of different morphology (or "morphotype"), in order to obtain an exhaustive database according to the specificities individuals. More generally, the number of individuals taken into account when learning is high and the better the performance of the neural network, especially in terms of "universality".
- step b) consists of learning the model using this database 20 and another database 41 comprising grossly estimated HRTFs from finite element modeling 49 (or "BEM") applied to morphological parameters 48 specific to the same individuals.
- BEM finite element modeling 49
- arbitrary directions i representative of HRTFs in a restricted number n are arbitrarily selected.
- This step 41 will be described in detail below, with reference to FIG. 4c.
- the three learning phases 21, validation 22 and test 23 are then conducted to build the model in step 44. It will be noted that it is possible to adjust the number of roughly estimated HRTFs to avoid the phenomenon of over-learning described above.
- the database 20 must be constituted under the most conventional and standard conditions to offer, at the output of the model, quality HRTFs that can be applied to rendering devices by providing satisfactory listening comfort. .
- a second type of measurement 48 is carried out on the same individuals on which the measurements constituting the measured HRTFs database have been carried out, and consisting in recording the morphological parameters of these M individuals (dimensions head, torso, neck, position and shape of ears, etc.).
- finite element modeling 49 is applied to obtain HRTFs estimated in at least a portion of the directions of space.
- a step 50 it is specified, at the input of the model, in which directions (0 j cal , ⁇ j cal ) the HRTFs will have to be calculated. Preferably, this will of course be the largest possible number of 3D space directions.
- a version of the model 44b, in the learning state calculates the HRTFs corrected in these directions (0j cal , ⁇ j cal ) from the roughly estimated HRTFs, in a following step 46b.
- the model compares these HRTFs computed and corrected with the HRTFs of the database 20 in the same directions (0j cal , ⁇ j cal ). If the deviation is judged to be too large (arrow N), the learning model 44b is perfected until this difference is reduced to an acceptable error (arrow O): the model then becomes definitive (end step 44).
- the individual IND is placed in a cabin CAB. It has its bust preferentially with respect to a top mark REP1 and a front mark REP2 provided in the cab CAB.
- This embodiment makes it possible to maintain the individual IND by being positioned correctly with respect to two means of shooting Si and S 2 according to two distinct angles O1 and ⁇ i and, consequently, to obtain a 3D topography of its bust, with in particular the dimensions of the individual's head, torso, neck, etc.
- the cabin comprises an ETA measurement standard which will serve as a scale for measuring these dimensions.
- means of shooting Si and S 2 incorporate, in their field, ETA yardstick with the bust of the individual IND.
- the images can be analyzed by shape recognition means to measure the morphological parameters of the individual.
- image signals are collected by an interface 51 of a CPU UC, which converts them into digital data. These data are then processed to determine the morphological parameters 48 and hence the coarse HRTFs by applying the BEM model (step 49). Finally, these grossly estimated HRTFs are processed by model 44 based on artificial neural network.
- the model 44 may be stored as a computer program product in a memory of the CPU.
- the HRTFs calculated for all the directions of the space that the model gives can then be stored in memory 52 or recorded on a removable medium (on diskette or engraved on CD-ROM) or communicated via a network such as the Internet or equivalent .
- the protocol for measuring the morphological parameters on the one hand, and the HRTFs measured in the base 20, on the other hand on the other hand, should preferably be defined in advance and be followed in substantially the same way, for all individuals.
- the network of neurons thus obtained is capable of calculating the HRTFs of any individual, in any direction, provided that measures of its morphological parameters are available.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0510995 | 2005-10-27 | ||
PCT/FR2006/002345 WO2007048900A1 (en) | 2005-10-27 | 2006-10-18 | Hrtfs individualisation by a finite element modelling coupled with a revise model |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1946612A1 true EP1946612A1 (en) | 2008-07-23 |
EP1946612B1 EP1946612B1 (en) | 2012-11-14 |
Family
ID=36658888
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP06820237A Active EP1946612B1 (en) | 2005-10-27 | 2006-10-18 | Hrtfs individualisation by a finite element modelling coupled with a corrective model |
Country Status (3)
Country | Link |
---|---|
US (1) | US20080306720A1 (en) |
EP (1) | EP1946612B1 (en) |
WO (1) | WO2007048900A1 (en) |
Families Citing this family (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2880755A1 (en) * | 2005-01-10 | 2006-07-14 | France Telecom | METHOD AND DEVICE FOR INDIVIDUALIZING HRTFS BY MODELING |
US9215544B2 (en) * | 2006-03-09 | 2015-12-15 | Orange | Optimization of binaural sound spatialization based on multichannel encoding |
FR2958825B1 (en) * | 2010-04-12 | 2016-04-01 | Arkamys | METHOD OF SELECTING PERFECTLY OPTIMUM HRTF FILTERS IN A DATABASE FROM MORPHOLOGICAL PARAMETERS |
WO2012028906A1 (en) * | 2010-09-03 | 2012-03-08 | Sony Ericsson Mobile Communications Ab | Determining individualized head-related transfer functions |
AU2012394979B2 (en) | 2012-11-22 | 2016-07-14 | Razer (Asia-Pacific) Pte. Ltd. | Method for outputting a modified audio signal and graphical user interfaces produced by an application program |
FR3000637A1 (en) * | 2012-12-28 | 2014-07-04 | Digital Media Solutions | DEVICE AND METHOD FOR SPACE INTERPOLATION OF SOUNDS |
CN108806704B (en) | 2013-04-19 | 2023-06-06 | 韩国电子通信研究院 | Multi-channel audio signal processing device and method |
CN104982042B (en) | 2013-04-19 | 2018-06-08 | 韩国电子通信研究院 | Multi channel audio signal processing unit and method |
US9883312B2 (en) | 2013-05-29 | 2018-01-30 | Qualcomm Incorporated | Transformed higher order ambisonics audio data |
US9466305B2 (en) | 2013-05-29 | 2016-10-11 | Qualcomm Incorporated | Performing positional analysis to code spherical harmonic coefficients |
US20140376754A1 (en) * | 2013-06-20 | 2014-12-25 | Csr Technology Inc. | Method, apparatus, and manufacture for wireless immersive audio transmission |
US9426589B2 (en) * | 2013-07-04 | 2016-08-23 | Gn Resound A/S | Determination of individual HRTFs |
US9319819B2 (en) * | 2013-07-25 | 2016-04-19 | Etri | Binaural rendering method and apparatus for decoding multi channel audio |
CN103731796B (en) * | 2013-10-10 | 2015-09-16 | 华南理工大学 | For many sound sources automatic measuring system of far field and Head-Related Transfer Function for Nearby Sources |
US9922656B2 (en) | 2014-01-30 | 2018-03-20 | Qualcomm Incorporated | Transitioning of ambient higher-order ambisonic coefficients |
US9489955B2 (en) | 2014-01-30 | 2016-11-08 | Qualcomm Incorporated | Indicating frame parameter reusability for coding vectors |
US10770087B2 (en) | 2014-05-16 | 2020-09-08 | Qualcomm Incorporated | Selecting codebooks for coding vectors decomposed from higher-order ambisonic audio signals |
US9620137B2 (en) | 2014-05-16 | 2017-04-11 | Qualcomm Incorporated | Determining between scalar and vector quantization in higher order ambisonic coefficients |
US9852737B2 (en) | 2014-05-16 | 2017-12-26 | Qualcomm Incorporated | Coding vectors decomposed from higher-order ambisonics audio signals |
US9747910B2 (en) | 2014-09-26 | 2017-08-29 | Qualcomm Incorporated | Switching between predictive and non-predictive quantization techniques in a higher order ambisonics (HOA) framework |
CN108028998B (en) * | 2015-09-14 | 2020-11-03 | 雅马哈株式会社 | Ear shape analysis device and ear shape analysis method |
EP3416105A4 (en) * | 2016-02-12 | 2019-02-20 | Sony Corporation | Information processing method and information processing device |
US10154365B2 (en) * | 2016-09-27 | 2018-12-11 | Intel Corporation | Head-related transfer function measurement and application |
US10306396B2 (en) | 2017-04-19 | 2019-05-28 | United States Of America As Represented By The Secretary Of The Air Force | Collaborative personalization of head-related transfer function |
US11363402B2 (en) | 2019-12-30 | 2022-06-14 | Comhear Inc. | Method for providing a spatialized soundfield |
KR102289707B1 (en) * | 2020-08-26 | 2021-08-13 | 현대제철 주식회사 | Method and system for temperature distribution prediction of winded coil |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AUPQ514000A0 (en) * | 2000-01-17 | 2000-02-10 | University Of Sydney, The | The generation of customised three dimensional sound effects for individuals |
JP3521900B2 (en) * | 2002-02-04 | 2004-04-26 | ヤマハ株式会社 | Virtual speaker amplifier |
AU2003260875A1 (en) * | 2002-09-23 | 2004-04-08 | Koninklijke Philips Electronics N.V. | Sound reproduction system, program and data carrier |
US7430300B2 (en) * | 2002-11-18 | 2008-09-30 | Digisenz Llc | Sound production systems and methods for providing sound inside a headgear unit |
US20090030552A1 (en) * | 2002-12-17 | 2009-01-29 | Japan Science And Technology Agency | Robotics visual and auditory system |
FR2851877B1 (en) * | 2003-02-28 | 2005-05-13 | METHOD OF MEASURING ACOUSTIC TRANSFER FUNCTIONS ASSOCIATED WITH THE MORPHOLOGY OF AN INDIVIDUAL | |
US7664272B2 (en) * | 2003-09-08 | 2010-02-16 | Panasonic Corporation | Sound image control device and design tool therefor |
FR2880755A1 (en) * | 2005-01-10 | 2006-07-14 | France Telecom | METHOD AND DEVICE FOR INDIVIDUALIZING HRTFS BY MODELING |
-
2006
- 2006-10-18 WO PCT/FR2006/002345 patent/WO2007048900A1/en active Application Filing
- 2006-10-18 EP EP06820237A patent/EP1946612B1/en active Active
- 2006-10-18 US US12/084,249 patent/US20080306720A1/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
See references of WO2007048900A1 * |
Also Published As
Publication number | Publication date |
---|---|
WO2007048900A1 (en) | 2007-05-03 |
US20080306720A1 (en) | 2008-12-11 |
EP1946612B1 (en) | 2012-11-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1946612B1 (en) | Hrtfs individualisation by a finite element modelling coupled with a corrective model | |
EP1836876B1 (en) | Method and device for individualizing hrtfs by modeling | |
EP2258119B1 (en) | Method and device for determining transfer functions of the hrtf type | |
EP2898707B1 (en) | Optimized calibration of a multi-loudspeaker sound restitution system | |
EP3348079B1 (en) | Method and system for developing a head-related transfer function adapted to an individual | |
EP1992198B1 (en) | Optimization of binaural sound spatialization based on multichannel encoding | |
EP1563485B1 (en) | Method for processing audio data and sound acquisition device therefor | |
EP2374124B1 (en) | Advanced encoding of multi-channel digital audio signals | |
EP1479266B1 (en) | Method and device for control of a unit for reproduction of an acoustic field | |
Yamamoto et al. | Fully perceptual-based 3D spatial sound individualization with an adaptive variational autoencoder | |
WO2007137232A2 (en) | Method of modifying audio content | |
EP1586220B1 (en) | Method and device for controlling a reproduction unit using a multi-channel signal | |
FR2839565A1 (en) | METHOD AND SYSTEM FOR REPRESENTING AN ACOUSTIC FIELD | |
EP3384688B1 (en) | Successive decompositions of audio filters | |
EP3449643B1 (en) | Method and system of broadcasting a 360° audio signal | |
Javeri et al. | A Machine Learning Approach to Predicting Personalized Head Related Transfer Functions and Headphone Equalization from Video Capture Data | |
EP3484185A1 (en) | Modelling of a set of acoustic transfer functions suitable for an individual, three-dimensional sound card and system for three-dimensional sound reproduction | |
EP3934282A1 (en) | Method for converting a first set of signals representing a sound field into a second set of signals and associated electronic device | |
COMB | 12/HRTF (q) cal cal j" 91'l | |
JP2023122018A (en) | Signal processor, signal processing program and signal processing method | |
EP1869949A1 (en) | Method and system for spatializing an audio signal based on its intrinsic qualities |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20080425 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
17Q | First examination report despatched |
Effective date: 20090122 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602006033117 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: H04S0001000000 Ipc: H04S0007000000 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04S 7/00 20060101AFI20120112BHEP |
|
RTI1 | Title (correction) |
Free format text: HRTFS INDIVIDUALISATION BY A FINITE ELEMENT MODELLING COUPLED WITH A CORRECTIVE MODEL |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: FRANCE TELECOM |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 584505 Country of ref document: AT Kind code of ref document: T Effective date: 20121115 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602006033117 Country of ref document: DE Effective date: 20130110 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20121114 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 584505 Country of ref document: AT Kind code of ref document: T Effective date: 20121114 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130225 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130215 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130314 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130214 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PUE Owner name: ORANGE, FR Free format text: FORMER OWNER: FRANCE TELECOM, FR |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
RAP2 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: ORANGE |
|
26N | No opposition filed |
Effective date: 20130815 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602006033117 Country of ref document: DE Effective date: 20130815 |
|
BERE | Be: lapsed |
Owner name: FRANCE TELECOM Effective date: 20131031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131031 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131018 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20061018 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131018 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20121114 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 12 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 18 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230920 Year of fee payment: 18 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20230920 Year of fee payment: 18 |