EP0790753A1 - System for sound spatial effect and method therefor - Google Patents

Abstract

The method involves using a binaural processor (*) with two channels operating for each monophonic sound channel which is to be located. The two channels include filters which are linked to combine signals linearly within the channels. The processor is linked to an orientating device (13) which determines the spatial location of a sound source and is itself connection to a position locating device (3,4,12). Convolution of the signals is performed between the monophonic signal and the user left ear transfer function and the user right ear transfer function. These functions are specific to the particular user. The transfer functions may also include use of a phase interpolation device to determine the appropriate phase for each signal within the two channels.

Description

The present invention relates to a sound spatialization system, as well as to a personalization method making it possible to implement sound spatialization.

An airplane pilot, in particular a fighter airplane, wears a stereophonic helmet which restores not only radio communications, but also various alarms and on-board communications. Radiocommunications can be satisfied with a stereophonic, or even monophonic reproduction, while the alarms and on-board communications cannot be located in relation to the pilot (or the co-pilot ...).

The subject of the present invention is an audiophonic communication system, which makes it possible to easily discriminate the location of a determined sound source, in particular in the case of the existence of several sound sources close to the user.

The sound spatialization system according to the invention comprises, for each monophonic channel to be spatialized, a binaural processor with two channels of convolution filters combined linearly in each channel, this processor (s) being connected to one orienting device for calculating the spatial location of sound sources, itself connected to localization devices, characterized in that it includes, for at least part of the channels, a complementary sound illustration device connected to the corresponding binaural processor, this device additional sound illustration comprising at least one of the following circuits: bandwidth widening circuit, background noise production circuit, circuit for simulating the acoustic behavior of a room, Doppler effect simulation circuit, circuit producing different sound symbols each corresponding to a specific source or alarm.

The personalization method according to the invention consists in estimating the head transfer functions of the user by measuring these functions at a finite number of points in the surrounding space, then by interpolating the values thus measured to be calculated the head transfer functions, for each of the user's two ears, at the point in the space where the sound source is located, and to create the "spatialized" signal from the monophonic signal to be processed by convolving it with each of the two transfer functions thus estimated. It is thus possible to "personalize" the convolution filters for each user of the system implementing this method. Each user will then be able to locate in the best possible way the virtual sound source reproduced by his audio equipment.

• la figure 1 est un bloc-diagramme d'un système de spatialisation sonore conforme à l'invention,
• la figure 2 est un schéma explicatif de l'interpolation spatiale réalisée suivant le procédé de l'invention
• la figure 3 est un bloc-diagramme fonctionnel des principaux circuits de spatialisation de l'invention, et
• la figure 4 est une vue simplifiée de l'appareillage de recueil des fonctions de transfert de tête conformément au procédé de l'invention.

The present invention will be better understood on reading the detailed description of an embodiment, taken by way of nonlimiting example and illustrated by the appended drawing in which:

• FIG. 1 is a block diagram of a sound spatialization system according to the invention,
• FIG. 2 is an explanatory diagram of the spatial interpolation carried out according to the method of the invention
• FIG. 3 is a functional block diagram of the main spatialization circuits of the invention, and
• Figure 4 is a simplified view of the apparatus for collecting head transfer functions according to the method of the invention.

The invention is described below with reference to an aircraft audio system, in particular a combat aircraft, but it is understood that it is not limited to such an application, and that it can be implementation as well in other types of vehicles (land or sea) as in fixed installations. The user of this system is, in this case, the pilot of a combat aircraft, but it is understood that there can be several users simultaneously, in particular if it is a combat aircraft. civil transport, specific devices for each user being provided in corresponding number.

The role of the spatialization module 1 shown in FIG. 1 is to make sound signals (tones, speech, alarms, etc.) heard using a stereo headset so that they are perceived by the listener as if they came from a particular point in space, this point can be the actual position of the sound source or an arbitrary position. Thus, for example, the pilot of a combat aircraft hears the voice of his co-pilot as if it actually came from behind him, or else an audible missile attack alert is positioned spatially at the point of arrival of the threat. In addition, the position of the sound source changes in function of the pilot's head movements and of the airplane's movements: for example, an alarm generated at the azimuth "3 hours" must be found at "noon" if the pilot turns his head 90 degrees to the right.

The module 1 is for example connected to a digital bus 2 from which it receives information supplied by: a head position detector 3, an inertial unit 4 and / or a location device such as a goniometer, a radar, etc. ., countermeasures devices 5 (detection of external threats such as missiles) and an alarm management device 6 (signaling in particular breakdowns of aircraft instruments or equipment).

The module 1 includes an interpolator 7, the input of which is connected to the bus 2 to which various sound sources are connected (microphones, alarms, etc.). In general, these sources are sampled at relatively low frequencies (6, 12 or 24 kHz for example). The interpolator 7 makes it possible to raise these frequencies to a multiple common, for example 48 kHz in the present case, a frequency necessary for the processors located downstream. This interpolator 7 is connected to n binaural processors, referenced 8 as a whole, n being the maximum number of channels to be spatialized simultaneously. The outputs of the processors 8 are connected to an adder 9 whose output constitutes the output of the module 1. The module 1 also comprises in the connection between at least one output of the interpolator 7 and the input of the corresponding processor of the assembly 8 an adder 10, the other input of which is connected to the output of a device 11 for additional sound illustration.

This device 11 produces a sound signal covering in particular the high frequencies (for example from 5 to 16 kHz) of the audio spectrum. It thus completes the useful bandwidth of the transmission channel to which its output signal is added. This transmission channel can advantageously be a radio channel, but it is understood that any other channel can be completed in this way, and that several channels can be completed in the same system, by providing a corresponding number of adders such as 10. In radio communications use reduced bandwidths (3 to 4 kHz in general). Such a bandwidth is insufficient for correct spatialization of the sound signal. Tests have shown that high frequencies (above about 14 kHz), located beyond the limit of the vocal spectrum, allow a better localization of the source of the sound. The device 11 is then a bandwidth widening device. The additional sound signal can for example be background noise characteristic of a radio link. The device 11 can also be, for example, a device simulating the acoustic behavior of a room, a building, or a device simulating a Doppler effect, or even a device producing different sound symbols each corresponding to a source or a specific alarm.

The processors 8 each generate a signal of stereophonic type from the monophonic signal coming from the interpolator 7 to which is added if necessary, the signal from the device 11, taking into account the data supplied by the head position detector 3 of the pilot.

The module 1 also includes a device 12 for managing the sources to be spatialized followed by an orienter 13 with n inputs (n being defined above) controlling the n different processors of the set 8. The device 13 is a calculator calculating, from the data provided by the pilot's head position detector, the orientation of the aircraft relative to the terrestrial reference (provided by the aircraft's inertial unit) and the location of the source, the coordinates in the space of the point where the sounds emitted by this source must appear to come from.

If we want to spatialize simultaneously, at n2 distinct points in space (with n2≤n) n2 distinct sources, we advantageously use, as device 13, an orienter with n2 inputs sequentially calculating the coordinates of each source to be spatialized . Because the number of sound sources that an average observer can distinguish is generally four, n2 is advantageously equal to four at most.

At the output of the adder 9, a single two-channel channel (left and right) is obtained which is transmitted via bus 2 to audio listening circuits 14.

The device 12 for managing the n sources to be spatialized is a computer which receives, via bus 2, information concerning the characteristics of the sources to be spatialized (site, deposit and distance by report to the pilot), personalization criteria at the user's choice and priority information (threats, alarms, important radio communications, etc.). The device 12 receives from the device 4 information concerning the evolution of the location of certain sources (or of all the sources, if applicable). From this information, the device 12 selects the source (or at most the n2 sources) to be spatialized.

Advantageously, a memory card reader 15 is used 16 for the device in order to personalize the management of sound sources by the device 12. The reader 15 is connected to the bus 2. The card 16 then contains the characteristics of the filtering carried out by the flags of each user's ears. In the preferred embodiment, it is a set of pairs of digital filters (that is to say coefficients representing their impulse responses) corresponding to the acoustic filtering "left ear" and "right ear" produced for various points in the space surrounding the user. The database thus formed is loaded, via bus 2, into the memory associated with the various processors 8.

The processors 8 each essentially comprise two channels (called "left ear" and "right ear") for convolution filtering. More precisely, the role of each of the processors 8 is on the one hand to calculate by interpolation the head transfer functions (right and left) at the point at which the source will be placed, on the other hand to create the spatial signal on two channels from the original monophonic signal.

The collection of head transfer functions requires spatial sampling: these transfer functions are only measured in a finite number of points (of the order of 100). However to correctly "spatialize" a sound, it would be necessary to know the transfer functions at the source point of the source, determined by the orienter 13. It is therefore necessary to be satisfied with an estimation of these functions: this operation is carried out by an interpolation "barycentric" of the four pairs of functions associated with the four measurement points closest to the point of the calculated space.

Thus, as shown diagrammatically in FIG. 2, measurements are made at different points of space regularly spaced in elevation and in bearing and located on the same sphere. FIG. 2 shows a part of the "grid" G thus obtained for the points Pm, Pm + 1, Pm + 2, ... Pp, Pp + 1 .... Or a point P of said sphere, determined by the orienter 13 as being located in the direction of the sound source to be "spatialized". This point P is inside the curvilinear quadrilateral delimited by the points Pm + 1, Pm + 2, Pn + 1, Pn + 2. The barycentric interpolation is therefore carried out for the position of P with respect to these four points. The different equipment determining the orientation of the sound source and the orientation and location of the user's head provide their respective data every 20 or 40 ms (ΔT), that is to say that every Δ T a couple of transfer functions are available. In order to avoid audible "jumps" during restitution (when the operator changes the orientation of his head, he must perceive an uninterrupted sound), the signal to be spatialized is in fact convoluted by a pair of filters obtained by "temporal" interpolation performed between the convolution filters spatially interpolated at times T and T + ΔT. It then remains only to convert the digital signals thus obtained into analog before their restitution in the user's earphones.

In the diagram of FIG. 3, which relates to a channel to be spatialized, the various attitude (position) sensors used have been shown. These are: a head attitude sensor 17, a sound source attitude sensor 18, and an attitude sensor 19 of the carrier mobile (airplane for example). The information from these sensors is supplied to the orienter 13, which determines from this information the spatial position of the source relative to the head of the user (in line of sight and in distance). The orienter 13 is connected to a database 20 (included in the card 16) of which it controls the loading to the processors 8 of the "left" and "right" transfer functions of the four points closest to the position of the source (see Figure 2), or possibly the measurement point (if the position of the source coincides with that of one of the measurement points in grid G). These transfer functions are subjected to a spatial interpolation at 21, then to a temporal interpolation at 22, and the resulting values are convolved at 23 with the signal 24 to be spatialized. Of course, functions 21 and 23 are performed by the same interpolator (interpolator 7 in FIG. 1), and the convolutions are performed by the binaural processor 8 corresponding to the spatialized channel. After convolution, a digital-analog conversion is carried out, in 25, and the sound reproduction (amplification and sending to a stereo headset) at 26. Of course, operations 20 to 23 and 25, 26 are done separately for the left channel and for the right channel.

The “personalized” convolution filters constituting the previously mentioned database are established from measurements using a method described below with reference to FIG. 4.

In an anechoic chamber, an automated mechanical tool 27 is installed, consisting of a semi-circular rail 28 mounted on a motorized pivot 29 fixed to the floor of this chamber. The rail 28 is arranged vertically, so that its ends are on the same perpendicular. On this rail 28, a support 30 moves on which a broadband speaker 31 is mounted. This device makes it possible to place the speaker at any point on the sphere defined by the rail when the latter performs a rotation of 360 degrees around a vertical axis passing through pivot 29. The precision of the positioning of the loudspeaker is one degree in elevation and in bearing, for example.

A first series of readings is taken: the loudspeaker 31 is placed successively at X points of the sphere, that is to say that the space is "discretized": it is a spatial sampling. At each measurement point, a pseudo-random code is generated and reproduced by the loudspeaker 31. The sound signal emitted is picked up by a pair of reference microphones placed at the center 32 of the sphere (the distance separating the microphones is the order of the width of the head of the subject whose transfer functions are to be collected), in order to measure the resulting sound pressure as a function of frequency.

We then carry out a second series of surveys: the method is the same but this time, the subject is placed so that his ears are located at the location of the microphones (the subject checks the position of his head by video feedback). The subject is provided with individualized shutter earplugs in which miniature microphones are placed. Complete obturation of the duct has the following advantages: the ear is acoustically protected, and the stapedial reflex (nonexistent in this case) does not modify the acoustic impedance of the assembly.

For each position of the loudspeaker, for each ear, after compensation for the responses of the miniature microphones and the loudspeaker, the acoustic pressures as a function of the frequency, measured in the two previous experiments, are carried out. X couples (left ear, right ear) of transfer functions are thus obtained.

According to the convolution technique used, the database of transfer functions can be made up either of pairs of frequency responses (convolution by multiplication in the frequency domain) or of pairs of impulse responses (classical temporal convolution), inverse Fourier transforms of the previous ones.

The use of a signal obtained by generation of a pseudo-random binary code makes it possible to have an impulse response of great dynamics with an average sound level emitted (70 dBa for example).

The use of acoustic sources emitting pseudo-random binary signals tends to be generalized in the impulse response measurement technique, especially with regard to the characterization of an acoustic room by the correlation method.

In addition to their characteristics (autocorrelation function) and their particular properties which lend themselves to optimizations (Hadamard transform), these signals make the assumption of linearity of the acoustic collection system acceptable. They also make it possible to overcome variations in the acoustic impedance of the ossicular chain by stapedial reflex, by limiting the emission level (70 dBa). Preferably, pseudo-random binary signals of maximum length sequence are produced. The advantage of maximum length sequences lies in their spectral characteristics (white noise) and their generation mode which allows optimization of the processing processor.

• J.K.Holmes : "Coherent spread spectrum systems". Wiky Interscience
• J. Borish and J.B. Angell : "An efficient algorithm for measuring the impulse response using pseudorandom noise" J. Audio Eng. Soc., Vol. 31, n° 7, July/August 1983.
• L. Otshudi, J.P. Quilhot: "Considérations sur les propriétés énergétiques des signaux binaires pseudo-aléatoires et sur leur utilisation comme excitateurs acoustiques". Acustica Vol. 90, pp 76-81, 1990.

The principles of measurement using pseudo-random binary signals implemented by the present invention are for example described in the following works:

• JKHolmes: "Coherent spread spectrum systems". Wiky Interscience
• J. Borish and JB Angell: "An efficient algorithm for measuring the impulse response using pseudorandom noise" J. Audio Eng. Soc., Vol. 31, No. 7, July / August 1983.
• L. Otshudi, JP Quilhot: "Considerations on the energetic properties of pseudo-random binary signals and their use as acoustic exciters". Acustica Vol. 90, pp 76-81, 1990.

They are therefore only briefly recalled here.

• génération d'un signal de référence et enregistrement concomitant des deux voies microphoniques,
• calcul de la réponse impulsionnelle du trajet acoustique, (diffraction)
• calcul de certains critères (gain de chaque voie, ordre du moyennage, niveau numérique de sortie, indicateur de stockage, mesure du retard binaural des 2 voies par corrélation, décalage pour simuler les retards géométriques, ...)
• visualisation des résultats, échogrammes, courbe de décroissance, sortie sur imprimante.

From the generation of pseudo-random sequences, the following main functions are carried out:

• generation of a reference signal and concomitant recording of the two microphone channels,
• calculation of the impulse response of the acoustic path, (diffraction)
• calculation of certain criteria (gain of each channel, order of averaging, digital output level, storage indicator, measurement of binaural delay of the 2 channels by correlation, offset to simulate geometric delays, ...)
• visualization of results, echograms, decay curve, output to printer.

The impulse response is obtained over time (2n-1) / fe where N is the order of the sequence and where fe is the sampling frequency. It is up to the experimenter to choose a couple of values (sequence order, fe) sufficient to have all the useful decrease of the response.

The sound spatialization device described above makes it possible to increase the intelligibility of the sound sources which it processes, to decrease the reaction time of the operator with respect to alarm, alert or warning signals. 'other sound indicators, the sources of which seem to be located respectively at different points in space, therefore easier to discriminate between them and easier to classify in order of importance or urgency.

Claims (8)

Method for customizing a sound spatialization system, characterized in that it consists in estimating the head transfer functions of the user by measuring these functions in a finite number of points in the surrounding space, then by interpolating the values thus measured to calculate the head transfer functions, for each of the user's two ears, at the point in the space where the sound source is located, and to create a spatialized signal from the monophonic signal at deal by convolving it with each of the two transfer functions thus estimated. Method according to claim 1, characterized in that the interpolation comprises a phase of spatial interpolation and a phase of temporal interpolation. Method according to claim 1 or 2, characterized in that the sound source emits a pseudo-random binary signal. Method according to any one of Claims 1 to 3, characterized in that the head transfer functions are estimated at around 100 points. Spatialization system of sound sources producing each of the monophonic channels, comprising, for each monophonic channel to be spatialized, a binaural processor (8) with two channels of convolution filters combined linearly in each channel, this processor (s) being connected (s) to an orienting device (13) for calculating the spatial location of sound sources, itself connected to at least one location device (3, 4, 12), characterized in that it comprises for at least part channels a complementary sound illustration device (11) connected to the corresponding binaural processor, this complementary sound illustration device comprising at least one of the following circuits: bandwidth widening circuit, background noise production circuit , circuit for simulating the acoustic behavior of a room, circuit Doppler effect simulation, circuit producing different sound symbols each corresponding to a specific source or alarm. System according to claim 5, characterized in that the location device is at least one of the following devices: inertial unit (4), head position detector (3), radar, goniometer. System according to one of claims 5 or 6, characterized in that it is connected to a countermeasuring device (5). System according to one of claims 5 to 7, characterized in that it is connected to a device (15) reader for memory cards (16) in each of which are stored the data characteristic of the filtering carried out by the flags of the ears of the corresponding user.

Images