EP1658755A1

EP1658755A1 - Sound source spatialization system

Info

Publication number: EP1658755A1
Application number: EP03748189A
Authority: EP
Inventors: Gérard; c/o THALES Intellectual Property REYNAUD
Original assignee: Thales SA
Current assignee: Thales SA
Priority date: 2002-07-02
Filing date: 2003-06-27
Publication date: 2006-05-24
Anticipated expiration: 2023-06-27
Also published as: FR2842064B1; IL165911A0; CA2490501A1; ES2302936T3; US20050271212A1; ATE390029T1; DE60319886D1; AU2003267499B2; AU2003267499A1; IL165911A; WO2004006624A1; FR2842064A1; EP1658755B1; DE60319886T2; AU2003267499C1

Abstract

The present invention relates to an enhanced-performance sound source spatialization system used in particular to produce a spatialization system compatible with an integrated modular avionics type system. It comprises a filter database comprising a set of head-related transfer functions specific to the listener, a data presentation processor receiving information from each source and comprising in particular a module for computing the relative positions of the sources in relation to the listener and a module for selecting the head-related transfer functions with a variable resolution suited to the relative position of the source in relation to the listener, a unit for computing said monophonic channels by convoluting each sound source with head-related transfer functions of said database estimated at said source position.

Description

SOUND SOURCES SPATIALIZATION SYSTEM

The present invention relates to a spatialization system of sound sources with improved performance allowing in particular the production of a spatialization system compatible with modular avionic data processing equipment of IMA type (abbreviation of the English expression “Integrated Modular Avionics ”) also called EMTI (for Modular Information Processing Equipment). In the on-board aeronautical field, the majority of reflections concerning the cockpit of the future leads to the need for a helmet-mounted visual worn by the head, associated with a very large format display presented at low head. This set should make it possible to improve the perception of the global situation (“awarness situation”) while reducing the burden on the pilot thanks to a presentation of a real-time synthesis of information from multiple sources (sensors, database) .

The 3D sound is in the same approach as the helmet visual by allowing pilots to acquire information of spatial situation (position of team members, threats, ...) in its own reference point, by a different communication channel than visual following a natural modality. In general, 3D sound enriches the transmitted signal with situation information, static or dynamic, in space. Its use, in addition to locating teammates or threats, can cover other applications such as multi-speaker intelligibility. In French patent application FR 2 744 871, the applicant has described a sound source spatialization system producing, for each source, spatialized monophonic channels (left / right) intended to be received by a listener using stereo headphones, such so that the sources are perceived by the listener as if they came from a particular point in space, this point being either the actual position of the sound source or an arbitrary position. The principle of sound spatialization is based on the calculation of the convolution of the sound source to be spatialized (monophonic signal) with transfer functions called head transfer functions (or HRTF according to the English expression “Head Related Transfer Functions ”) Specific to the auditor and which were measured during a prior recording phase. So the system described in the above-mentioned request includes in particular for each source to be spatialized, a binaural processor with two convolutional channels whose role is on the one hand to calculate by interpolation the head transfer functions (left / right) at the point in which the sound source will be placed, on the other hand to create the spatialized signal on two channels from the original monophonic signal.

The aim of the present invention is to define a spatialization system with improved performance such that it is able to be integrated into an avionics modular information processing equipment (EMTI) which imposes constraints in particular on the number of processors and their type.

For this, the invention proposes a spatialization system in which it is no longer necessary to make an interpolation calculation of the head transfer functions. It is then possible to carry out the convolution operations in order to create the spatialized signals to have only one single computer instead of the n binaural processors necessary in the system according to the prior art for spatializing n sources.

More specifically, the invention relates to a spatialization system of at least one sound source creating for each source two spatialized monophonic channels intended to be received by a listener, comprising

- a filter database comprising a set of head transfer functions specific to the auditor,

a data presentation processor receiving the information from each source and comprising in particular a module for calculating the relative positions of the sources with respect to the listener,

a unit for calculating said monophonic channels by convolution of each sound source with head transfer functions from said database estimated at said position of the source, the system being characterized in that said data presentation processor comprises a module selection of head transfer functions with variable resolution adapted to the relative position of the source with the listener.

The use of pilot head transfer function bases adapted to the precision required for a given piece of information to be spatialized (threat, position of a drone, etc.), combined with optimal use of the spatial information contained in each of the positions of these bases makes it possible to considerably reduce the number of operations to be carried out for spatialization without degrading performance. Other advantages and characteristics will appear more clearly on reading the description which follows, illustrated by the appended figures which represent:

- Figure 1, a general diagram of a spatialization system according to the invention; - Figure 2, a block diagram of an exemplary embodiment of the system according to the invention;

- Figure 3, the diagram of a calculation unit of a spatialization system according to the example of Figure 2;

- Figure 4, a layout diagram of the system according to the invention in a modular avionics type equipment

IMA. The invention is described below with reference to an aircraft audio system, in particular a combat aircraft, but it is understood that it is not limited to such an application and that it can be implemented. used in other types of vehicles (land or sea) as well as in fixed installations. The user of this system is, in this case, the pilot of an airplane but there can be several users simultaneously, in particular if it is a civil transport airplane, devices specific to each user. then being provided in sufficient number.

FIG. 1 represents a general diagram of a spatialization system of sound sources according to the invention, the role of which is to make a listener hear sound signals (tones, words, alarms, etc.) using a stereophonic headphones so that they are perceived by the listener as if they came from a particular point in space, this point being able to be the effective position of the sound source or an arbitrary position. For example, the detection of a missile by a countermeasure device may generate a sound the origin of which seems to come from the origin of the attack, allowing the pilot to react more quickly. These sounds (monophonic sound signals) are for example recorded under digital form in a “sound” database. Furthermore, account is taken of the evolution of the position of the sound source as a function of the movements of the pilot's head and of the movements of the airplane. Thus, an alarm generated at the azimuth "3 hours" must be found at "noon" if the pilot turns his head 90 ° to the right.

The system according to the invention mainly comprises a processor CPU1 for presenting data and a calculation unit CPU2 generating the monophonic spatialized channels. The data presentation processor CPU1 notably comprises a module 101 for calculating the relative positions of the sources with respect to the listener, that is to say in the listener's head reference. These positions are for example calculated from information received by an attitude detector 11 from the listener's head and by a module 12 for determining the position of the source to be restored (this module can include an inertial unit, a tracking device such as a goniometer, a radar, etc.). The processor CPU1 is connected to a “filter” database 13 comprising a set of head transfer functions (HRTF) specific to the listener. The head transfer functions are for example acquired during an earlier learning phase. They are specific to the inter-aural delay of the listener (delay in the arrival of sound between the two ears), of the physiognomic characteristics of each listener. It is these transfer functions that give the listener the feeling of spatialization. The computing unit CPU2 generates the monophonic channels G and D spatialized by convolution of each monophonic sound signal characteristic of the source to be spatialized and contained in the “sounds” database 14 with head transfer functions from said database 13 estimated at the position of the source in the head reference.

In spatialization systems according to the prior art, the calculation unit comprises as many processors as there are sound sources to be spatialized. Indeed, it is necessary in these systems to carry out a spatial interpolation of the head transfer functions in order to know the transfer functions at the point at which the source will be placed. This architecture requires multiplying the number of processors in the computing unit, which is incompatible with a modular spatialization system for integration into modular avionics information processing equipment.

The spatialization system according to the invention has a specific algorithmic architecture which makes it possible in particular to reduce the number of processors of the computing unit. The applicant has shown that the computing unit CPU2 can then be produced by means of a programmable component of the EPLD type (abbreviation of “Programmable electronics by logic gates”). To do this, the data presentation processor of the system according to the invention comprises a module 102 for selecting head transfer functions with a variable resolution adapted to the relative position of the source with the listener (or position of the source in the head reference). Thanks to this selection module, it is no longer necessary to carry out interpolation calculations to estimate the transfer functions at the location where the sound source must be located. It is therefore possible to considerably simplify the architecture of the calculation unit, an exemplary embodiment of which will be described later. Furthermore, the selection module operating a selection of the resolution of the transfer functions as a function of the relative position of the sound source with respect to the listener, it is possible to work with a database 13 of the head transfer functions comprising a large number of functions distributed regularly throughout the space, knowing that only a part of these will be selected to perform the convolution calculations. Thus, the applicant worked with a database in which the transfer functions are collected with a step of 7 ° in azimuth, from 0 to 360 °, and with a step of 10 ° in elevation, from -70 ° to + 90 °.

Furthermore, the applicant has shown that, thanks to the resolution selection module 102 of the system according to the invention, it is possible to limit the number of coefficients of each head transfer function used to 40 (compared to 128 or 256 in most systems of the prior art) without degrading the sound spatialization results, which further reduces the computing power necessary for the spatialization function.

The applicant has thus demonstrated that the use of bases for transfer functions of the pilot's head adapted to the precision required for given information to be spatialized, combined with optimal use of the spatial information contained in each of the positions of these bases. makes it possible to considerably reduce the number of operations to be carried out for spatialization without degrading performance.

The calculation unit CPU2 can thus be reduced to a component of the EPLD type for example, even when several sources have to be spatialized, which makes it possible to dispense with the protocols of dialogue between the different binaural processors necessary for processing the spatialization of several sound sources in prior art systems.

This optimization of the computing power in the system according to the invention also makes it possible to introduce other functions which will be described later.

FIG. 2 represents a functional diagram of an exemplary embodiment of the system according to the invention.

The spatialization system comprises a data presentation processor CPU1 receiving the information from each source and a calculation unit CPU2 of the spatialized right and left monophonic channels. The processor CPU1 notably comprises the module 101 for calculating the relative position of a sound source in the listener's head reference frame, this module receiving in real time information on the head attitude (listener position) and on the position of the source to be restored, as described above. According to the invention, the module 102 for selecting in resolution the HRTF transfer functions contained in the database 13 makes it possible to select, for each source to be spatialized, as a function of the relative position of the source, the transfer functions which will be used for the generation of spatialized sounds. In the example of FIG. 2, a sound selection module 103 connected to the sound database 14 makes it possible to select the monophonic signal from the database which will be sent to the computing unit CPU2 to be convoluted to the functions adapted left and right head transfer. Advantageously, the sound selection module 103 operates a hierarchy between the sound sources to be spatialized. Depending on system events and the choice of platform management logic, a choice of concomitant sounds to be spatialized will be made. All of the information used to define this spatial presentation priority logic travels on the EMTI broadband bus. The sound selection module 103 is for example connected to a configuration and parameterization 104 in which personalization criteria specific to the auditor are recorded.

The data concerning the choice of the HRTF transfer functions as well as the sounds to be spatialized are sent to the computing unit CPU2 by means of a communication link 15. They are temporarily stored in a filtering memory and digital sounds 201. The part of the memory containing the digital sounds called "earcons" (name assigned to the sounds used as alarms or alerts and having a strong significant value) is for example loaded at initialization. It contains the samples of audio signals previously digitized in the sound database 14. At the request of the host CPU1, the spatialization of one or more of these signals will be activated or suspended. As long as the activation persists, the signal concerned is read in a loop. The convolution calculations are performed by a computer 202, for example a component of the EPLD type which generates the spatialized sounds as has been described previously.

In the example of FIG. 2, a processor interface 203 constitutes a memory used for the filtering operations. It is composed of buffer registers for sounds, HRTF filters, as well as coefficients used for other functions such as soft switching and the simulation of atmospheric absorption which will be described later.

With the spatialization system according to the invention, two types of sounds can be spatialized: the earcons (or audible alarms) or sounds coming from radios directly (UHF / VHF) called “live sounds” in FIG. 2.

FIG. 3 represents the diagram of a calculation unit of a spatialization system according to the example of FIG. 2.

Advantageously, the spatialization system according to the invention comprises an audio input / output conditioning module 16 which recovers the monophonic left and right spatialized channels at the output in order to format them before sending them to the listener. Optionally, if “live” communications must be spatialized, these communications are shaped by the conditioning module with a view to their spatialization by the computer 202 of the computing unit. By default, a sound coming from a so-called live source will always have priority over the sounds to be spatialized.

We find the processor interface 203 which constitutes a short term memory for all the parameters used. The computer 202 constitutes the heart of the calculation unit. In the example in FIG. 3, it includes a module 204 for activating and selecting sources, performing the mixing function between the live inputs and the earcons type sounds.

Thanks to the system according to the invention, the computer 202 can perform the calculation functions for the n sources to be spatialized. In the example in Figure 3, four sound sources can be spatialized.

It includes a double spatialization module 205, which receives the adapted transfer functions and performs the convolution with the monophonic signal to be spatialized. This convolution is carried out in time space by using the shifting capacities of the FIR filters (finite impulse response filters) associated with the inter-aural delays.

Advantageously, it includes a soft switching module 206, connected to a calculation parametering register 207 optimizing the choice of transition parameters as a function of the speed of movement of the source and of the listener's head. The soft switching module allows a transition, without audible switching noise, when switching from one pair of filters to the next. This function is performed by double linear weighting ramp. It involves a double convolution: each sample of each output channel results from the weighted sum of two samples, each being obtained by convolution of the input signal with a spatialization filter, element of the HRTF base. At a given instant, there are therefore in input memory two pairs of spatialization filters per channel to be processed.

Advantageously, it includes a module 208 for simulating atmospheric absorption. This function is for example performed by a linear filtering with 30 coefficients and a gain, realized on each channel (left, right) of each channel, after spatialization processing. This function allows the auditor to perceive the depth effect necessary for his operational decision. Finally, dynamic weighting modules 209 and summation 210 are provided for performing the weighted sum of the channels of each channel to provide a single stereophonic signal compatible with the output dynamics. The only constraint associated with this stereophonic reproduction is linked to the bandwidth necessary for sound spatialization (typically 20 kHz).

FIG. 4 diagrams the hardware architecture of an avionics modular equipment 40 for processing information of the EMTI type. It includes a broadband bus 41 to which all the equipment functions are connected, including in particular the sound spatialization system according to the invention 42 as described above, the other man machine interface functions 43 such as by example voice control, head-up symbology management, helmet display, etc., and a system management card 44 which functions as an interface with the other equipment of the aircraft. The sound spatialization system 42 according to the invention is connected to the high speed bus via the data presentation processor CPUL II also comprises the calculation unit CPU2, as described above and formed for example of a component EPLD, compatible with the technical requirements of EMTI (number and type of operations, memory space, coding of audio samples, digital bit rate).

Claims

1- Spatialization system (42) of at least one sound source creating for each source two spatialized monophonic channels (L, R) intended to be received by a listener, comprising

- a filter database (13) comprising a set of head transfer functions (HRTF) specific to the auditor,

- a data presentation processor (CPU1) receiving the information from each source and comprising in particular a calculation module (101) of the relative positions of the sources with respect to the listener,

a calculation unit (CPU2) of said monophonic channels by convolution of each sound source with head transfer functions from said database estimated at said position of the source, the system being characterized in that said data presentation processor comprises a module (102) for selecting head transfer functions with a variable resolution adapted to the relative position of the source with the listener.

2- Spatialization system according to claim 1, characterized in that the transfer functions (HRTF) included in the database (13) are collected with a step of 7 ° in azimuth, from 0 to 360 °, and with a step of 10 ° in elevation, from -70 ° to + 90 °.

3- Spatialization system according to one of claims 1 or 2, characterized in that the number of coefficients of each head transfer function is approximately 40. 4- Spatialization system according to one of the preceding claims, characterized in that it comprises a sound database (14) containing in digital form a monophonic sound signal characteristic of each source to be spatialized, this sound signal being intended to be convoluted with the head transfer functions selected. 5- sound spatialization system according to claim 4, characterized in that the data presentation processor (CPU1) comprises a module (103) for selecting sounds connected to the sound database (14) and operating a hierarchy between the concomitant sound sources to be spatialized. 6- sound spatialization system according to claim 5, characterized in that the data presentation processor (CPU1) comprises a module (104) for configuration and parameterization to which the sound selection module (103) is connected and in which personalization criteria specific to the auditor are recorded.

7- Spatialization system according to one of the preceding claims, characterized in that it comprises a module (16) for audio input / output conditioning which recovers the monophonic channels (G, D) spatialized as output in order to shape them before send them to the listener.

8- Spatialization system according to claim 7, characterized in that “live” communications having to be spatialized, these communications are shaped by the conditioning module (16) with a view to their spatialization by the computing unit (CPU2 ). 9- Sound spatialization system according to one of the preceding claims, characterized in that the calculation unit (CPU2) comprises a processor interface (203) in connection with the data presentation unit (CPU1) and a computer ( 202) for the generation of spatialized monophonic channels (L, R). 10- Sound spatialization system according to claim 9, characterized in that the system comprising a sound database (14), the processor interface (203) comprises buffer registers for the transfer functions of the filter database ( 13) and the sounds from the sound database (14). 11- Spatialization system according to one of claims 9 or

10, characterized in that the computer (202) is produced by a programmable component of the EPLD type.

12- Spatialization system according to one of claims 10 or

11, characterized in that the computer (202) comprises a module (204) for activating and selecting sources, performing the function of mixing between “live” communications and the sounds of the sound database (14).

13- Spatialization system according to one of claims 9 to

12, characterized in that the computer (202) comprises a double spatialization module (205), which receives the adapted transfer functions and performs the convolution with the monophonic signal to be spatialized. 14 - Spatialization system according to one of claims 9 to 13, characterized in that the computer (202) comprises a module (206) of soft switching produced by double linear weighting ramp.

15- Spatialization system according to one of claims 9 to 14, characterized in that the computer (202) comprises a module (208) for simulating atmospheric absorption.

16- Spatialization system according to one of claims 9 to 15, characterized in that the computer (202) comprises a dynamic weighting module (209) and a summation module (210) for performing the weighted sum of the channels of each channel and provide a single stereophonic signal compatible with the output dynamics.

17- Modular avionics information processing equipment (40) comprising a high speed bus (41) to which the sound spatialization system (42) according to one of the preceding claims is connected via the presentation processor. data (CPU1).