GB2403114A - Audio user interface with movable synthesised sound sources - Google Patents

Abstract

An audio user interface is provided in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate. The field is explorable both by rotating it about a predetermined axis and by displacing it in a direction parallel to this axis. In one embodiment, the sound sources (40) are notionally arranged as on discrete floors (1 to 4) of a building with only the sound sources on the current floor being audible; in this case, summary sound sources (60, 61) are provided in the audio field to indicate what sound sources are available above and below the current floor.

Description

24031 1 4 Audio User Interface

Field of the Invention

The present invention relates to an audio user interface.

Backaround of the Invention

The human auditory system, including related brain functions, is capable of localizing sounds in three dimensions notwithstanding that only two sound inputs are received (left and right ear). Research over the years has shown that localization in azimuth, elevation and range is dependent on a number of cues derived from the received sound. The nature of these cues is outlined below.

Azimuth Cues - The main azimuth cues are Interaural Time Difference (ITD sound on the right of a hearer arrives in the right ear first) and Interaural lutensity Difference (I1O sound on the right appears louder in the right ear). ITD and IIT cues are complementary inasmuch as the former works better at low frequencies and the latter better at high frequencies.

Elevation Cues - The primary cue for elevation depends on the acoustic properties of the outer ear or pinna. In particular, there is an elevation-dependent frequency notch in the response of the ear, the notch frequency usually being in the range 6 -16 kHz depending on the shape of the hearer's pinna. The human brain can therefore derive elevation information based on the strength of the received sound at the pinna notch frequency, having regard to the expected signal strength relative to the other sound frequencies being received.

Range Cues - These include: - loudness (the nearer the source, the louder it will be; however, to be useful, something must be known or assumed about the source characteristics), - motion parallax (change in source azimuth in response to head movement is range dependent), and - ratio of direct to reverberant sound (the fall-off in energy reaching the ear as range increases is less for reverberant sound than direct sound so that the ratio will be large for nearby sources and small for more distant sources).

It may also be noted that in order avoid source-localization errors arising from sound reflections, humans localize sound sources on the basis of sounds that reach the ears first (an exception is where the direct/reverberant ratio is used for range determination). s

Getting a sound system (sound producing apparatus) to output sounds that will be localized by a hearer to desired locations, is not a straightforward task and generally requires an understanding of the foregoing cues. Simple stereo sound systems with left and right speakers or headphones can readily simulate sound sources at different azimuth positions; however, adding variations in range and elevation is much more complex. One known approach to producing a 3D audio field that is often used in cinemas and theatres, is to use many loudspeakers situated around the listener (in practice, it is possible to use one large speaker for the low frequency content and many small speakers for the high-frequency content, as the auditory system will tend to localize on the basis of the high frequency component, this effect being known as the Franssen effect). Such many-speaker systems are not, however, practical for most situations.

For sound sources that have a fixed presentation (non-interactive), it is possible to produce convincing 3D audio through headphones simply by recording the sounds that would be heard at left and right eardrums were the hearer actually present. Such recordings, known as binaural recordings, have certain disadvantages including the need for headphones, the lack of interactive controllability of the source location, and unreliable elevation effects due to the variation in pinna shapes between different hearers.

To enable a sound source to be variably positioned in a 3D audio field, a number of systems have evolved that are based on a transfer function relating source sound pressures to ear drum sound pressures. This transfer function is known as the Head Related Transfer Function ( HRTF) and the associated impulse response, as the Head Related Impulse Response (HRIR). If the HRTF is known for the left and right ears, binaural signals can be synthesized from a monaural source. By storing measured HRTF (or HRIR) values for various source locations, the location of a source can be interactively varied simply by choosing and applying the appropriate stored values to the sound source to produce left and right channel outputs. A number of commercial 3D audio systems exist utilizing this principle. Rather than storing values, the HRTF can be modeled but this requires considerably more processing power.

S The generation of binaural signals as described above is directly applicable to headphone systems. However, the situation is more complex where stereo loudspeakers are used for - sound output because sound from both speakers can reach both ears. In one solution, the transfer functions between each speaker and each ear are additionally derived and used to try to cancel out cross-talk from the left speaker to the right ear and from the right speaker to the left ear.

Other approaches to those outlined above for the generation of 3D audio fields are also possible as will be appreciated by persons skilled in the art. Regardless of the method of generation ofthe audio field, most 3D audio systems are, in practice, generally effective In achieving azimuth positioning but less effective for elevation and range. However, in many applications this is not a particular problem since azimuth positioning is normally the most important. As a result, systems for the generation of audio fields giving the perception of physically separated sound sources range from full 3D systems, through two dimensional systems (giving, for example, azimuth and elevation position variation), to one dimensional systems typically giving only azimuth position variation (such as a standard stereo sound system). Clearly, 2D and particularly ID systems are technically less complex than 3D systems as illustrated by the fact that stereo sound systems have been around for very many years.

In terms of user experience, headphone-based systems are inherently "head stabilized" - that is, the generated audio field rotates with the head and thus the position of each sound source appears stable with respect to the user's head. In contrast, loudspeaker-based systems are inherently "world stabilized" with the generated audio field remaining fixed as the user rotates their head, each sound source appearing to keep its absolute position when the hearer's head is turned. In fact, it is possible to make headphone-based systems "world stabilized" or loudspeaker-based systems "head stabilized" by using head-tracker apparatus to sense head rotation relative to a fixed frame of reference and feed corresponding signals to the audio field generation system, these signals being used to modify the sound source positions to achieve the desired effect. A third type of stabilization is also sometimes used in which the audio field rotates with the user's body rather than with their head so that a user can vary the perceived positions of the sound sources by rotating their head; such "body stabilized" systems can be achieved, for example, by using a loudspeaker-based system with small loudspeakers mounted on the user's upper body or by a headphone - based system used in conjunction with head tracker apparatus sensing head rotation relative to the user's body.

As regards the purpose of the generated audio field, this is frequently used to provide a complete user experience either alone or in conjunction with other artificially-generated sensory inputs. For example, the audio field may be associated with a computer game or other artificial environment of varying degree of user immersion (including total sensory immersion). As another example, the audio field may be generated by an audio browser operative to represent page structure by spatial location.

Alternatively, the audio field may be used to supplement a user's real world experience by providing sound cues and information relevant to the user's current real-world situation. In this context, the audio field is providing a level of"augmented reality".

It is an object of the present invention to provide an organisation of sound sources in an audio field suitable for use in an audio user interface.

Summary of the Invention

According to one aspect of the present invention, there is provided an audio user- interfacing method in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source from where sounds related to the item appear to emanate, the method comprising the steps of: (a) determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field, (b) generating, using audio output devices, an audio field in which said sound sources are synthesized at their associated rendering positions to provide sounds related to the items concerned; (c) exploring the audio field by rotating it about a predetermined axis; and (d) exploring the audio by displacing it in a direction parallel to said axis; S with steps (c) and (d) being effected in any order or together.

According to another aspect of the present invention, there is provided apparatus for providing an audio user interface in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source from where sounds related to the item appear to emanate, the apparatus comprising: - rendering-position determining means for determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field, the rendering-position determining means comprising: - means for setting the location of each said sound source relative to an audio

field reference;

- offset means for controlling an offset between the audio-field reference and a presentation reference determined by a mounting configuration of the audio output devices, the offset means including user input means and being operative to enable a user both: - to set a rotation of the audio field about a predetermined axis, and - to set a displacement of the audio field relative to the presentation reference in a direction parallel to said axis; - means for deriving the rendering position of each sound source based on the location of the sound source in the audio field and said offset; and - rendering means, including audio output devices, for generating an audio field in which said sound sources are synthesized at their associated rendering positions to provide sounds related to the items concerned.

Brief Description of the Drawines

Embodiments of the invention will now be described, by way of nonlimiting example, with reference to the accompanying diagrammatic drawings, in which: Figure 1 is a functional block diagram of a first audio-field generating apparatus; Figure 2 is a diagram illustrating a coordinate system for positions in a spherical

audio field;

Figure 3 is a diagram illustrating rotation of an audio field relative to a presentation reference vector; 5. Figure 4 is a diagram illustrating a user exploring a body-stabilized audio field by head rotation; Figure 5 is a diagram illustrating a user exploring a body-stabilized audio field by

rotating the field in azimuth;

Figure 6 is a diagram illustrating a general cylindrical organization of an audio field; 10. Figure 7 is a diagram illustrating a first specific form of the Figure 6 cylindrical organization; Figure 8 is a diagram illustrating a second specific form of the Figure 6 cylindrical organization; and Figure 9 is a functional block diagram of a variant of the Figure 1 apparatus.

Best Mode of Carrvinz Out the Invention The forms of apparatus to be described below are operative to produce an audio field to serve as an audio interface to services such as communication services (for example, e mail, voice mail, fax, telephone, etc.), entertainment services (such as interact radio), information resources (including databases, search engines and individual documents), transactional services (for example, retail and banking web sites), augmented-reality services, etc. When the apparatus is in a "desktop" mode, each service is represented in the audio field through a corresponding synthesized sound source presenting an audio label (or "earcon") for the service. The audio label associated with a service can be constituted by any convenient audio element suitable for identifying that service - for example, an audio label can be the service name, a short verbal descriptor, a characteristic sound orjingle, or even a low-level audio feed from the service itself. The sound sources representing the services are synthesized to sound, to a user, as though they exist at respective locations in the audio field using any appropriate spatialisation method; these sound sources do not individually exist as physical sound output devices though, of course, such devices are involved in the process of synthesizing the sound sources. Furthermore, the sound sources only have a real-world existence to the extent that service-related sounds are presented at the sound source locations. Nevertheless, the concept of sound sources located at specific locations in the audio field is useful as it enables the sound content that is to be presented in respect of a service to be disassociated from the location and other presentation parameters for those sounds, these parameters being treated as associated with the corresponding sound source.

Thus, the present specification is written in terms of such sound sources spatialized to

specific locations in the audio field.

Upon a service presented through a sound source being selected (in a manner to be described hereinafter), the apparatus changes from the desktop mode to a service mode in which only the selected service is ouipuL, a filli service audio feed now bemg presented in whatever sound spatialisation is appropriate for the service. When a user has finished using the selected service, the user can switch back to the desktop mode.

It will be appreciated that other possibilities exist as to how the services are presented and accessed - for example, the feed from a selected service can be output simultaneously with background presentation of audio labels for the other available services. Furthermore, a service can provide its data in any form capable of being converted in audible form; for example, a service may provide its audio label in text form for conversion by a text-to- speech converter into audio signals, and its full service feed as digitised audio waveform signals.

It is also possible in the desktop mode to use more than one sound source to represent a particular service and/or to associate more than one audio label with each sound source as will be seen hereinafter.

Audio Field Organisation- Spherical Field Example

Considering now the first apparatus (Figure I), in the form of the apparatus primarily to be described below, the audio field is a 2D audio field configured as the surface of a sphere I (or part of a sphere). Such a spherical-surface audio field is depicted in Figure 2 where a spatialised sound source 40 (that is, a service audio label that has been generated so as to appear to come from a particular location in the audio field) is represented as a hexagon positioned on the surface of a sphere 41 (illustrated in dashed outline). Itmaybenotedthat although such a spherical surface exists in three- dimensional space, the audio field is considered to be a 2 dimensional field because the position of spatialised sound sources in the audio field, such as source 40, can be specified by two orthogonal measures; in the present case these measures are an azimuth angle X and an elevation angle Y . The azimuth angle is measured relative to an audio-field reference vector 42 that lies in a horizontal plane 43 and extends from the centre of sphere 41. The elevation angle is the angle between the horizontal and the line joining the centre of the sphere and the sound source 40.

In fact, the Figure 1 apparatus is readily adapted to generate a 3L) audio field with the third dimension being a range measure Z. also depicted in Figure 2, that is the distance from the centre of sphere 41 to the spatialised sound source 40. Conversely, the Figure 1 apparatus can be adapted to generate a ID audio field by doing away with the elevation dimension of the spatialised sound sources.

The Figure 1 apparatus supports azimuth rotation of the audio field, this potentially being required for implementing a particular stabilization (that is, for example, head, body, vehicle or world stabilization) of the audio field as well as providing a way for the user to explore the audio field by commanding a particular rotation of the audio field. As is illustrated in Figure 3, the azimuth rotation of the field can be expressed in terms of the angle R between the audio-field reference vector 42 and a presentation reference vector 44.

This presentation reference vector corresponds to the straight-ahead centreline direction for the configuration of audio output devices 11 being used. Thus, for a pair of fixed, spaced loudspeakers, the presentation reference vector 44 is the line of equidistance from both speakers and is therefore itself fixed relative to the world; for a set of headphones, the presentation reference vector 44 is the forward facing direction of the user and therefore changes its direction as the user turns their head. When the field rotation angle R=0 , the audio-field reference vector 42 is aligned with the presentation reference vector 44. The user is at least notionally located at the origin of the presentation reference vector.

The actual position at which a service-representing sound source is to be rendered in the audio output field (its "rendering position") by the Figure 1 apparatus, must be derived relative to the presentation reference vector since this is the reference used by the spatialisation processor 10 of the apparatus. The rendering position of a sound source is a combination of the intended position of the source in the audio field judged relative to the audio-field reference vector, and the current rotation of the audio field reference vector relative to the presentation reference vector.

As already intimated, apart from any specific azimuth rotation of the audio field deliberately set by the user, the audio field may need to be rotated in azimuth to provide a particular audio-field stabilization. Whether this is required depends on the selected audio field stabilization and the form of audio output devices. Thus, by way of example, unless otherwise stated, it will be assumed below that the audio output devices 11 of Figare 1 apparatus are headphones and the audio field is to be body-stabilised so that the orientation! of the audio field relative to the user's body is unaltered when the user turns their head this is achieved by rotation of the audio field relative to the presentation reference vector for which purpose a suitable head-tracker sensor 33 is provided to measure the azimuth rotation of the user's head relative to its straight-ahead position (that is, relative to the user's body). As the user turns their head, the angle measured by sensor 33 is used to rotate the audio field by the same amount but in the opposite direction thereby stabilising the rendering positions of the sound sources relative to the user's body.

It will be appreciated that had it been decided to head-stabilise the field, then for audio output devices in the form of headphones, it would have been unnecessary to modify the orientation of the audio field as the user turned their head and, in this case, there would be no need for the head-tracker sensor 33. This would also be true had the audio output devices 11 taken the form of fixed loudspeakers and the audio field was to be world- stabilized. Where headphones are to be used and the audio field is to be world stabilised, the orientation of the audio field must be modified by any change in orientation of the I user's head relative to the world, whether caused by the user turning their head or by body movements; a suitable head-tracker can be provided by a head-mounted electronic compass. Similarly, if the audio output devices 11 are to be provided by a vehicle sound system and the audio field is to be world stabilised, the orientation of the audio field must be modified by any change in orientation of the vehicle as determined by any suitable sensor. It may be generally be noted that where a user is travelling in a vehicle, the latter serves as a local world so that providing vehicle stabilization of the audio field is akin to providing world stabilization (whether the audio output devices are headphones, body mounted or vehicle mounted) but with any required sensing of user head/body rotation relative to the world now being done with respect to the vehicle.

It is also to be noted that the audio-field rotation discussed above only concerned azimuth rotation - that is, rotation about a vertical axis. It is, of course, also possible to treat rotation of the field m elevation in a similar manner both to track head movements (nodding up and down) to achieve a selected stabilization and to enable the user to command audio-field elevation-angle changes; appropriate modifications to the Figure 1 apparatus to handle rotation in elevation in this way will be apparent to persons skilled in the art.

Considering Figure 1 in more detail, services are selected by subsystem 13, these services being either local (for example, an application running on a local processor) or accessible via a communications link 20 (such as a radio link or fixed wire connection providing internet or intranet access). The services can conveniently be categorised into general services such as e-mail, and services that have relevance to the immediate vicinity (augmentation services). The services are selected by selection control block 17 according to predetermined user-specified criteria and possibly also by real-time user input provided via any suitable means such as a keypad, voice input unit or interactive display.

A memory 14 is used to store data about the selected services with each such service being given a respective service ID. For each selected service, memory 14 holds access data (e.g. address of service executable or starting URL) and data on the or each sound source specified by the service or user to be used to represent the service with each such sound source being distinguished by a suitable suffix to the service ID. For each sound source, the memory holds data on the or each associated audio label, each label being identified by a further suffix to the suffixed service ID used to identify the sound source. The audio labels for the selected services are either provided by the services themselves to the subsystem 13 or are specified by the user for particular identified services. The labels are preferably provided and stored in text-form for conversion to audio by a text-to-speech converter (not shown) as and when required by the spatialisation processor. Where the audio label associated with a service is to be a low-level live feed, memory 14 holds an indicator indicating this. Provision may also be made for temporarily replacing the normal audio label of a service sound source with a notification of a significant service-related event (for example, where the service is an e-mail service, notification of receipt of a message may temporarily substitute for the normal audio label of the service).

As regards the full service feed of any particular service, this is not output from subsystem 13 until that service is chosen by the user by input to output selection block 12.

Rather than the services to be represented in the audio interface being selected by block 17 from those currently found to be available, a set of services to be presented can be pre- specified and the related sound-source data (including audio labels) for these services stored in memory 14 along with service identification and access data. In this case, when the apparatus is in its "desktop" mode, the services in the pre-specified set of services are represented in the output audio field by the stored audio labels without any need to first contact the services concerned; upon a user selecting a service and the apparatus changing to its service mode, the service access data for the selected service is used to contact that service for a full service feed.

With respect to the positioning ofthe service-representing sound sources in the audio field when the apparatus is in its desktop mode, each service may provide position information either indicating a suggested spatialised position in the audio field for the sound source(s) through which the service is to be represented, or giving a real-world location associated with the service (this may well be the case in respect of an augmented reality service associated with a location in the vicinity of the user). Where a set of services is pre specified, then this position information can be stored in memory 14 along with the audio labels for the services concerned.

For each service-representing sound source, it is necessary to determine its final rendering position in the output audio field taking account of a number of factors. This is done by injecting a sound-source data item into a processing path involving elements 21 to 30. This sound-source data item comprises a sound source ID (such as the related suffixed service ID) for the sound source concerned, any service-supplied position information for the sound source, and possibly also the service type (general service / augmentation service).

The subsystem 13 passes each sound-source data item to a source-position set/modify block 23 where the position of the sound source is decided relative to the audio-field reference vector, either automatically on the basis of the supplied type and/or position information, or from user input 24 provided through any suitable input device including a keypad, keyboard, voice recognition unit, or interactive display. I'hese positions are constrained to conform to the desired form (spherical or part spherical; ID, 2D, or 3D) of the audio field. The decided position for each source is then temporarily stored in memory against the source ID.

Provision of a user input device for modifying the position of each sound source relative to the audio field reference, enables the user to modify the layout ofthe service-representing sound sources (that is, the dispositions of these sound sources relative to each other) as desired.

With respect to a service having an associated real-world location (typically, an augmented realityservice), whilst it is possible to position the corresponding sound source in the audio field independently of the relationship between the associated real-world location of the service and the location ofthe user, it will often be desired to place the sound source in the field at a position determined by the associated real-world location and, in particular, in a position such that it lies in the same direction relative to the user as the associated real world location. In this latter case, the audio field will generally be worldstabilised to maintain the directional validity of the sound source in the audio field presented to the user; for the same reason, user- commanded rotation ofthe audio field should be avoided or inhibited. Positioning a sound source according to an associated real-world location is achieved in the present apparatus by a real-world location processing functional block 21 that forms part of the source-position set/modify block 23. The real-world location processing functional block 21 is arranged to receive and store real-world locations passed to it from subsystem 13, these locations being stored against the corresponding source IDs.

Block 21 is also supplied on input 22 with the current location ofthe user determined by any suitable means such as a GPS system carried by the user,or nearby location beacons (such as may be provided at point-of-sale locations). The block 21 first determines whether the real-world location associated with a service is close enough to the user to qualify the corresponding sound source for inclusion in the audio field; if this test is passed, the azimuth and elevation coordinates of the sound source are set to place the sound source in the audio field in a direction as perceived by the user corresponding to the direction ofthe real world location from the user. This requires knowledge of the real-world direction of pointing of the un- rotated audio-field reference vector 42 (which, as noted above, is also the direction of pointing of the presentation reference vector). This can be derived for example, by providing a small electronic compass on a structure carrying the audio output devices 11, since this enables the real-world direction of pointing of presentation reference vector 44 to be measured; by noting the rotation angle of the audio-field reference vector 42 at the moment the real-world direction of pointing of vector 44 is measured, it is then possible to derive the real-world direction of pointing ofthe audio-field reference vector42 (assuming that the audio field is being world-stabilised). It maybe noted that not only will there normally be a structure carrying the audio output devices 11 when these are constituted by headphones, but this is also the case in any mobile situation (for example, in a vehicle) where loudspeakers are involved.

If the audio field is a 3D field, then as well as setting the azimuth and elevation coordinates of the sound source to position it in the same direction as the associated real-world location, block 21 also sets a range coordinate value to represent the real world distance between the user and the real-world location associated with the sound source.

Of course, as the user moves in space, the block 21 must reprocess its stored real-world location information to update the position ofthe corresponding sound sources in the audio field. Similarly, if updated real-world location information is received from a service, then the positioning of the sound source in the audio field must also be updated.

Returning to a general consideration of the Figure I apparatus, an audiofield orientation modify block 26 is used to specify any required changes in orientation (angular offset) of the audio-field reference vector relative to presentation reference vector. In the present example where the audio field is to be body-stabilized and the output audio devices are headphones, the apparatus includes the afore-mentioned head tracker sensor 33 and this sensor is arranged to provide a measure of the turning of a user's head relative to their body to a first input 27 of the block 26. This measure is combined with any user-commanded field rotation supplied to a second input of block 26 in order to derive a field orientation angle that is stored in memory 29.

As already noted, where headphones are used and the audio field is to be world stabilised (for example, where augmented-reality service sound sources are to be maintained in positions in the field consistent with their real world positions relative to the user), then the head-tracker sensor needs to detect any change in orientation ofthe user's head relative to the real world so that the audio field can be given a counter rotation. Where the user is travelling in a vehicle and the audio field is to be vehicle- stabilised, the rotation of the user's head is measured relative to the vehicle (the user's "local" world, as already noted).

Each source position stored in memory 25 is combined by combiner 30 with the field orientation (rotation) angle stored in memory 29 to derive a rendering position for the sound source, this rendering position being stored, along with the source ID, in memory 15.

The combiner operates continuously and cyclically to refresh the rendering positions in memory 1 5.

Output selection block 12 sets the current apparatus mode according to user input, the available modes being a desktop mode and a service mode as already discussed above.

When the desktop mode is set, the spatialisation processor 10 accesses the rendering position memory 15 and the memory 14 holding the service audio labels to generate an audio field, via audio output devices 11, in which the (or the currently-specified) audio label associated with each sound source is spatialized to a position set by the corresponding rendering position in memory 15. In generating the audio-label field, the processor 10 can function asynchronously with respect to the combiner 30 due to the provision of memory 15. The spatialisation processor 10 operates according to any appropriate sound spatialisation method, including those mentioned in the introduction to the present specification. The spatialisation processor 10 and audio output devices together form a rendering subsystem serving to render each sound source at its derived final rendering position.

When the service mode is set, the full service audio feed for the chosen service is rendered by the spatialisation processor 10 according to whatever position information is provided by the service. It will be appreciated that, although not depicted, this service position information can be combined with the field orientation angle information stored in memory 29 to achieve the same stabilization as for the audio-field containing the service audio labels; however, this is not essential and, indeed, the inherent stabilization of the audio output devices (head-stabilised in the case of headphones) maybe more appropriate for the full service mode.

As an alternative to the full service feed being spatialised by the spatialisation processor 10, the full service feed may be provided as pre-spatialized audio signals and fed directly to the audio output devices.

With the Figure 1 apparatus set to provide a body-stabilised audio field through headphones, the user can explore the audio field in two ways, namely by turning their head and by rotating the audio field. Figure 4 illustrates a user turning their head to explore a 2D audio field restricted to occupy part only of a spherical surface. In this case, six spatialised sound sources 40 are depicted. Ofthese sources, one source 40A is positioned in the audio field at an azimuth angle of X1 and elevation angle Y1 relative to the audio-field reference vector 42. The user has not commanded any explicit rotation of the audio field.

However, the user has turned their head through an angle X2 towards the source 40A. In order to maintain body-stabilisation of the audio field, the audio-field reference vector 42 - has been automatically rotated an angle (- X2 ) relative to the presentation reference vector 44 to bring the vector 42 back in line with the user's body straight ahead direction; the rendering position of the source relative to the presentation reference vector is therefore: Azimuth = X1 - X2 Elevation = Y1 this being the position output by combiner 30 and stored in memory 15. The result is that turning of the user's head does indeed have the effect of turning towards the sound source 40A.

Figure 5 illustrates, for the same audio field as represented in Figure 4, how the user can bring the sound source 40A to a position directly ahead of the user by commanding a rotation of (-X1 ) ofthe audio field by user input 28 to block 26 (effected, for example, by a rotary input device). The azimuth rendering position of the sound source 40A becomes (Xl X1 ), that is, 0 - the source 40A is therefore rendered in line with the presentation reference vector 44. Of course, if the user turns their head, the source 40A will cease to be directly in front of the user until the user faces ahead again.

Audio Field Organisation- Cylindrical Field Example The Figure 1 apparatus can be adapted to spatialize the sound sources 40 in an audio field conforming to the surface of a vertically-orientated cylinder (or part thereof). Figure 6 depicts a general case where the audio field conforms to a notional cylindrical surface 50.

This cylindrical audio field, like the spherical audio field previously described with reference to Figure 2, is two dimensional inasmuch as the position of a sound source 40 in the field it can be specified by two coordinates, namely an azimuth angle X and an elevation (height) distance Y. both measured relative to an horizontal audio-field reference vector 52. It will be appreciated that a 3D audio field can be specified by adding a range coordinate Z. this being the distance from the axis of the cylindrical audio field. As with the spherical audio field described above, the cylindrical audio field may be rotated (angularly offset by angle R ) relative to a presentation reference vector 54, this being done either in response to a direct user command or to achieve a particular field stabilisation in the salne manner as already described above for the spherical audio field. In addition, the audio field can be axially displaced to change the height (axial offset) of the audio-fil]d reference vector 52 relative to the presentation reference vector 54.

Since it is possible to accommodate any desired number of sound sources in the audio field without over crowding simply by extending the elevation axis, there is a real risk of a "Tower of Babel" being created if all sound sources are active together. Accordingly, the general model of Figure 6 employs a concept of a focus zone 55 which is a zone of the cylindrical audio field bounded by upper and lower elevation values determined by a currently commanded height H so as to keep the focus zone fixed relative to the assumed user position (the origin of the presentation reference vector); within the focus zone, the sound sources 40 are active, whilst outside the zone the sources 40 are muted (depicted by dashing of the hexagon outline of these sources in Figure 6) except for a limited audio leakage 56. In Figure 6, the focus zone (which is hatched) extends by an amount C above and below the commanded height H (and thus has upper and lower elevation values of (H + C) and (H - C) respectively. In the illustrated example, H=0 and C is a constant; C need not be constant and it would be possible, for example, to make its value dependent on the value of the commanded height H. The general form of cylindrical audio field shown in Figure 6 can be implemented in a variety of ways with respect to how leakage into the focus zone is effected and how a user moves up and down the cylindrical field (that is, changes the commanded height and thus the current focus zone). Figures 7 and 8 illustrate two possible implementations in the case where the audio field is of semi-cylindrical form (azimuth range from +90 to -90 ).

In Figure 7, leakage takes the form of the low-volume presence of sound sources 40W in upper and lower "whisper" zones 56, 57 positioned adjacent the focus zone 55. Also, the commanded height value is continuously variable (as opposed to being variable in steps).

The result is that the user can effectively slide up and down the cylinder and hear both the sound sources 40 in the focus zone and, at a lower volume, sound sources 40W in the whisper zones.

In Figure 8, the service sound sources are organised to lie at a number of discrete heights, in this case, four possible heights effectively corresponding to four "floors" here labelled "l" to "4". Preferably, each "floor" contains sound sources associated with services all of the same type with different floors being associated with different service types. The user can only command step changes in height corresponding to moving from floor to floor (the extent ofthe focus zone encompassing one floor). Leakage takes the form of an upper and lower advisory sound source 60, 61 respectively positioned just above and just below the focus zone at an azimuth angle of 0 . Each ofthese advisory sound sources 60, 61 provides a summary of the services (for example, in terms of service types) available respectively above and below the current focus zone. This permits a user to determine whether they need to go up or down to find a desired service.

It will be appreciated that the forms of leakage used in Figures 7 and 8 can be interchanged or combined and that the Figure 8 embodiment can provide for sound sources 40 on il.re same floor to reside at different heights on that floor. It is also possible to provide each floor of the Figure 8 embodiment with a characteristic audio theme which rather than being associated with a particular source (which is, of course, possible) is arranged to surround the user with no directionality; by way of example, a floor containing museum services could have a classical music theme.

In arranging for the Figure 1 apparatus to implement a cylindrical audio field such as depicted in any of Figures 4-6, the positions set for the sound sources by block 23 are specified in terms ofthe described cylindrical coordinate system and are chosen to conform to a cylindrical or part-cylindrical organization in 1, 2, or 3D as required. The orientation and vertical positioning of the audio field reference vector 42 are set by block 26, also in terms of the cylindrical coordinate system. Similarly, combiner 30 is arranged to generate the sound-source rendering positions in terms of cylindrical coordinates. The spatialisation processor must therefore either be arranged to understand this coordinate system or the rendering positions must be converted to a coordinate system understood by the spatialisation processor 10 before they are passed to the processor. This latter approach is preferred and thus, in the present case, assuming that the spatialisation processor is arranged to operate in terms of the spherical coordinate system illustrated in Figure 2, a converter 66 (see Figure 9) is provided upstream of memory 15 to convert the rendering positions from cylindrical coordinates to spherical coordinates.

Whilst it would be possible to use a single coordinate system throughout the apparatus regardless of the form of audio field to be produced (for example, the positions of the sound sources in the cylindrical audio field could be specified in spherical coordinates), this complicates the processing because with an appropriately chosen coordinate system most operations are simple additions or subtractions applied independently to the individual coordinates values ofthe sound sources; in contrast, if, for example, a spherical coordinate system is used to specify the positions in a cylindrical field, then commanded changes in the field height (discussed further below) can no longer simply be added/subtracted to the sound source positions to derive their rendering heights but instead involve more complex processing affecting both elevation angle and range. Indeed, by appropriate choice of coordinate system for different forms of audio field, equivalent operations with respect to the fields translate to the same operations (generally add/subtract) on the coordinate values being used so that the operation ofthe elements 25, 26, 29 and 30 of the apparatus is unchanged. In this case, adapting the apparatus to a change in audio-field form, simple requires the block 23 to use an appropriate coordinate system and for converter 66 to be set to convert from that coordinate system to that used by the spatialisation processor 10.

With respect to adaptation of the Figure I apparatus to provide the required capability of commanding changes in height for the cylindrical audio field systems illustrated in Figures 4-6, such height changes correspond to the commanding of changes in the elevation angle already described for the case of a spherical audio field. Thus, a height change command is supplied to the block 26 to set a field height value (an axial offset between the field reference vector and the presentation reference vector) which is then combined with the elevation distance value Y of each sound source to derive the elevation value for the rendering position of the source.

As regards how the focus zone and leakage features are implemented, Figure 9 depicts a suitable variation of the Figure 1 apparatus for providing these features. In particular, a source parameter set/modifyblock 70 is interposed between the output of combiner 30 and the converter 66. This block 70 comprises one or more units for setting and/or modifying one or more parameters associated with each sound source to condition how the sound source is to be presented in the audio field. The block 70 can include a range of different type of units that may modify the rendering position of a source and/or set various sounding effect parameters for the source. In the present case, the block 70 comprises a cylindrical filter 71 that sets a audibility (volume level) sounding-effect parameter for each sound source. The set parameter value is passed to memory 15 for storage along with the source ID and rendering position. When the spatialisation processor comes to render the sound source audio label according to the position and audibilityparametervalue stored in memory 15, it passes the audibility value to a sounding effector 74 that conditions the audio label appropriately (in this case, sets its volume level).

In the case of the Figure 7 arrangement, the cylinder filter 7 l is responsive to the current field height value (as supplied from memory 29 to a reference input 72 of block 70) to set the audibility parameter value of each sound source: to 100% (no volume level reduction) for sound sources in the focus zone 55; to 50% for sound sources in the "whisper" zones 56 and57; and to 0% (zero volume) for all other sound sources. As a result, the sounding effecter 74 mutes out all sound sources not in the focus or whisper zones, and reduces the volume level of sound sources in the whisper zones.

In the case of the Figure 8 arrangement, the cylinder filter 71 performs a similar function except that now there are no whisper zones. As regards the upper and lower advisory sound sources 60 and 61, the subsystem 13 effectively creates these sources by: - creating a ghost advisory service in memory 14 with two sound sources, the IDs of these sources being passed to block 23 as for any other service; - creating for each sound source a respective set of summary audio labels, each set being stored in memory 14 and specifying for each floor an appropriate label summarising the service types either above or below the current floor, depending on the set concerned.

The source IDs passed to the block 23 are there associated with null position data before being passed on via memory25 and combiner 30 to arrive at the cylinder filter 71 of block l 70. The filter 71 recognises the source IDs as upper and lower advisory sound source IDs and appropriately sets position data for them as well as setting the audibility parameter to 100% and setting a parameter specifying which summary audio label is appropriate for the current floor. This enables the spatialisation processor to retrieve the appropriate audio label when it comes to render the upper or lower advisory sound source.

It will be appreciated that partially or fully muting sound sources outside of a focus zone can also be done where the apparatus is set to generate a spherical audio field. In this case, the apparatus includes blocks 70 and 74 but now the cylinder filter 71 is replaced by a "spherical filter" muting out all sound sources beyond a specified angular distance from a current facing direction of the user. The current facing direction relative to the presentation reference vector is derived byblock 26 and supplied to the filter 71. It maybe noted that in the case where the audio output devices 1 1 are constituted by headphones, the direction of facing of the user corresponds to the presentation reference vector so it is a simple matter to determine which sound sources have rendering positions that are more than a given angular displacement from the facing direction. Along with the implementation of a focus zone for a spherical audio field, it is, of course, also possible to provide the described implementations of a leakage feature.

Variants It will be appreciated that many variants are possible to the above described embodiments of the invention. For example, in relation to the cylindrical audio field forms described above, whilst these have been described with the axis of the cylindrical field in a vertical orientation, other orientations of this axis are possible such as horizontal. Also with respect to the cylindrical field form embodiments, it is possible to implement such embodiments without the use of leakage into the focus zone and, indeed, in appropriate circumstances, even without the use of a focus zone.

It will be appreciated that most ofthe functionality ofthe functional blocks ofthe various forms of apparatus described above, will typically be implemented in software for controlling one or more general-purpose or specialised processors according to modern programming techniques. Furthermore, whilst a number of separate memories have been illustrated the described embodiments, it will be appreciated that this is done to facilitate a clear description of the operation of the apparatus; memory organisations and data structures different to those described above are, of course, possible.

It should also be understood that the term "services" as used above has been used very broadly to cover any resource item that it may be useful to indicate to the user in much the same way as a PC visual desktop can be used to represent by visible icons a wide variety of differing resource items including local software applications and individual documents as well as remote services. However, the described forms of apparatus can also be used to present items that are not simply place- holders for underlying services but provide useful information in their own right. 23 i

Claims (18)

1. An audio user-interfacing method in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source from where sounds related to the item appear to emanate, the method comprising the steps of: (a) determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field; (b) generating, using audio output devices, an audio field in which said sound sources are synthesized at their associated rendering positions to provide sounds related to the items concerned; (c) exploring the audio field by rotating it about a predetermined axis; and (d) exploring the audio by displacing it in a direction parallel to said axis; with steps (c) and (d) being effected in any order or together.

2. A method according to claim 1, in which in step (d) the audio field is displaced in said direction in discrete steps of predetermined magnitude.

3. A method according to claim 2, wherein said axis is vertically disposed, the sound sources being notionally grouped at differing levels corresponding to floors of a building, the predetermined magnitude of said discrete steps corresponding to moving up or down one floor.

4. A method according to claim 2, wherein the sound sources are arranged in groups with the sound sources in each group being at the same position along said axis and the groups being separated one from another along said axis by distances corresponding to multiples, including one, of said predetermined magnitude.

5. A method according to claim 1, wherein sound sources located in the audio field outside of a focus zone fixed relative to a notional user position, are at least partially muted relative to sound sources inside the focus zone; the sound sources being un-muted and muted as they move into and out of the focus zone in response to displacement of the audio field in said direction parallel to the predetermined axis. Id

6. A method according to claim S. wherein sound sources adjacent to, but outside of, the focus zone are partially muted whilst those further from the focus zone are fully muted.

7. A method according to claim 5, wherein sound sources outside of the focus zone are fully muted, an audio indication of the sound sources existing beyond the focus zone in at least one direction along said axis being provided un-muted in the audio field.

8. A method according to any one of claims 1 to 7, wherein the audio field is stabilised relative to one of: - a user's head; - a user's body; - a vehicle in which the user is travelling; - the world; this stabilization taking account of whether the audio output devices are world, vehicle, body or head mounted, and, as appropriate, rotation of the user's head or body, or of the vehicle, about an axis parallel to the said predetermined axis.

9. Amethod according to anyone of claims 1 to 8, wherein said axis is vertically disposed.

10. A method according to any one of claims 1 to 8, wherein said axis is horizontally disposed.

11. A method according to any one of claims 1 to 10, wherein at least some of the said items represented by the sound sources are audio labels for services, the method further involving selecting a service by selecting the corresponding audio-label sound source.

12. Apparatus for providing an audio user interface in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source from where sounds related to the item appear to emanate, the apparatus comprising: - rendering-position determining means for determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound l in the audio field, the rendering- position determining means comprising: - means for setting the location of each said sound source relative to an audio

field reference;

- offset means for controlling an offset between the audio-field reference and a presentation reference determined by a mounting configuration of the audio output devices, the offset means including user input means and being operative to enable a user both: - to set a rotation of the audio field about a predetermined axis, and - to set a displacement of the audio field relative to the presentation reference in a direction parallel to said axis; - means for deriving the rendering position of each sound source based on the location of the sound source in the audio field and said offset; and - rendering means, including audio output devices, for generating an audio field in which said sound sources are synthesized at their associated rendering positions to provide sounds related to the items concerned.

13. Apparatus according to claim 12, wherein the offset is arranged to permit the audio field to be displaced in said direction only in discrete steps of predetermined magnitude.

14. Apparatus according to claim 12, further comprising a muting filter operative to at least partially mute sound sources with rendering positions located in the audio field outside of a focus zone fixed relative to said presentation reference.

15. Apparatus according to claim 14, wherein the muting filter is operative to onlypartially mute sound sources adjacent to, but outside of, the focus zone but to fully mute sound sources further from the focus zone.

16. Apparatus according to claim 14, wherein the muting filter is operative to fully mute sound sources outside of the focus zone, the apparatus including means for providing an un-muted audio indication of the sound sources existing beyond the focus zone in at least one direction along said axis.

21. Apparatus according to any one of claims 14 to 20, wherein at least some of the said items represented by the sound sources are audio labels for services, the apparatus including a selection arrangement for selecting a service by selecting the corresponding audio-label sound source.

22. Apparatus according to claim 14, wherein the offset means further includes stabilization means for varying the said offset such as to stabilise the audio field reference relative to one of: - a user's head; a user's body; - a vehicle mounting the apparatus; - the world.

Amendments to the claims have been filed as follows

1. An audio user-interfacing method in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source Ol1 where sounds related to the item appear to emanate, the method comprising the steps of: (a) determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field; (b) generating, using audio output devices, an audio field in which said sound sources are synthesized at their associated rendering positions to provide sounds related to the items concerned; (c) exploring the audio field by rotating it about a predetermined axis; and (d) exploring the audio by displacing it in a direction parallel to said axis; xvith steps (c) and (d) being effected in any order or togcthe,r.

2. A method according to claim 1, in which in step (d) the audio field is displaced in said direction in discrete steps of predetermined magnitude.

3. A method according to claim 2, wherein said axis is vertically disposed, the sound sources being notionally grouped at differing levels corresponding to floors of a building, the predetermined magnitude of said discrete steps corresponding to moving up or down one floor.

4. A method according to claim 2, wherein the sound sources are arranged in groups with the sound sources in each group being at the same position along said axis and the groups being separated one from another along said axis by distances corresponding to multiples, including one, of said predetermined magnitude.

5. A method according to claim 1, wherein sound sources located in the audio field outside of a focus zone fixed relative to a notional user position, are at least partiallymuted relative to sound sources inside the focus zone; the sound sources being un-muted and muted as they move into and out of the focus zone in response to displacement of the audio field in said direction parallel to the predetermined axis.

6. A method according to claim 5, wherein sound sources adjacent to, but outside of, the focus zone are partially muted whilst those further from the focus zone are fully notated.

7. A method according to claim 5, wherein sound sources outside of the focus zone are fully muted, an audio indication of the sound sources existing beyond the focus zone in at least one direction along said axis being provided un-muted in the audio field.

8. A method according to any one of claims 1 to 7, wherein the audio field is stabilised relative to one of: - a user's head; - a user's body; : - a vehicle in which the user is travelling; - the world; this stabilization taking account of whether the audio output devices are world, vehicle, body or head mounted, and, as appropriate, rotation of the user's head or body, or of the! vehicle, about an axis parallel to the said predetermined axis.

9. A method according to any one of claims 1 to 8, wherein said axis is vertically disposed. ] 10. A method according to any one of claims 1 to 8, wherein said axis is horizontally disposed.

11. A method according to any one of claims 1 to 10, wherein at least some of the said items represented by the sound sources are audio labels for services, the method further involving selecting a service by selecting the corresponding audio-label sound source.

12. Apparatus for providing an audio user interface in which each of a plurality of items is represented in an audio field by at least one respective synthesized sound source from where sounds related to the item appear to emanate, the apparatus comprising: - rendering-position determining means for determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound i in he audio field, the rendering-position determining means comprising: - means for setting the location of each said sound source relative to an audio-

field reference;

- offset means for controlling an offset between +,he audio-fil]d reference and a presentation reference determined by a mounting configuration of the audio output devices, the offset means including user input means and being operative to enable a user both: - to set a rotation of the audio field about a predetermined axis, and - to set a displacement of the audio field relative to the presentation reference in a direction parallel to said axis; - means for deriving the rendering position of each sound source based on the location of the sound source in the audio field and said offset; and À rendering means, including audio output devices, for generating an audio field in which said sound sources are synthesized at their associated rendering 1 S positions to provide sounds related to the items concerned.

13. Apparatus according to claim 12, wherein the offset is arranged to permit the audio field to be displaced in said direction only in discrete steps of predetermined magnitude.

14. Apparatus according to claim 12, further comprising a muting filter operative to at least partially mute sound sources with rendering positions located in the audio field outside of a focus zone fixed relative to said presentation reference.

15. Apparatus according to claim 14, wherein the muting filter is operative to onlypartially mute sound sources adjacent to, but outside of, the focus zone but to fully mute sound sources further from the focus zone.

16. Apparatus according to claim 14, wherein the muting filter is operative to fully mute sound sources outside of the focus zone, the apparatus including means for providing an un-muted audio indication of the sound sources existing beyond the focus zone in at least one direction along said axis.

17. Apparatus according to any one of claims 14 to 16, wherein at least some of the said items represented by the sound sources are audio labels for services, the apparatus including a selection arrangement for selecting a service by selecting the corresponding audio-label sound source.

18. Apparatus according to claim 14, wherein the offset means further includes stabilization means for varying the said offset such as to stabilise the audio field reference 1 relative to one of: - a user's head; - a user's body; - a vehicle mounting the apparatus; - the world.