RU2780508C2 - Method and system for use of time effects in multichannel audio playback system - Google Patents

Abstract

FIELD: acoustics.

SUBSTANCE: method for processing of signals for creation of time effects in N-channel input audio signal for playback on a group of loudspeakers having a set configuration is proposed. The first M-channel audio signal is formed of N-channel input audio signal. Each channel of the first M-channel audio signal is associated with a subgroup of loudspeakers. The second M-channel audio signal is formed of the first M-channel audio signal in accordance with MxM matrix, wherein each element a_ij in MxM matrix includes a delay parameter. The minimum value of the delay parameter in each element a_ij is determined in accordance with a distance between at least two loudspeakers in at least one of subgroups i and j of loudspeakers and in accordance with the minimum value of delay of the time effect applied to each channel of the second M-channel audio signal. Then, K-channel audio signal is formed of the specified one second M-channel audio signal or each second M-channel audio signal.

EFFECT: increase in the sound quality.

14 cl, 9 dwg

Description

The field of technology to which the invention belongs

The present invention relates to methods and systems for applying timing effects to a multi-channel audio playback system. Time effects means processing based on, but not limited to, delays and/or reverbs. These effects can be obtained using various methods known in the art, guaranteeing the causality of the signal. The effects can be processed in the time domain, such as in networks with feedback delay, or in the Fourier domain, such as with split convolutional signal processing.

State of the art

Multi-channel audio systems used in large venues such as a concert hall may have more than two speakers to provide more even sound pressure to the audience area. For example, loudspeakers can be installed to the side and back of the audience area to prevent lower sound pressure levels for audiences further away from the stage. This is known as "audio amplification" and consists of playing to the sides and back of the audience the same audio channels that are played on stage or in front of the audience. The term loudspeaker can refer to a single device or multiple speaker heads and devices operating from the same input signal, so that a multi-channel audio system has two or more signals, each of which is reproduced through the speakers.

As an alternative or in addition to the audio channels reproduced by loudspeakers in multi-channel audio systems used in large rooms, signal processing can be applied. This signal processing can contribute to the "acoustic improvement" of sound in the audience area. For example, one or more of the channels reproduced by the side or rear speakers may have reverb or "artificial echo" or echo and other signal processing applied. Reverb, echo and other signal processing effects are well known in the art. For example, US2011/0261966 to Dolby International AB describes a system for applying reverb to downmix channels that are then upmixed for playback on loudspeakers.

In large halls, it is possible that listeners further away from the stage may hear the sound from one of the side or rear speakers before the sound from the front speakers. This has the undesirable effect that listeners will hear sound coming from the back or side of the room while seeing the performance from the front. To avoid this, a fixed time delay or "pre-delay" is applied to the audio signal played back from the speakers at a distance from the stage. The time delay is chosen so that the sound from the front speakers arrives at the listeners at least 15 ms before the sound from the rear or side speakers to maintain the perceived sound direction coming from the front/stage. One of the terms used to describe this is to maintain a "precedence effect" in the sound.

The essence of the invention

In accordance with one aspect of the present invention, there is provided a signal processing system for applying temporal effects to an N-channel input audio signal for playback on a group of speakers having a predetermined configuration, comprising:

a first subsystem that receives an N-channel input audio signal and generates a first M-channel audio signal therefrom;

at least one second subsystem, each receiving a first M-channel audio signal, each second subsystem comprising:

an effect block for applying a time effect to each channel of the M-channel audio signal, said time effect having a minimum delay value;

a signal distribution unit that:

associates each channel of the first M-channel audio signal with a subgroup of speakers; and

generates a second M-channel audio signal from the first M-channel audio signal in accordance with the MxM matrix, each element a _ij in the MxM matrix includes a delay parameter, while the signal distributor determines the minimum value of the delay parameter in each element a _ij in accordance with the distance between at least two speakers in at least one of the subgroups i and j of the speakers and in accordance with the specified minimum delay value;

said effect block being configured to apply a time effect to each channel of the second M-channel audio signal;

a mixing unit that generates a K-channel audio signal from said one or every second M-channel audio signal.

Preferably, the signal distributor determines the minimum value of the delay parameter in each element a _ij to be at least the time the sound travels the maximum distance between speakers in speaker subgroups i and j.

Preferably, the signal processing system comprises several second subsystems.

Preferably, the effect block of each second subsystem is configured to apply multiple time effects having a first minimum delay value or a second minimum delay value.

Preferably, the signal distribution block of each second subsystem determines the minimum value of the delay parameter in each element a _ij in accordance with:

(a) the distance between adjacent loudspeakers in the j loudspeaker subgroup;

(b) the time it takes the sound to travel the maximum distance between loudspeakers in subgroups i and j of loudspeakers.

Preferably, the signal distribution block of each second subsystem is configured to determine the minimum value of the delay parameter in each element a _ij in accordance with the criteria (a), if the minimum delay value of the effect block of the second subsystem is less than a predetermined threshold value.

Preferably, the signal distributor of each second subsystem is configured to add a predetermined fixed delay value to a specified minimum value of the delay parameter in each element a _ij .

In accordance with another aspect of the present invention, there is provided a digital signal processing method for applying timing effects to an N-channel input audio signal for playback on a set of configured speakers, comprising the following steps, implemented by a processor:

generating a first M-channel audio signal from the N-channel input audio signal;

associating each channel of the first M-channel audio signal with a subgroup of speakers;

generating at least one second M-channel audio signal from the first M-channel audio signal in accordance with the MxM matrix, wherein each element a _ij in the MxM matrix includes a delay parameter, also determining the minimum value of the delay parameter in each element a _ij in accordance with the distance between at least two speakers in at least one of the i and j speaker subgroups and in accordance with the minimum delay value;

applying a time effect to each channel of the second M-channel audio signal, said time effect comprising a specified minimum delay value;

generating a K-channel audio signal from said one or every second M-channel audio signal.

Preferably, the minimum value of the delay parameter in each element a _ij is defined as at least the time during which the sound travels the maximum distance between the speakers in speaker subgroups i and j.

Preferably, the method also comprises generating a plurality of second M-channel audio signals from the first M-channel audio signal in accordance with a corresponding MxM matrix for each second M-channel audio signal.

Preferably, said time effect comprises a first minimum delay value or a second minimum delay value.

Preferably, the minimum value of the delay parameter in each element a _ij is determined in accordance with:

a) the distance between adjacent loudspeakers in the j subgroup of loudspeakers;

b) the maximum distance between loudspeakers in loudspeaker subgroups i and j.

Preferably, the minimum value of the delay parameter in each element a _ij is determined according to criteria (a) if the minimum delay value applied to that channel by the time effect is less than a predetermined threshold value.

Preferably, the method comprises adding a predetermined fixed delay value to the minimum value of the delay parameter in each element a _ij .

Brief description of the drawings

The invention will now be described by way of example with reference to the accompanying drawings, in which:

fig. 1 is an illustration of an exemplary performance venue in which embodiments of the invention may be used;

fig. 2 shows a system for signal processing in accordance with embodiments of the invention;

fig. 3A and FIG. 3B illustrate exemplary speaker and audio path configurations used in embodiments of the signal processing system of FIG. 2;

fig. 4A and FIG. 4B illustrate ranges of time delays applied by the signal processing system of FIG. 2;

fig. 5A and FIG. 5B illustrate the distances used to determine the minimum time delay value in element a _ij in exemplary configurations of the signal processing system of FIG. 2; and

fig. 6 shows a digital signal processing method in accordance with embodiments of the present invention.

Description of Preferred Embodiments

Fig. 1 is an illustration of an exemplary performance venue 10 in which embodiments of the invention may be used. The performance venue 10 has a stage 12 that houses a plurality of microphones 14. The term "microphone" is used here to refer to any device that captures sound and includes, for example, a guitar pickup.

The performance venue 10 includes an audience area 16 . From the point of view of a person in the audience area 16, the stage 12 is in front, with the terms "behind" and "sides" having their usual meanings from these words.

The group of loudspeakers, indicated generally at 18, is located on the periphery of the audience area 16 and consists of front loudspeakers 18a, right loudspeakers 18b, rear loudspeakers 18c and left loudspeakers 18d. The number, placement, and configuration of loudspeakers 18 may vary from location to location.

A signal processing system 20 is provided for applying temporal effects to an N-channel input audio signal for playback on a group of speakers 18, as will be described in more detail below. In some embodiments, the signals from the microphones 14 may form an N-channel audio input signal. In other embodiments, the signals from the microphones 14 may be pre-processed to derive from an N-channel audio input signal, for example, by combining groups of signals from the microphones 14. It will be understood that in some applications, the signal processing system 20 may be used with pre-recorded N-channel audio input signals. .

Turning now to FIG. 2, the signal processing system 20 comprises a direct audio processing unit 22, a first subsystem 24, at least one second subsystem 26, and a mixing unit 28.

The direct audio processing unit 22 receives an N-channel input audio signal and generates a K-channel direct audio signal 23 from it, for example, using an NxK matrix. Direct audio processing unit 22 may also apply other signal processing used in the art to direct or "raw" audio channels. In embodiments of the invention, the direct audio processing unit 22 may be configured to apply a fixed time delay to the channels in the K-channel direct audio signal to be played by the side speakers 18b, 18d, and the rear speakers 18c in order to preserve the "precedence effect" in the sound.

The first subsystem 24 receives an N-channel input audio signal and generates a first M-channel audio signal 30 from it. Each channel of the first M-channel audio signal 30 forms part of the sound field.

As shown in FIG. 2, there may be multiple direct audio processing units 22 and first subsystems 24 each receiving and processing n channels of the N channels in an N-channel input audio signal. Typically, every n channels represent a sound object such as lead vocals, guitar, etc., in this case, n is usually 1 channel or 2 channels, although more channels may be used.

In some embodiments, the first M-channel audio signal 30 may be a speaker-independent sound field encoding based on a set of virtual microphones derived from an n-order ambiphonic B-field including full spherical and planar B-fields. Each channel has a known location in the sound field, as determined by the directions of the virtual ambiophonic microphones.

In other embodiments, the spatial distribution of channels in the first M-channel audio signal 30 may be determined in accordance with the configuration of a particular group of speakers, as described in detail below.

Fig. 3A and FIG. 3B illustrates the distribution of M channels for two exemplary speaker configurations. In FIG. 3A, loudspeakers 18 are arranged in a rectangular configuration that completely surrounds the audience area. With this arrangement, the minimum azimuth (to the speakers) = -180° and the maximum azimuth (to the speakers) = 180°, where 0° corresponds to the direction forward, for example, facing the stage. The M channels are evenly distributed between the minimum and maximum azimuths and are shown in FIG. 3A by arrows 32. In FIG. 3A shows a circuit where M=8, however, other values of M may be used. FIG. 3B, the loudspeakers 18 are arranged in a line in which the minimum azimuth (towards the loudspeakers) = -45° and the maximum azimuth (towards the loudspeakers) = 45°. The M-channels are evenly distributed between the minimum and maximum azimuths and are shown in FIG. 3B arrows 32'.

In the case of a loudspeaker configuration with a height component, such as in a full sphere and hemisphere configuration, the orientation of each of the M channels is determined by the first subsystem 24 and is determined by the azimuth value and the height value. The M-channels are preferably evenly distributed between the azimuth and height values determined by the loudspeaker configuration. Preferably, the azimuth and height values defined for the M-channels define a regular grid of space defined by the speaker configuration. For any given speaker configuration -180° <= minimum azimuth (to loudspeakers) < maximum azimuth (to loudspeakers) <= 180° and -90° <= minimum height (to loudspeakers) < maximum height (to loudspeakers) <= 90° .

Once the channel allocation is determined, or if speaker-independent coding is used, the first subsystem 24 then allocates each channel of the n-channel input audio signal to one or more channels of the first M-channel audio signal 30, for example, using an nxM matrix. The elements of the matrix are determined in accordance with the spatial parameters of each channel of the n-channel input audio signal, such as azimuth, height, distance. Processing each n channels of the N channels individually allows each audio object represented by each n channels to be individually positioned in the M channels using spatial parameters such as azimuth, altitude and distance associated with the n channels.

Each second subsystem 26 receives the first M-channel audio signal 30 and generates from it a second M-channel audio signal 40 to which a time effect is applied as described below. Each second subsystem 26 contains a signal distribution block 36 and an effects block 38 . The second M-channel audio signals 40 generated by the second subsystem 26 are "processed" audio channels as opposed to the "raw" audio channels generated by the direct audio processor 22.

The signal distributor 36 associates each channel of the M channels in the first and second signals 30, 40 with a subgroup of speakers 18 for the particular speaker configuration used, namely the speakers on which that channel will be reproduced. In one example, this association can be determined by the presence of a non-zero value in the MxK array used by mixing unit 28, as described below. It is understood that the subgroups may overlap in some configurations, i.e. a given loudspeaker 18 may be used to reproduce more than one channel of the first M-channel audio signal 30.

The signal distributor 36 then generates a second M-channel audio signal 40 from the first M-channel audio signal 30 in accordance with the MxM matrix. Each element a _ij in the MxM matrix includes a delay parameter and may include a gain parameter such that each channel in the second M-channel audio signal 40 is a weighted sum of the delayed channels in the first M-channel audio signal 30. The gain parameters in the MxM matrix may be defined by the user. The signal distribution block 36 determines the minimum value of the delay parameter in each element a _ij according to the distance between at least two speakers in at least one of the speaker subgroups i and j and according to the minimum delay value applied by the effect block 38, as described below. The signal distributor 36 may also apply other signal processing used in the art, such as phase decorrelation of each MxM matrix input by filtering.

In some embodiments, the signal distributor 36 is configured to add a predetermined fixed delay value to the minimum value of the delay parameter in each element a _ij .

Effect block 38 applies a time effect to each channel of second M-channel audio signal 40. In some embodiments, effect block 38 applies a mono echo/reverb algorithm, examples of which are known in the art, to each channel of second M-channel audio signal 40. May be used any suitable time delay/reverb algorithm known to those skilled in the art.

The time effect applied by effect block 38 contains a minimum delay value 42 as shown in FIG. 4A, in which the input channel from the first M-channel audio signal 30 is designated as "direct", while the output signal from the effect unit 38 typically contains a plurality of time-delayed signals received from the input channel. As shown, the time-delayed signals output from the effect block 38 have a minimum delay value 42 corresponding to the minimum time offsets from the direct signal, after which the output signals from the effect block 38 are produced.

A mixing unit 28 which generates a K-channel audio signal 44 from every second M-channel audio signal 40, for example using an MxK matrix. If desired, decorrelation filters may be applied by the mixer 28 to each channel of the K-channel audio signal 44. The mixer 28 includes a summer 46 that combines the K-channel direct audio signal 23 with the K-channel audio signal 44 to generate a K-channel output signal for amplification and reproduction on a group of loudspeakers 18. Although not essential, it is preferred that M<K for efficient processing, especially in real life, in which case the MxK matrix allocates each of the M channels to more than one of the K channels using known panning methods.

In some embodiments, the mixer 28 may be configured to add a predetermined delay to one or more channels of the K-channel audio signal 44 to obtain a precedence effect.

In some embodiments, one second subsystem 26 may be used, but more often more than one second subsystem 26 is used. Where more than one second subsystem 26 is used, a second adder 48 is provided to combine the plurality of second M-channel audio signals 40 before being processed by the mixer 28.

As will be understood by those skilled in the art, the signal processing system 20 has more than one possible configuration, as the following examples illustrate.

Example 1

In this configuration, each signal distributor 36 determines the minimum value of the delay parameter in each element a _ij , which should be at least the time for the sound to travel the maximum distance between speakers in speaker subgroups i and j. Fig. 5A illustrates this configuration. Examples of i and j channels of the first M-channel audio signal 30 are shown, the respective speaker subgroups are shown as 18i and 18j. The signal distribution block 36 determines the maximum distance between any speaker in subgroup 18i and any speaker in subgroup 18j, shown by dashed line 50, and determines the minimum value of the delay parameter in each element a _ij , which must be equal to at least the sound travel time for this distance.

Example 2

In this configuration example, pairs of second subsystems 26 are provided. One effects unit 38 in each pair of second subsystems 26 is configured to apply time effects having a first minimum delay value 42a, while another effects unit 38 in each pair of second subsystems 26 is configured to apply time effects having a second minimum delay value 42b. In FIG. 4B shows an example of delay minimums 42a, 42b.

In the preferred embodiment of this configuration, the minimum delay value 42a corresponds to early reflections and the minimum delay value 42b corresponds to late reflections. This configuration allows the distribution unit 36 in each pair of second subsystems 26 to determine the minimum value of the delay parameter in each element a _ij , given the configuration of the loudspeaker and the minimum delay value of the effects unit. For example, the block 36 of signal distribution of each second subsystem 26 may be configured to determine the minimum value of the delay parameter in each element a _ij in accordance with the time it takes the sound to travel:

a. distances d between adjacent loudspeakers in subgroup j of loudspeakers; or

b. maximum distance between loudspeakers in subgroups i and j of loudspeakers.

In FIG. 5B shows this configuration, in which the distance d in speaker subgroup j is shown in addition to the maximum distance between speakers in speaker subgroups i and j, indicated by dashed line 50.

When the effects unit 38 is configured to apply time effects having a minimum delay value 42a, the signal distribution unit 36 in this second subsystem 26 is configured to determine the minimum value of the delay parameter in each element a _ij according to criterion (a). When the effects unit 38 is configured to apply time effects having a minimum delay value 42b, the signal distribution unit 36 in this second subsystem 26 is configured to determine the minimum value of the delay parameter in each element a _ij according to criterion (b). This has the advantage that early reflections can be delayed for a shorter time than late reflections, while still maintaining the foreground effect in the audience area, resulting in a more natural sound.

Other configurations are possible. For example, the signal distributor 36 of each second subsystem 26 may be configured to determine if the effector 38 of the second subsystem 26 has a minimum delay value that is less than a predetermined threshold. If so, then the block 36 distribution of signals calculates the minimum value of the delay parameter in each element a _ij in accordance with the criteria (a) above, otherwise, in accordance with the criterion (b).

In addition, not all second subsystems 26 need to be configured in the same way. Some second subsystems 26 may be configured as described in Example 1 above, while others may be configured as described in Example 2 above.

It will be understood that the specific values of the minimum delay values 42a, 42b will depend on the configuration of the loudspeakers. For example, for loudspeakers located 6 m apart, the minimum delay value 42a may be about 15-23 ms, while for loudspeakers located in a rectangular configuration of 25 x 40 m, the minimum delay value 42b is typically between 50 up to 100 ms. In FIG. 6A shows a signal processing method 100 for applying timing effects to an N-channel input audio signal for playback on a group of speakers having a predetermined configuration. Method 100 includes steps implemented by a processor, as described below.

First, step 102 includes generating a first M-channel audio signal from an N-channel input audio signal. Step 103 includes associating each channel of the first M-channel audio signal with a subgroup of speakers.

Next, step 104 includes generating at least one second M-channel audio signal from the first M-channel audio signal in accordance with the MxM matrix, each element a _ij in the MxM matrix includes a gain parameter and a delay parameter, and further the step includes determining a minimum delay parameter values in each element a _ij according to the distance between at least two speakers in at least one of the i and j speaker subgroups and according to the range of delay values.

In some embodiments, at step 104, a plurality of second M-channel audio signals are generated from the first M-channel audio signal according to a corresponding MxM matrix for each second M-channel audio signal.

In some embodiments, the minimum value of the delay parameter in each element a _ij is defined as at least the time during which the sound travels the maximum distance between the speakers in speaker subgroups i and j.

In some embodiments, the minimum value of the delay parameter in each element a _ij is determined in accordance with:

a. the distance d between adjacent loudspeakers in the subgroup j of loudspeakers; or

b. the maximum distance between loudspeakers in subgroups i and j of loudspeakers.

In some embodiments, a predetermined fixed delay value is added to the minimum value of the delay parameter in each element a _ij .

Step 106 includes applying a time effect to each channel of the second M-channel audio signal, with the time effect containing a minimum delay value.

In some embodiments, the time effect comprises either a first minimum delay value or a second minimum delay value.

In some embodiments, the minimum value of the delay parameter in each element a _ij is determined at step 104 in accordance with the criteria (a), if the minimum delay value applied to this channel due to the time effect is less than a predetermined threshold value.

Finally, step 108 includes generating a K-channel audio signal from one or every second M-channel audio signal.

While aspects of the present invention have been specifically shown and described with reference to the embodiments described above, those skilled in the art will appreciate that various additional embodiments may be considered by modification of the disclosed machines, systems, and methods without departing from the spirit and scope of the present invention. . It should be understood that such embodiments are within the scope of the present invention, which is defined by the claims and any equivalents thereof.

Claims (29)

1. A signal processing system for applying temporal effects to an N-channel input audio signal for playback on a group of pre-configured speakers, comprising: a first subsystem that receives an N-channel input audio signal and generates a first M-channel audio signal therefrom; at least one second subsystem, each receiving a first M-channel audio signal, each second subsystem comprising: an effect block for applying a time effect to each channel of the M-channel audio signal, the time effect having a minimum delay value; a signal distribution unit, configured to associate each channel of the first M-channel audio signal with a subgroup of speakers; and generate a second M-channel audio signal from the first M-channel audio signal in accordance with the MxM matrix, each element a _ij in the MxM matrix includes a delay parameter, while the signal distribution unit is configured to determine the minimum value of the delay parameter in each element a _ij according to the distance between at least two speakers in speaker subgroup i and/or speaker subgroup j and according to the minimum delay value; a mixing unit configured to generate a K-channel audio signal from said second M-channel audio signal or every second M-channel audio signal; said effect block being configured to apply a time effect to each channel of the second M-channel audio signal. 2. The signal processing system according to claim 1, in which the signal distribution unit is configured to determine the minimum value of the delay parameter in each element a _ij , which should be at least the sound propagation time over a distance equal to the maximum distance between speakers in subgroups i and j speakers. 3. The signal processing system according to claim 1, containing several second subsystems. 4. The signal processing system of claim 3, wherein the effects unit in each second subsystem is configured to apply multiple time effects having a first minimum delay value or a second minimum delay value. 5. The signal processing system according to claim 3 or 4, in which the signal distribution unit in each second subsystem is configured to determine the minimum value of the delay parameter in each element a _ij in accordance with: (a) the distance between adjacent loudspeakers in subgroup j of loudspeakers; or (b) the maximum distance between loudspeakers in subgroups i and j of loudspeakers. 6. The signal processing system according to claim 5, in which the signal distribution unit in each second subsystem is configured to determine the minimum value of the delay parameter in each element a _ij in accordance with criterion (a), if the minimum delay value of the effect unit in the second subsystem is less than the specified threshold value. 7. The signal processing system according to any one of paragraphs. 1-6, in which the signal distribution block in each second subsystem is configured to add a predetermined fixed delay value to a specified minimum value of the delay parameter in each element a _ij . 8. A signal processing method for applying temporal effects to an N-channel input audio signal for playback on a group of speakers having a given configuration, including the following steps implemented by the processor: generating a first M-channel audio signal from the N-channel input audio signal; associating each channel of the first M-channel audio signal with a subgroup of speakers; generating at least one second M-channel audio signal from the first M-channel audio signal according to an MxM matrix, each element a _ij in the MxM matrix including a delay parameter; determining a minimum delay parameter value in each element a _ij according to a distance between at least two speakers in speaker subgroup i and/or speaker subgroup j and in accordance with the minimum delay value; applying a time effect to each channel of the second M-channel audio signal, said time effect comprising a specified minimum delay value; generating a K-channel audio signal from said one second M-channel audio signal or every second M-channel audio signal. 9. The method of claim 8, wherein said minimum value of the delay parameter in each element a _ij is defined as at least the propagation time of sound over a distance equal to the maximum distance between loudspeakers in subgroups i and j of loudspeakers. 10. The method of claim. 8, which generates several second M-channel audio signals from the first M-channel audio signal in accordance with the corresponding MxM matrix for each second M-channel audio signal. 11. The method of claim 10, wherein said time effect comprises a first minimum delay value or a second minimum delay value. 12. The method according to claim 10 or 11, in which the specified minimum value of the delay parameter in each element a _ij is determined in accordance with: (a) the distance between adjacent loudspeakers in subgroup j of loudspeakers; or (b) the maximum distance between loudspeakers in subgroups i and j of loudspeakers. 13. The method of claim 12, wherein said minimum delay parameter value in each element a _ij is determined according to criterion (a) if the minimum delay value applied to that channel by said time effect is less than a given threshold value. 14. The method according to any one of paragraphs. 8-13, further adding a predetermined fixed delay value to a specified minimum delay parameter value at each element a _ij .

Images