EP2421182A1

EP2421182A1 - Method and device for automatically controlling audio digital mixers

Info

Publication number: EP2421182A1
Application number: EP10384003A
Authority: EP
Inventors: Frances Jarque Moyano; Giulio Cengarle; Antonio Mateos Sole; Pau Arumi Albo
Original assignee: MediaProduccion SL
Current assignee: MediaProduccion SL
Priority date: 2010-08-20
Filing date: 2010-08-20
Publication date: 2012-02-22

Abstract

Method and device for the automated control of audio mixers (3, 4). Audio mixer receives audio signals coming from microphones (10-20) spread in a physical space (9). In turn, this mixer is connected to the automated control device provided with the description of microphones, consisting of at least the position of each microphone in said space and the mixer channel to which each microphone is connected. The method includes the stages to define the position of a point of maximum interest PMI (100) in said physical space and to create control events necessary for the automation of mixer volumes associated with microphones, said events being generated through determination of respective volumes of the microphones according to the PMI position and microphone position. In live events, for instance, football matches, minimising the number of microphones necessary for PMI capture, both in open and closed premises, and maintaining the accoustic energy is possible, this being compatible with present production equipment.

Description

TECHNICAL FIELD OF THE INVENTION

The present invention relates to the field of audiovisual industry and more particularly to the production and transmission of live events where a number of micro-phones are used for, by means of mixing them In a mixer, picking up the sound produced in a specific area of the event. Particularly, the present invention sets out a method and a device to automate and optimise the mix of said micro-phones, to be applied to mixers receiving audio signals coming from several microphones situated in a physical space along with a description of same, consisting of at least the position of each microphone in said physical space and the mixer canal to which each microphone is connected.

Background of the invention

Presently the production of live audio events is based on the placing of several microphones with two different purposes: picking up the background sound of the attend-ant public, and picking up the sound produced in the "maximum interest point" (MIP) of the event The situation in which the number of microphones used to pick up the sound of MIP is high is common. Note that the situation of the MIP uses to be very variable throughout the duration of the event.
Sporting events are an example of this principle, wherein a number of microphones are placed directed towards the rows of seats in order to capture the sounds coming from the public, as well as other microphones are set around the ground to pick up sounds such as ball strikes, players' voices, etc. In this case the MIP usually corresponds to the position of the ball at every moment.
During the production of these events, the sound engineer usually makes a sound mix of all the microphones dedicated to picking up the MIP manually, starting from the selection and continuous manipulation of the levels of said microphones, emphasising the levels of the nearest microphones and attenuating the levels of the rest in order to avoid the introduction of unwnted sounds.
Usually, this mix is made manually by means of the control interface of an audio mixer which receives signals from the microphones and processes them, both by aplication of volumes and of other audio process effects: dynamic com-pression, equalisation, etc. to obtain a final mix for the transmission.
Patent document US 2008/0247567 A1 to Kjoierbakken, Jahr and Hafizovic, proposes a system to direct and focus the audio captured by microphones selecting an area that can be automatised by the coordinates of a camera or manually by an operator. This system is based on the audio capture from a microphone matrix (in its commercial version, 300 micro-phones) situated in the room ceiling and a subsequent process to obtain a very precise sound. This system, apart from demanding a very complex audio process, poses the problem of its application involving a large logistic infrastructure for the installation (300 microphones and the control system) and its application only brings suitable for sports in an enclosed room, where the surface to cover is not very extensive and the possibility exists to anchor the system in a room ceiling.
Patent document US 2005/0281410 A1 to Hewlett Packard proposes a method for audio processing based on: characterising a group of sounds recorded from different sources with a time-space reference; analysis of accoustic sources, and generating a new modified audio representing sound picked up by a virtual microphone desplacing through the accoustic stage, thus generating a virtual sound tour on the stage. The proposed method has the disadvantage of requiring an audio process and it is not targeted towards to the production of live events, but realistic interactive systems of accoustic immersion seeking a sound character-isation from the position of the listener.
Another common problem in this kind of situation is that the manual manipulation to focus the maximum interest point does not warrant energy conservation, that is, changes in accoustic level can appear not to correspond to the physical reality, showing unwanted volume changes caused by failure of precision due to the operator.
The present invention aims to find a solution to the above mentioned problems, for which a methodology and a device are proposed allowing the automation of these manual processes in live events, by considering the following criteria:

to minimize the number of microphones necessary for picking up the MIP;
adapted to opened and closed premises;
accoustic energy conservation; and
being compatible with the present production equipment.

Explanation of the invention

To that end, according to a first aspect of the invention, a method is proposed to automatically control a multichannel audio mixer that is characterised in that, according to the characterising part of claim 1, comprises the steps of:

defining the position of a point of maximum interest in
that physical space, and
creating the control events necessary for the automation of sound volumes coming from the microphones, said events being generated after determining the particular volume of microphones depending on the position of the MIP and the position of at least one of the microphones.

claims

According to a second aspect of the invention, a device is described for the automatised control of a multichannel audio mixer in order to implement the above mentioned method, that in essence is characterised, according to the characterising part of claim 7, provided with a data pocessing control module including:

the means to receive, from the input control port, in particular:
- . the description of microphones, consisting at least of
  the position of each microphone in said physical space,
- . and the position of a maximum interest point referred to
  said physical space;
the means for processing input controls and for generating volume control events of sound coming from the microphones present in the received description, said events being generated on the basis of determining the particular volume, depending on the position of said maximum interest point and the position of at least one of the microphones, so that a mix of audio signals coming from said microphones according to the calculated volumes would optimise the capture of sound existing in the maximum interest point.

In claims 8 to 13 several forms of preferred implemen-tation of the method according to the invention are described.
According to a third aspect of the invention, software is provided for the implementation of the method applied to a device as described by the invention.
It is important to note that, in short, the purpose of the invention is a new method and device that, given a description of a number of microphones placed in a physical space and connected to an audio mixer where, for each microphone, in configuration time,

the situation of the microphone,
the orientation,
the directional pattern,
the audio mixer channel to which it is connected, and
optionally the calibration of volume and equallsation
are defined;
and in execution time,
a location of the maximum interest point (MIP) in this physical space, changing through the time, would create the control events necessary for controlling the digital audio mixer in order to obtain a single-or multi-channel sub-mix of the sound produced in the maximum interest point.

The methodology is designed for the obtention of a mix of signals captured by microphones optimizing the capture of sound present in the maximum interest point (MIP), that, in the case where the physical space is a football ground, it could correspond to the position of the ball, of the referee, of a coach or of a certain player.
The method for the generation of control events is based on norms determining the volumes of microphones depending on the position of the MIP and on the position of at least one of the microphones.
The above mentioned control events are aimed at changing the gain applied to each microphone in the audio mixer, so that the attenuation would increase at a greater distance from the microphone to the maximum interest point.
The invention, in its first version, is based on a hardware with an user's interface (for instance, a tactile screen) for the introduction of the maximum Interest point by the sound engineer, and a logic performing calculations and transmiting parameters.
The invention includes the following processes:

The configuration of the interface with the digital audio mixer (by the user or read from a memory):
- . selection of the sports ground,
- . configuration of position and orientation of microphones,
- . configuration of interface microphones with the channels of digital audio mixer, and
- . configuration of other factors (as per influence factor polar response of microphones, etc.)
- - The operation by the sound engineer:
  - . Selection of accoustic interest point in user's interface;
  - . Calculation of attenuation levels and other possible parameters for each microphone, and
  - . Transmission of parameters for each microphone to digital audio mixer.

In a second version, the operation process can be automated through the interconnection of the tactile device with automatic tracking systems for players and ball so that MIP is calculated by other systems external to the invention.
In a third version, the tactile device including the programming logics could be incorporated Into the mixer.

Brief description of the drawings

A detailed description of preferred but not exclusive forms of implementation of the method and device according to the invention is set forth below, together with the accom- panying drawings, in which:

Fig. 1 is a general diagram of the system showing the tactile control device connected to the digital audio mixer for the data transmission between them.
Fig. 2 shows a version in which the tactile device is integrated Into the digital audio mixer.
Fig. 3 shows the configuration process of the system applied to the area of football using the tactile device by selecting the type of scenery and the position of microphones.
Fig. 4 shows in a diagram similar to fig. 5 a second example of configuration for the influence factor of micro-phones, in this case with a high selectivity algorithm, for the same point of interest.

Detailed description of the drawings

A detailed description of the invention, both at design level and operation to obtain a high quality sound production in the point of interest, is offered in the accompanying figures.
Figs. 1 and 2 show the global architecture of the invention in two possible versions:

an autonomous tactile device 1, for instance, a tactile
screen, directly connected to the digital audio mixer 3 by
means of some data protocol, as may be MIDI or IP, allowing the transmission of commands between said devices througout transmission means 2 (fig. 1), and
a tactile device 5, possibly a tactile screen, integrated
in the digital audio mixer 4 and in the logics of mixer
(fig. 2).

In a well known mode, the digital audio mixer 3, 4 produces the "addition" or mix of audio signals coming from microphones 10-20 and affected by a gain as described below; the result goes out by an output control port (well known and not shown).
In order to operate the device, according to the invention, a configuration process similar to one shown in fig. 3 can be applied. Two stages can be observed.
Selecting, in the tactile device 1 or 5, the specific scenario, in a processing block 6 from among the ones programmed in the interface. In the illustrated example it corresponds to a football ground 9, more specifically shown in fig. 4.
Consecutively, in a block 7 the different microphones (microphone typology, position and orientation) are added, as well as being linked to the corresponding audio channel in digital audio mixer 3. Microphones are related to numerical references 10 to 20 that, for clarity reasons, are shown only in certain figures.
In block 8 (fig. 3) the output of each microphone is assigned to an input channel of the digital audio mixer 3.
On the basis of the configured device, the operator or sound engineer can start the operation. This is depicted on fig. 4. It is a process in which the sound engineer, on grounds of display 22 (fig. 4) of the television signal, puts on the tactile device 1 (or 5) a point 100' corresponding to the maximum interest point 100 of field 9 where sound is to be picked up. In block 21 the device logic calculates optimum levels for each microphone (10 to 20) and transmits them to digital audio mixer 3, 4.
The level calculation is based on a generic geometric code founded on the distance between the point of interest 100' and each microphone 10 to 20. As a particular case, it is proposed to set the output level v₁ for each microphone on basis of the formula $v_{i} = \frac{1}{d_{i}^{\exp}}$

where d₁ is the distance between the centre of interest point 100' and the microphone i, and exp is an exponent determining the range of influence distance of the micro-phones, and therefore makes the algorithm more or less selective with the number of microphones.
In figs. 5 and 6 the effect of exp for a same point of interest 100' is shown by setting a number of circles around microphones 10 to 20; each radius represent the influence of a particular microphone. In fig. 5 one can see that, with a small exp value, the algorithm is not very selective and therefore a lot of microphones contribute to the mix, while in fig. 6 exp is of high value and its operation is much more selective.
Formula (1) assigns gain values inversely proportional to the distance of microphones; therefore it provides higher attenuation to microphones situated at at a longer distance. In the case of fig. 5, the microphones contributing signal to the total signal are units 10, 11, 18, 19 and 20, while in the case of fig. 6 just microphones 10 and 20 nearer to point 100' (for instance, corresponding to point 100 where the ball or the referee are) are intervening. In this later case, the algorithm is more selective and sound signal would be more "focused". In the first case, with a lower selec-tivity, there could be a greater "wealth" of sources, thus favouring the environmental sound.
Knowing the polar response diagram of microphones 10 to 20, one can calculate the angle of the real position of interest point 100 in field 9 and the direction in which each microphone is aimed, allowing to compensate the attenuation due to the angular response of microphones. A generic form for the calculation would be the following expression: $O (θ) = (1 - a) + a . cos θ$

where O(θ) represents the output level referred to 1 depending of angle θ, and a is a parameter between 0 and 1 defining the type of pattern of the microphone; a=0) is an omnidirectional behaviour a = (0.0,5) supercardioide, a = 0,5 cardioide, a=(0,5,1) hipercardioide and a=1 an eight-shape figure or bidirectional.
Therefore, formula (1) modified to include compensation of the polar response of microphones would be $v_{i} = \frac{1}{d_{i}^{\exp}} \frac{1}{(1 - a) + α \cdot \cos (θ)} .$
Other methodologies exist to compensate the polar response of the microphone in case of not being able (or not wanting) to operate through an analytical function similar to the proposed one. An alternative form would be by means of tables of measured values, stored in memory.
To ensure uniformity and coherence of the mix with the motion of point 100 of interest in each case, the gains of microphones 10 to 20 are divided by total energy (correspond ing to the root of the amount of gains at square), as per the following formula: ${\overline{v}}_{l} = \frac{v_{l}}{Σ v_{l} 2} .$
The final volume applied to each microphone is v₁ . This value is transmitted to the digital audio mixer by means of the accepted communication protocol.
As a particular case, the methodology could calculate, on the basis of a matrix, the contribution percentage for each microphone 10 to 20 at different output channels, so that the point of interest 100 is not single-channeled but multi-channeled.
Thanks to the method and device of this invention, the sound mix heard by the listener can be optimized, simply by shoeing the operator (or sound engineer), for instance, with a finger on a tactile screen, the position of the maximum interest point, and by adequately modifying, according to the characteristics desired in the re-transmission, the values of gains and directionality of microphones.
Experts in the matter will appreciate that the method and device of the present invention allow obtention of the desired purposes, specifically attaining the goals of:

- minimizing the number of necessary microphones for picking up the point of maximum interest,
- being apropriate for open and closed premises,
- conserving the accoustic energy, and
- being compatible with the present production equipment.

Having sufficiently described the nature of this invention, as well as the manner of implementation, we state that everything not altering, changing or modifying its basic principles will be subject to minor variations.
In this sense, tactile devices 1, 5 could be substituted with "mouse" type pointer or other similar technical devices, all of them remaining within the scope of the inventive concept, such as, for instance, a pointering device, an user's interface, a video camera tracking system, an automated system for detecting and following the maximum interest point 100, a computer memory and a time reference or an external system.

Claims

A method for the automated control of a multichannel audio mixer (3, 4) receiving audio signals coming from multiple and respective microphones (10-20) distributed in a physical space (9) along with the description of same, consisting of at least the position of each microphone in said physical space and the mixer channel (3, 4) to which each microphone (10-20) is connected, characterised In that the method includes the stages of:
- defining the position of a maximum interest point (100) in said physical space (9),

- creating the control events necessary for the automation of mixer (3, 4) volumes associated to microphones (10-20), said events being generated by setting the particular volume of microphones depending on the position of said maximum interest point (100) and the position of at least one of the microphones.
A method according to claim 1, characterised in that the description of at least one of the microphones further includes a characterisation of its directivity pattern and of its orientation in the physical space, and in that, in the aforesaid stage of creating control events, these events are generated by setting microphone gain depending on
- the position of said maximum interest point (100),

- at least one of the following parameters:
. the position of at least one microphone (10-20), and/or

. the orientation of at least one microphone (10-20).
A method according to claim 1, characterised in that at least one of the following parameters:
- the position of the maximum interest point (100),

- the position of at least one microphone (10-20), and

- the orientation of at least one microphone (10-20),
is continuously updated throughout the time reading the data of at least one device selected from among: a tactile screen (1, 5), a pointing device, a user's interface, a video camera tracking system, an automated system for detecting and following the maximum interest point (100), a computer memory and a time reference, and an external system.
A method according to claim 1, characterised in that the audio signal generated by microphones (10-20) is substi-tuted with audio signals coming from a computer memory.
A method according to claim 1 or 4, characterised in that at least one of the following parameters of the description of at least one of the microphones (10-20):
- the position in the physical space (9),

- the audio mixer (3, 4) channel to which it is connected,

- the directivity pattern, and

- the orientation,
is read in a computer memory.
A method according to claim 1, characterised in that the aforesaid stage of creating control events includes a mix of signals picked up by microphones (10-20) to optimise the capture of sound existing in the maximum interest point (100), said mix consisting of one or more output audio channels and being based on a volume matrix determining the contribution of each microphone to each output audio channel.
A device for the automated control of a digital audio mixer (3, 4), appropriate for implementing the method according to one of the claims 1 to 6, receiving audio signals coming from multiple microphones (10-20) spread in a physical space (9), characterised in that said device includes:
- means for introducing the following control parameters in the mixer (3, 4):
. the description of microphones (10-20) consisting of at least the position of each microphone in said physical space (9), and

. the position of a maximum interest point (100) referred to said physical space (9);

- means for processing input controls and for generating control events for volumes of microphones (10-20) present in the received description, said events being generated on the basis of determining the particular volume of microphones according to the position of said maximum interest point (100) and the positionof at least one of the microphones, so that a mix of audio signals from said microphones according to calculated volumes would optimise the capture of sound existing in the maximum interest point (100).
A device according to claim 7, characterized in that it includes means to continuously update from time to time at least one of the following parameters:
- the position of the maximum interest point (100),

- the position of at least one microphone (10-20),

- the orientation of at least one microphone (10-20)
by means of data reading of at least a device selected among a tactile screen (1, 5), a pointing device, a user's interface, a video camera tracking system, an automated detecting and following system for the maximum interest point (100), a computer memory and a time reference, and an external control system of volumes generated at the output control port.
A device according to claim 7, characterized in that the means for processing input controls and for generating control events include means to determine volume of micro-phones according the position of the maximum interest point (100) and of at least one of the following parameters:
- the position of at least one microphone, and/or

- the orientation of at least one microphone.
A device according to claim 7, characterised in that
it is adapted to process at least one of the following para-meters:
- descriptors of audio signal captured by at least one of the microphones (10-20), and/or

- descriptors of audio signal obtained through processing, by an external device, the audio signal picked up by, at least, one of microphones (10-20).
A device according to claim 7, characterized in that it includes the means for continuously updating, by means of data reading, at least one of the following devices:
- a pointering device (1, 5),

- a user's interface,

- a video camera follower system,

- an automated system for detecting and following the maximum interest point (100),

- a computer memory and a time reference, and

- an external control system of volumes generated at the output control port.
A device according to claim 7, characterized in that it includes a visualization unit (1, 5) to depict, at least, - one of the following input controls:
. the position of at least one of microphones (10-20),

. the position of maximum interest point (100),

. the orientation of at least one of microphones (10-20),

. the directivity pattern of at least one of microphones (10-20);

- the volume generated by at least one of microphones (10-20).
A device according to one of claims 7 to 11, characterized in that the control events generated for automation of mixer volumes (3, 4) associated to microphones (10-20) are destined to obtain a mix of signals captured by microphones optimising the capture of sound existing in maximum interest point (100), the mix consisting of one o more output audio channels and based on a volume matrix determining the contribution of each microphone (10-20) to each output audio channel.
A computer program for implementing the method according to any one of claims 1 to 6, applied to a device according to any one of claims 7 to 13.