GB2446015A

GB2446015A - Preventing the loss of data at the final stage of midi synthesis when it is desired to create a 3d effect

Info

Publication number: GB2446015A
Application number: GB0701475A
Authority: GB
Inventors: Richard Clemow
Original assignee: Sonaptic Ltd
Current assignee: Sonaptic Ltd
Priority date: 2007-01-25
Filing date: 2007-01-25
Publication date: 2008-07-30
Anticipated expiration: 2027-01-25
Also published as: GB0701475D0; GB2446015B

Abstract

A sound data processing circuit comprises an input, for receiving a plurality of channels of MIDI data, a plurality of pairs of 3D positioning filters, each pair of 3D positioning filters being connected to a respective one of the inputs such that both filters of the pair receive the MIDI data from the respective input a first mixer, for adding output signals from a first 3D positioning filter from each pair of 3D positioning filters to form a first stereo output signal and a second mixer for adding output signals from a second 3D positioning filter from each pair of 3D positioning filters to form a second stereo output signal. The invention aims to prevent the loss of data at the final stage of MIDI synthesis (the mixer) when it is desired to create a 3D effect to enhance the music.

Description

ENHANCING MIDI WITH 3D POSITIONING

BACKGROUND

MIDI is a well-known sound format. It is widely used where the amount of bandwidth available to transmit (or memory available to store) music or sound effects is limited, such that it would be impractical to use formats such as PCM or MP3. One important application is mobile devices, where these constraints are common.

The MIDI format specifies the volume, pitch and timing of musical notes, as well as the "instrument" or "voice" that plays them. MIDI allows for multiple simultaneous voices, and for grouping into channels, allowing complex music to be represented. Sound effects can be generated in a similar way. The term "music" is used here to mean any *:*::* such sounds. *.S.

To recreate music from MIDI data, a MIDI "synthesizer" is required, implemented in hardware or software. The MIDI synthesizer interprets the MIDI data stream and generates the required sound with the specified timing. The MIDI data includes a stereo pan control for each channel. A stereo mix of all the channels is performed to S...

: 20 generate the final stereo output.

S * S S * S.

This conventional architecture is shown in Fig 1, as an example with 3 channels.

It is not new to consider adding 3D positioning to MIDI, and this has been implemented in various formats and standards, for example Faith's MFi 4.0 standard, used in mobile phones in Japan. These formats include additional data in the MlDl data stream to specify the 3D position of each channel. The MIDI synthesizer includes additional features to provide the 3D positioning implementation. This is shown in Fig 2.

When the MIDI data does not include this 3D positioning information, it is well known that it is possible to apply a "stereo surround" or "stereo 3D" effect to the stereo output of the MIDI synthesizer, and this has been done in products on the market. This is shown in Fig 3.

THE INVENTION

This invention relates to the situation where 3D positioning data is not included in the MIDI data. This is the situation with the majority of MIDI content. It is desired to create a 3D effect to enhance such music, but the known method of applying a "stereo surround" algorithm to the stereo output of the MIDI synthesizer (Fig 3) does not produce very good results.

According to the present invention, there is provided a sound data processing circuit, comprising: an input, for receiving a plurality of channels of MIDI data; a plurality of pairs of 3D positioning filters, each pair of 3D positioning filters being connected to a respective one of the inputs such that both filters of the pair receive the MIDI data from the respective input; a first mixer, for adding output signals from a first 3D positioning filter from each pair of 3D positioning filters to form a first stereo output signal; and a second mixer, for adding output signals from a second 3D positioning filter from each pair of 3D positioning filters to form a second stereo output signal. "S. * 15 S...

BRIEF DESCRIPTION OF THE DRAWINGS

* S*S.. * S

Figure 1 shows a conventional MIDI synthesiser; *...

Figure 2 shows 3D position data in a MIDI stream; Figure 3 shows conventional MIDI plus stereo surround; Figure 4 shows conventional MIDI plus 3D positioning; Figure 5 shows basic stereo surround; Figure 6 shows basic stereo surround with panned input; Figure 7 is a mathematically equivalent diagram to Figure 6; and Figure 8 shows basic 3D positioning with mono inputs.

The inventors have realised the reason for the relatively poor performance of the system of Figure 3, which is that valuable information is lost at the final stage of the MIDI synthesizer (i.e. the stage labelled Mixer'), where the stereo panned channels are mixed together into the final stereo mix. This mixing process irreversibly loses valuable information because some contribution from each channel is mixed into the left output and some into the right output of the stage labelled Mixer. If the left and right channels output from the Mixer stage are each positioned in 3D space, there is a "common mode" component which is common to both of the signals being positioned. This interferes with the positioning process. Figs 5-7 illustrate this.

Fig 5 shows how a basic stereo surround function is structured. More sophisticated algorithms can improve matters, but this diagram illustrates the basic principle which causes the problem in all such algorithms. Each input is treated as a mono source and a 3D positioning algorithm using two filters is applied to it. The filters are chosen so that the left input is positioned somewhere on the left and the right input is positioned somewhere on the right (maybe at +1-90 degrees, but other angles can be used). S..

*,** 15 Fig 6 shows what happens when you feed such a stereo surround block with a panned mono signal, i.e. the type of signal that a MIDI synthesizer produces (for simplicity, * .* . .* * . assume that the MlDl synthesizer is only producing one channel of output, but the same arguments apply if more than one channel is active). *..*

This structure can be exactly mathematically simplified to the equivalent shown in Fig * 7, which shows the input fed into a pair of filters, and the actual filter shapes being affected by the pan control setting (in a complex way). Fig 7 has the same structure as a single channel of 3D positioning, so it appears as if this should work. However, the key difference between this structure and true 3D positioning of a mono source is that the filters in Fig 7 are not valid 3D positioning filters. They are a combination of the filters in Fig 5, but this combination does not produce a good set of positioning filters.

This is reflected in the quality of the perceived result. The root cause of the problem is the fact that both inputs are derived from the same mono source. This allows the mathematical combination of the filters performed to arrive at Fig 7. If this common mode component did not exist, there would be no problem. It tends to pull all the sounds towards the centre, making it impossible to produce a wide 3D effect. Once you have mixed more than one panned stereo signal together, it is mathematically impossible to separate them again. There is therefore more information available to us prior to the mixer compared to after it.

The invention divides the MIDI synthesizer into two parts -the synthesis part and the mixing part, and inserts the 3D positioning technology between the two. Using this method, each channel is separately positioned in 3D space. So in the prior art, you apply pan controls to a channel to set the left'right balance, combine all left outputs for all channels and feed to your stereo surround algorithm and at the same time combine all right outputs for all channels and feed that to the stereo surround algorithm. But with your system, you don't do the combination step prior to the algorithm step. Instead, you do the algorithm step on each separate, panned channel. The MIDI synthesizer does not actually need to do the panning, as we actually want the unpanned mono signal. We can process the panned signal, but we would simply add the left and right channels to convert it back to mono, before feeding to the algorithm. This is shown in Fig 4, and can be implemented in hardware, software, or a combination.

Fig 8 shows 3 channels of positioning. Note that each input is now only fed into one *:* 15 filter-pair, unlike Fig 6, where each input is fed into two filter-pairs, causing the common-mode problem. It is by this means possible to achieve results which are as good as having 3D position data embedded in the MIDI stream, although the content provider does not have direct control of the 3D positioning. S...

: 20 Because there is no 3D positioning data in the MIDI data, it must be created artificially.

* . There are several options for doing this: 1) The 3D positioning data can be created from the stereo Pan\data as shown in Fig 4. This can be done in a number of ways: a. Fully panned left can be interpreted as a position to the far left (an "azimuth" of -90 degrees). Fully panned right can be interpreted as a position to the far right (an "azimuth" of +90 degrees). Pans in between these two extremes would be positioned in 3D space proportionately between these two extremes.

b. The azimuth range could be reduced to create a less extreme effect, for example +/-45 degrees.

c. The azimuth range could be increased to create a more extreme effect, for example +/-135 degrees, with some channels being behind the listener.

d. The azimuth range could be increased to the maximum of +/-180 degrees to encompass a complete circle.

e. The user could be given control over the width, i.e. the azimuth range which corresponds to the full stereo pan range.

2) Individual voices or channels could be allocated preset 3D positions, or the user could be allowed to choose such positions.

3) The 3D positioning could be random or based on some other algorithm which spread the 3D positions of the various channels in some pleasing way.

4) The 3D positions could be made to move in some way to create swirling or other effects.

5) Any of the above effects could be combined with control over the vertical axis, or elevation, and/or of the distance, i.e. volume.

6) The 3D position of a channel could depend on its volume or other MIDI data, such as note duration. This would create interesting moving effects. This is shown by the dashed line in Fig 4. S... * S S... SSSS* * S *... *. * . S * S S *

Claims

1. A sound data processing circuit, comprising: an input, for receiving a plurality of channels of MIDI data; a plurality of pairs of 3D positioning filters, each pair of 3D positioning filters being connected to a respective one of the inputs such that both filters of the pair receive the MIDI data from the respective input; a first mixer, for adding output signals from a first 3D positioning filter from each pair of 3D positioning filters to form a first stereo output signal; and a second mixer, for adding output signals from a second 3D positioning filter from each pair of 3D positioning filters to form a second stereo output signal.

2. A circuit as claimed in claim 1, wherein the input is suitable for receiving mono MIDI data.

3. A circuit as claimed in claim 1, wherein the input is suitable for receiving a plurality of channels of panned MIDI data, each comprising respective first and second signals, and wherein the circuit further comprises a plurality of adders, each adder being connected to a respective input to add the first and second signals to form a summed signal, and to apply the summed signal to a respective pair of 3D positioning filters.

4. A method of processing sound data, the method comprising: receiving a plurality of channels of MlDl data; applying each channel of MIDI data to a respective pair of 3D positioning filters, such that both filters of the pair receive the MIDI data of the respective channel, to form * first and second filtered signals; * forming a first stereo output signal as a sum of the first filtered signals from each ***I** * of the 3D positioning filters; and forming a second stereo output signal as a sum of the second filtered signals * from each of the 3D positioning filters. * *

5. A method of generating a 3D sound signal from a plurality of channels of mono MIDI data, the method comprising: performing a respective 3D positioning algorithm on each of the channels of data to form respective left and right stereo signals; and

S

adding the left stereo signals obtained from each channel to form an output left stereo signal, and adding the right stereo signals obtained from each channel to form an output right stereo signal. S. * S *

* *SS. * S *S * S..CLME: * SS.* * * * S. * S * S.. *

S SS*S * *