JP4861593B2  Multichannel surround sound mastering and playback method for preserving 3D spatial harmonics  Google Patents
Multichannel surround sound mastering and playback method for preserving 3D spatial harmonics Download PDFInfo
 Publication number
 JP4861593B2 JP4861593B2 JP2001578151A JP2001578151A JP4861593B2 JP 4861593 B2 JP4861593 B2 JP 4861593B2 JP 2001578151 A JP2001578151 A JP 2001578151A JP 2001578151 A JP2001578151 A JP 2001578151A JP 4861593 B2 JP4861593 B2 JP 4861593B2
 Authority
 JP
 Japan
 Prior art keywords
 sound
 speaker
 sound field
 harmonics
 position
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Active
Links
 230000000875 corresponding Effects 0 claims description 16
 238000000034 methods Methods 0 description 15
 239000000203 mixtures Substances 0 description 7
 230000000694 effects Effects 0 description 5
Images
Classifications

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04S—STEREOPHONIC SYSTEMS
 H04S3/00—Systems employing more than two channels, e.g. quadraphonic
 H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04S—STEREOPHONIC SYSTEMS
 H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
 H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04S—STEREOPHONIC SYSTEMS
 H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
 H04S2420/11—Application of ambisonics in stereophonic audio systems
Description
[0001]
(Crossreference to related applications)
This application is a continuationinpart of application Serial No. 08 / 936,636, filed September 24, 1997, which is incorporated herein by reference.
[0002]
BACKGROUND OF THE INVENTION
The present invention relates generally to electronic sound transmission, recording and playback methods, and more particularly to improvements in surround sound technology.
[0003]
[Prior art]
Improvements in sound reproduction quality and presence have been steadily made over the past several decades. Stereo (2 channel) recording and playback via spatially separated loudspeakers greatly improves the realistic feel of the playback sound compared to previous mono (1 channel) sound playback. More recently, audio signals have been encoded in two channels to drive four or more loudspeakers arranged to surround the listener. With this surround sound, the realism of the playback sound has increased further. Multichannel (3 or more) recordings are used in most movie soundtracks, which can be dramatic in theaters appropriately equipped with sound systems that include loudspeakers placed around their walls to surround the audience. An audio effect is brought about. A standard for multichannel recording on a small optical CDS (compact disc), which is expected to be very popular for home use, is now being developed. Recent DVD (Digital Video Disc) standards have defined multiple channels of PCM (Pulse Code Modulation) audio in CDs that may or may not contain video.
[0004]
Theoretically, the most accurate reproduction of the audio wavefront would be obtained by recording and reproducing an acoustic hologram. However, it may be necessary to record tens of thousands or even millions of separate channels. In order to accurately reproduce the original acoustic wavefront, a twodimensional array of speakers should be placed around the home or theater at intervals not exceeding onehalf of the highest frequency wavelength to be reproduced, i.e. less than one centimeter apart. It will be necessary to place it. A separate channel needs to be recorded for each of this very large number of speakers, and would involve the use of as many microphones during the recording process. Thus, such accurate reproduction of acoustic wavefronts is not practical at all for audio reproduction systems used in homes, theaters and the like.
[0005]
If the desired playback is threedimensional and the speakers are no longer in the same plane, these complications will correspondingly increase and this type of playback will become even more impractical. The extension to three dimensions allows special effects in movie or music recording mastering, as well as when not limited to the original sound source plane. For example, even in the case of a musician recording on a flat stage, the resulting ambient sound environment will have a threedimensional character due to reverberations and variations in instrument placement that can be captured and played. Quantification is more difficult than sound source localization, but the inclusion of three dimensions increases this “spread” and depth perception of the sound field, even when the actual sound source is in the same plane. .
[0006]
[Problems to be solved by the invention]
Accordingly, it is possible to provide an improved and realistic sound reproduction technology by multichannel recording as defined in the emerging audio standard with approximately the same number of loudspeakers currently used in surround sound systems. It is the main and general purpose of the present invention.
[0007]
It is another object of the present invention to provide a method and / or system for playing recorded or transmitted multichannel sound at home, theatre, or other listening locations, which allows the user to use it. The electronic matrix can be set at the listening position for a specific arrangement of the loudspeakers being used.
[0008]
It is a further object of the present invention to extend these techniques and methods to 3D sound field capture and playback when the loudspeakers are placed in a noncoplanar arrangement.
[0009]
[Means for Solving the Problems]
These and additional objectives are realized by the present invention, where the acoustic field is ensured by multiple signals via four or more loudspeakers arranged in a simple and overall manner to surround the listening area. And the signal is processed to reproduce a specified number of spatial harmonics of the acquired acoustic field with virtually any arrangement of speakers around the listening area. This increases the sensation of sound reproduction without imposing any specific restrictions on the position of the loudspeaker.
[0010]
Rather than requiring the speakers to be placed in a particular pattern before the system can play a specified number of spatial harmonics, any speaker location present will have this beneficial result in the playback layout. Is used as a parameter in the electronic encoding and / or decoding of a multichannel sound signal. If one or more speakers are moved, these parameters are changed to preserve the spatial harmonics of the reproduced sound. The use of five channels and five speakers is described below to illustrate various aspects of the present invention.
[0011]
According to one particular aspect of the present invention, the individual monaural sounds are mixed together using a matrix that arranges the mono sounds at an angle when forming a recording or audio transmission. When these are played through the speaker arrangement assumed around the listener, the realism is improved. Rather than simply sending a given mono audio to the two channels driving the speakers on each side of the sound location, the sound is reproduced at the desired spatial harmonics, as is currently done by standard panning techniques Therefore, all channels are potentially included. A typical use is in mastering recordings of multiple musicians playing together. First, the sound of each instrument is recorded separately and then mixed to be placed around the listening area during playback. By using all channels to maintain spatial harmonics, the reproduced sound field is closer to the sound field present in the room where the musician is playing.
[0012]
According to another particular aspect of the present invention, multichannel sound can be rematrixed at home, theatre, or playback location to accommodate a different speaker arrangement than originally envisioned when mastered. The desired spatial harmonics are accurately reproduced using different actual arrangements of speakers. This allows for free speaker placement, which is particularly important in homes where constraints are often imposed on speaker placement, without losing the sensation of improved sound.
[0013]
According to a further particular aspect of the invention, the sound field is first obtained with direction information using a plurality of directional microphones. The microphone output, or the spatial harmonic signal as a result of the initial partial matrixing of the microphone output, is recorded or transmitted to the listening position by a separate channel. The transmitted signal is then used to reproduce the recorded sound field, along with a certain number of spatial harmonics matched to the spatial harmonics of the recording location, to determine the actual speaker position at home or other listening location. Rematrix is taken into account.
[0014]
These various aspects may use twodimensional or threedimensional spatial harmonics. In the twodimensional case, whether the initial recording is based on twodimensional spatial harmonics or by threedimensional projection onto the speaker surface, the audio wavefront is played back mainly by a loudspeaker arrangement in the same plane. The In 3D playback, one or more speakers are placed at a different height than the 2D plane. Similarly, a threedimensional sound field is acquired by noncoplanar arrangement of multidirectional microphones.
[0015]
Additional objects, features and advantages of various aspects of the present invention will become apparent from the preferred embodiments of the present invention, which should be construed in conjunction with the accompanying drawings. .
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Let us discuss the method of spatial harmonics in a twodimensional plane. Some of the results of this methodology are as follows: That is, (1) a surround sound recording method that can be used to supply any number of speakers, (2) a monaural sound panning method for accurately creating a predetermined spatial harmonic set, and (3) three surround sounds. A method of storing or transmitting on a channel, so that two of the channels are standard stereo mixes and a third channel can be used to reproduce a surround feed that preserves the original spatial harmonics.
[0017]
Following the twodimensional discussion, this same theory is extended to three dimensions. In two dimensions, spatial harmonics are based on a single variable, the Fourier sine and cosine series of the angle φ. Unfortunately, the 3D version of mathematics is not as clean and compact as 2D. There is no particularly good way to reduce complexity, so a 2D version is presented first.
[0018]
In order to extend the spatial harmonic method to three dimensions, the Legendre function and the spherical harmonic function will now be discussed briefly. In a sense, this is a generalization of the Fourier sine and cosine series. The Fourier series is a function of one angle, φ. The series is periodic. This can be thought of as a function representation on a circle. The spherical harmonic function is defined on the spherical surface and is a function of two angles, θ and φ. φ is the azimuth angle defined when 0 degrees is straight forward, 90 ° is left, and 180 ° is straight backward. θ is a declination angle (upper and lower), which is straight overhead at 0 °, a horizontal plane at 90 °, and straight down at 180 °. These are shown in FIG. 9 for the point (θ, φ). Note that the range of θ is 0180 °, while the range of φ is 0360 ° (or 180 ° 180 °).
[0019]
Spatial harmonics in two dimensions
In FIG. 1, the person 11 is shown in the middle of the listening area, which is surrounded by speakers SP1, SP2, SP3, SP4 and SP5, which send out their sound at the center. It is aimed so. An angular coordinate system has been set for explanation in this application. The forward direction of the listener 11 facing the front speaker SP1 is (θ_{1} , Φ_{1} ) = (90 °, 0 °). The angular positions of the remaining speakers SP2 (front left), SP3 (rear left), SP4 (rear right) and SP5 (front right) are respectively (θ_{2}, Φ_{2}), (Θ_{Three}, Φ_{Three}), (Θ_{Four}, Φ_{Four}), And (θ_{Five}, Φ_{Five}). Here, the speakers are placed in a typical arrangement defining a surface that is substantially planar, and in one example is a horizontal plane of θ = 90 ° parallel to the floor of the room in which the speakers are placed. In this situation, θ_{1}−θ_{Five}Are each 90 ° and these θ are not explicitly represented for the time being and are omitted from FIG. One or more speakers need not be higher than one or more other speakers, but may be made to fit a limited space. θ_{i} One or more cases of ≠ 90 ° are discussed below.
[0020]
A monaural sound 13, like a single instrument, with an angle φ from the zero reference_{0}It is desirable to arrange the speaker at a position where there is no speaker. There are other monaural sounds that are typically desired to be placed simultaneously at other angles, but for simplicity of explanation only source 13 is shown here. For multiinstrument music sources, for example, the sound of individual instruments typically varies at different angles φ around the listening area during the mastering process._{0}Placed in. The sound of each instrument is acquired by one or more microphones that record mono on at least one separate channel. These monaural recordings act as sound sources during the mastering process. Alternatively, mastering can be performed in real time from a separate instrument microphone.
[0021]
Before describing the mastering process, reference is made to FIGS. 2AD to illustrate the concept of spatial frequency. FIG. 2 (A) shows the space surrounding the listening area from the viewpoint of the angular position. For the five positions of the speakers SP1, SP2, SP3, SP4 and SP5, the desired position of the sound source 13 is shown as it is. The sound source 13 can be viewed as a spatial impulse that can be expressed as a Fourier expansion as follows:
Where m is an integer of the individual spatial harmonics, 0 to M harmonics are reconstructed and are the coefficients of one component of the individual harmonics, a_{m}Is the coefficient of one component of each harmonic, b_{m}Is the coefficient of the orthogonal component of each harmonic. Therefore, the value a_{0}Is the zeroorder value of the spatial function.
[0022]
The spatial zero order is shown in FIG. 2B, and has the same size around the entire space that rises and falls according to the size of the spatial impulse sound source 13. FIG. 2C shows a first order spatial function, which has one complete cycle around the space, whereas it is the maximum at the angle of the impulse 13. As illustrated in FIG. 2D, the quadratic spatial function has two complete cycles around the space. Mathematically, the spatial impulse 13 is accurately represented by a number of orders, but due to the fact that only a few speakers are used, the number of spatial harmonics that can be included in the reproduced sound field. There are limits. If the number of speakers is (1 + 2n) or more (n is the number of harmonics to be reproduced here), zero to n spatial harmonics of the reproduced sound field exist in the original sound field. You can play virtually exactly what you want. Conversely, the spatial harmonics that can be accurately reproduced are zero to n harmonics, where n is the largest integer that is less than or equal to half of the integer less than the number of speakers placed around the listening area. It is. Instead, a number smaller than this maximum number of possible spatial harmonics can be selected to be reproduced in a particular system.
[0023]
One particular aspect of the present invention is illustrated in FIG. 3, which schematically illustrates some of the functions of a sound console used to master multichannel recording. In this example, the five signals S1, S2, S3, S4 and S5 are recorded, possibly in digital form, on five separate channels of a suitable recording medium such as tape. Each of these signals will drive a separate speaker. It is illustrated that two monaural sound sources 17 and 19 are mixed and become recorded signals S1S5. These sound sources 17 and 19 can be, for example, live or recorded signals of various instruments that are mixed together. One or both of the sound sources 17 and 19 may be synthetically generated or naturally recorded sound effects, voices and the like. In practice, usually more than two such signals are used for recording. Individual signals can be added one by one to the recording track or mixed together for simultaneous recording.
[0024]
Illustrated in FIG. 3 is a monaural “positioning” technique. That is, the apparent position of each of the sound sources 17 and 19 when the recording is played back through the surround sound system is set during the mastering process as described above with respect to FIG. Currently, the mastering console's normal panning technique sends a monaural sound to only two of the recorded signals S1S5, to the speakers on either side of the location where it is desired for that sound, and to the sound source listener. Supplied with a relative amplitude that determines the apparent position relative to. However, this lacks some kind of realism. Thus, as shown in FIG. 3, each sound source has a number of spatial harmonics, at least zero and first harmonics, of the sound field emanating from that location to each of the five channels. Supplied with the relative gain set to build the set. One or more channels may still not receive any portion of a particular signal, but this is the result of a predetermined number of spatial harmonics being preserved, and the signal is artificially transmitted to only two channels. Not because it is limited.
[0025]
The relative contribution of the source 17 signal to the five separate channels S1S5 is indicated by the individual variable gain amplifiers 21, 22, 23, 24 and 25. The gain g of each of these amplifiers_{1}, G_{2}, G_{Three}, G_{Four}And g_{Five}Is set in the circuit 27 by a control signal from the control processor 29. Similarly, the sound signal of source 19 is routed through each amplifier 31, 32, 33, 34 and 35 to each of channels S1S5. Each gain g of amplifiers 3135_{1}', G_{2}', G_{Three}', G_{Four}'And g_{Five}'Is also set by the control processor 29 via the circuit 37. These sets of gains are calculated by the control processor 29 from input from the sound engineer via the control board 45. These inputs are the desired placement angle Φ (FIG. 1) of the sound from sources 17 and 19 and the assumed speaker set placement angle φ._{1}−φ_{Five}Is included. The calculated parameters can also optionally be fed through circuit 47 and recorded. The individual outputs of each amplifier 2125 are combined with the outputs of amplifiers 3135 by respective summing nodes 39, 40, 41, 42 and 43 to provide five channel signals S1S5. These signals S1S5 are eventually reproduced via each one of the speakers SP1SP5.
[0026]
The control processor 29 includes a DSP (digital signal processor) that operates to solve the simultaneous equations from the input information in order to calculate a set of relative gains for each of the monaural sound sources. The principle set of linear equations solved for the placement of individual separately placed sound sources can be expressed as follows:
Where φ_{0}Represents the angle of the desired apparent position of the sound, φ_{i}And φ_{j}Represents the angular position corresponding to the placement of the loudspeakers for the individual channels, each of i and j has an integer value from 1 to the number of channels, m represents the spatial harmonic, and this spatial harmonic Reaches from 0 to the number of harmonics matched to the harmonics of the original sound field during playback, N is the total number of channels, g_{i}Represents the relative gain of the individual channels, i ranging from 1 to the number of channels. It is for this set of relative gains that the equations are solved. The use of subscripts i and j follows the usual mathematical notation for matrices, where i is the number of rows in the matrix term and j is the number of columns.
[0027]
In a specific example where the number of channels N and the number of speakers is also 5 and the zero and first order spatial harmonics are accurately reproduced, the above linear equations can be expressed as the following matrix.
This general matrix is expressed as relative gain g_{1}G_{Five}Solve for the desired set of.
[0028]
This is a rank 3 matrix, meaning that there are many relative gain values that satisfy this. Another constraint is added to provide a unique gain set. One such constraint is that the second spatial harmonic is zero, which changes the lower two rows of the above matrix.
[0029]
Another constraint that can be imposed on the general matrix solution is that the velocity vector (frequency below the transition frequency in the range of about 7501500 Hz) and the power vector (at frequencies above this transition) must be substantially aligned. It must be. As is well known, the human ear identifies the direction of sound by different mechanisms in the frequency range above and below the transition. Thus, the apparent position of the sound that potentially extends to both frequency ranges is made to appear to the ears to come from the same location. This is obtained by considering the equations for each angular direction of these vectors equally as follows:
The definition of the velocity vector direction is to the left of the equal sign, and the definition of the power vector is to the right. For the power vector, taking the square of the gain term is an approximation of a model of how the human ear responds to a high frequency range, and therefore can vary somewhat between individuals.
[0030]
Once a set of relative gains for each of the sounds to be placed around the listener 11 is calculated by the control processor 29, the resulting signals S1S5 are reproduced from the recording 15 and are output from the speakers SP1SP5. One can be driven individually. If the speaker is in the angular position φ around the listener 11 that is assumed when calculating each sound source_{1}−φ_{Five}If it is accurately or very close to their location, the location of all sound sources will appear to the listener where the sound engineer has attempted to place them. The zero order, first order, and any higher order spatial harmonics included in these calculations will also be faithfully reproduced.
[0031]
However, physical constraints in the home, theatre, or other places where recordings should be played often limit where the sound system speakers can be placed. If placed around the listening area at an angle different from that assumed during recording, the spatialization of the individual sound sources may not be optimal. Thus, according to another aspect of the present invention, signals S1S5 are rematrixed by the listener's sound system, as illustrated in FIG. The sound channels S1S5 reproduced from the recording 15 are, in a specific embodiment, firstly represented by a spatial matrix signal a by means of the harmonic matrix 51._{0}(Zero harmonic), a_{1}And b_{1}Converted to (first harmonic). 1st harmonic signal a_{1}And b_{1}Are orthogonal to each other.
[0032]
If higher orders than the 0th and 1st spatial harmonics are to be preserved, two additional quadrature signals are generated by the matrix 51 for each additional harmonic. These harmonic signals then serve as inputs to the speaker matrix 53, which converts these signals into a modified set of signals S1 ′, S2 ′, S3 ′, S4 ′ and S5 ′, which This signal drives the uniquely positioned speakers to provide improved realism of the playback sound intended when the recording 15 was initially mastered assuming various speaker positions. This is achieved by the relative gain set in the matrices 51 and 53 via respective gain control circuits 55 and 57 from the control processor 59. The processor 59 determines the mastering parameters recorded and reproduced by the sound track, the first assumed speaker angle φ._{1}, Φ_{2}, Φ_{Three}, Φ_{Four}, And φ_{Five}, As well as the corresponding actual speaker angle β supplied to the control processor by the listener via the control panel 61_{1}, Β_{2}, Β_{Three}, Β_{Four}And β_{Five}Calculate these gains from
[0033]
The algorithm of the harmonic matrix 51 is illustrated using 15 variable gain amplifiers arranged in 5 groups of 3 each. Three of the amplifiers are connected to receive each of the sound signals S1S5 reproduced from the recording. Amplifiers 63, 64 and 65 receive the S1 signal, amplifiers 67, 68 and 69 receive the S2 signal, and so on. The output from one amplifier in each of these five groups is a_{0}Connected to a summing node 81 having an output signal, the output from each separate amplifier in each of these five groups is a_{1}Connected to a summing node 83 having an output signal, the output from each group of third amplifiers is connected to a third summing node 85, the output of which is the b1 signal.
[0034]
The matrix 51 represents the audio signals S1S5 reproduced from the recording 15 and the speaker angle φ assumed during mastering._{1}, Φ_{2}, Φ_{Three}, Φ_{Four}, And φ_{Five}Only from the intermediate signal a_{0}, A_{1}And b_{1}Calculate
Thus, in the description of this algorithm shown as matrix 51, amplifiers 63, 67, 70, 73 and 76 have unity gain and amplifiers 64, 68, 71, 74 and 77 are cosine functions of the assumed speaker angles. And amplifiers 65, 69, 72, 75 and 78 have a gain less than that which is a sinusoidal function of the assumed speaker angle.
[0035]
Matrix 53 takes these signals and provides new signals S1 ', S2', S3 ', S4' and S5 'to drive a speaker having a unique position surrounding the listening area. The description of the processing shown in FIG._{0}, Five amplifiers 8791, five amplifiers 9297 receiving signal a1, and b_{1}15 variable gain amplifiers 87103 grouped by five amplifiers 98103. The unique output of each of these three groups of amplifiers provides an input to summing node 105, the output of each other of these groups provides the input to summing node 107, and the other amplifiers are shown in the figure. As such, their outputs are connected to nodes 109, 111 and 113 in a similar manner.
[0036]
The relative gain of amplifiers 87103 is set to satisfy the following set of simultaneous equations that depend on the actual speaker angle β.
Where N = 5 in this example, and as a result, i and j have values of 1, 2, 3, 4 and 5. The result is that a home, theater or other user can “dial in” a specific angle taken by the loudspeaker position, which maintains the improved spatial performance provided by the mastering technology. You can even change it from time to time.
[0037]
The matrix equation of the above simultaneous equations for the actual speaker position angle β is as follows, where a second spatial harmonic condition equal to zero is also imposed.
The value of the relative gain of the amplifier 87103 is the actual speaker position angle β_{1}Β_{Five}The coefficient a which is the result of solving the above matrix for the output signals S1'S5 'of the circuit matrix 53._{0}, A_{1}And b_{1}Chosen to meet.
[0038]
In the foregoing description, the mastering and playback process was treated as including recording, as indicated by block 15 in each of FIGS. However, these processes may also be used when there is realtime transmission of mastered sound via block 15 to one or more playback positions.
[0039]
The description with respect to FIGS. 3 and 4 is mainly directed to the mastering of a threedimensional sound field or at least contributes to those from individual monophonic sources. Referring to FIG. 5, a technique for mastering recording or sound transmission from a signal representing a sound field in three dimensions is illustrated. The three microphones 121, 123, and 125 are identical and are arranged with respect to the sound field, and an audio signal m containing sound field information that enables reproduction of the sound field in a set of surround sound speakers._{1}, M_{2}And m_{Three}To produce. By placing such a microphone in, for example, a symphony hole, a signal is created, and an acoustic effect with a realistic directivity can be reconstructed from this signal.
[0040]
As shown at 127, these three signals can be immediately recorded or delivered by transmission in three channels. Next, m_{1}, M_{2}And m_{Three}The signal is played, processed and played at home, theater and / or elsewhere. The reproduction system includes a microphone matrix circuit 129 and a speaker matrix circuit 131, which are operated by a control processor 133 via respective circuits 135 and 137. Thereby, in order to accurately reproduce the original sound field with a specific unique arrangement of loudspeakers around the listening area, the microphone signal is controlled at the listening position so as to optimize the signals S1S5 supplied to the speakers. And can be processed. The matrix 129 represents the microphone signal m_{1}, M_{2}And m_{Three}To zero order and first order spatial harmonic signals a_{0}, A_{1}And b_{1}Expand. The speaker matrix 131 captures these signals and generates individual speaker signals S1S5 with the same algorithm as described for the matrix 53 of FIG. The control panel 139 allows the user at the listening position to specify the exact speaker position for use by the matrix 131 and any other parameters required.
[0041]
The arrangement of FIG. 6 is very similar to the arrangement of FIG. 5 except that it differs in the signal that is recorded or transmitted. Instead of recording or transmitting microphone signals at 127 (FIG. 5), microphone matrixing 129 is performed at the sound generation location (FIG. 6) and the resulting spatial harmonics a of the sound field a_{0}, A_{1}And b_{1}Is recorded or transmitted at 127 '. The control processor 141 and the control panel 143 are used at the mastering site. The control processor 145 and the control panel 147 are used at a listening place. The advantage of the system of FIG. 6 is that the recorded or transmitted signal does not depend on the type and placement of the microphone used, so this information does not need to be known at the listening location.
[0042]
An example of the microphone matrix 129 of FIGS. 5 and 6 is shown in FIG. 3 microphone signals m_{1}, M_{2}And m_{Three}Each is an input to a bank of three variable gain amplifiers. Signal m_{1}Is sent to amplifier 151153 and signal m_{2}Are connected to amplifiers 154156 to signal m_{Three}Applies to amplifiers 157159. One output of each bank of amplifiers results in a zerospace harmonic signal a_{0}Is connected to the addition node. Also, another one of the amplifier outputs of each bank is connected to summing node 163, resulting in a first spatial harmonic signal a_{1}become. In addition, the outputs of the third amplifiers in each bank are connected together at summing node 165 to provide a first harmonic signal b._{1}Supply.
[0043]
The gains of the amplifiers 151159 are individually set by the control processor 133 or 141 (FIG. 5 or FIG. 6) via the circuit 135. These gains define the transfer function of the microphone matrix 129. The required transfer function depends on the type and placement of the microphones 121, 123 and 125 used. FIG. 8 illustrates one specific microphone arrangement. The microphones can be the same, but this need not be the case. Only one of the microphones can be omnidirectional. As a specific example, each is a sound pressure gradient microphone having a cardioid pattern. These are arranged in a Yshaped pattern and their main sensitivity axes are pointed outward in the direction of the arrows. The directions of the microphones 121 and 125 are arranged at an angle α on the opposite side of the direction axis of the other microphones 123.
[0044]
In this particular example, the microphone signal can be expressed as follows, where ν is the sound source angle with respect to the direction axis of the microphone 123.
m_{1} = 1 + cos (ν−α)
m_{2} = 1−cosν (9)
m_{Three} = 1 + cos (ν + α)
The three spatial harmonic outputs of matrix 129 are as follows for the three microphone signal inputs:
Since these are linear equations, the gain of amplifiers 151159 is the m of these equations._{1}, M_{2}And m_{Three}It is the coefficient of each term.
[0045]
For clarity of explanation, various sound processing algorithms have been described for analog circuits. Some or all of the described matrices can be implemented in this way, but these can be used in digital sound mastering consoles that are commercially available when encoding signals for recording or transmission, and in the digital circuitry of the playback device. It is more convenient to implement the algorithm at the listening position. The matrix is then formed in digital form within the device in response to supplied software or firmware code that executes the algorithm described above.
[0046]
In both mastering and playback, the matrix is formed by parameters that include expected or actual speaker positions. There are almost no restrictions on the positions of these speakers. Whatever the speaker position, they are considered as parameters in various algorithms. Improved realism is needed by others, such as the use of diametrically opposed speaker pairs, speakers placed in the corners of rectangular room floors and ceilings, other specific linear arrangements, etc. Obtained without the need for suggested specific speaker positions. Rather, the process of the present invention allows the speakers to be first placed at a desired location around the listening area, and then their position is used as a parameter in signal processing to obtain a signal, which signal Through to reproduce the sound by a specified number of spatial harmonics that are substantially exactly the same as the spatial harmonics of the original audio wavefront.
[0047]
The spatial harmonics faithfully reproduced in the above example are zero and first harmonics, but higher order harmonics can be reproduced if sufficient speakers are used for that purpose. In addition, signal processing is the same for all frequencies being played back, and high quality systems extend from tens of hertz down to over 20,000 Hz. Separate signal processing in the two frequency bands is not necessary.
[0048]
3D representation
What has been discussed so far has been to present a method of twodimensional spatial harmonics by placing a load speaker and a sound source on one plane. This same theory can be extended to three dimensions. In that case, four channels are required to transmit the zeroth and first terms of the threedimensional spatial harmonic expansion. This has the same matrixing characteristics such that the two channels carry a standard stereo mix and the other two channels provide a feed for any number of speakers around the listener. Unfortunately, the mathematics for the 3D version is not as clean and compact as the 2D version. There is no good way to reduce complexity.
[0049]
In order to extend the spatial harmonic method to three dimensions, the Legendre function and the spherical harmonic function need to be discussed briefly. In a sense, this is a generalization of the Fourier sine and cosine series. The Fourier series is a function of one angle, φ. This series is periodic and can be used to represent a function on a circle. A spherical harmonic is a complete set of orthogonal functions defined on the surface of a sphere, such that the Fourier sine and cosine series are a complete set of orthogonal functions on a circle. As such, any function on the sphere can be represented by a spherical harmonic in the generalized Fourier series.
[0050]
The spherical harmonic function is a function of two coordinates on the sphere, that is, angles θ and φ. These are shown in FIG. 9, where the points on the sphere are represented by pairs (θ, φ), where φ is the azimuth. 0 degrees is straight ahead. 90 ° is on the left. 180 ° is straight rearward. θ is a declination (up and down). 0 degrees is straight up. 90 ° is a horizontal plane and 180 ° is straight down. Note that the range of θ is 0 to 360 °, whereas the range of φ is 0 to 360 ° (or −180 ° to 180 °). In the twodimensional discussion, the angular variable θ has been turned down and made equal to 90 °. More generally, both angles are included. For example, the positions of the speakers SP1, SP2, SP3, SP4 and SP5 in FIG._{1}, Φ_{1}), (Θ_{2}, Φ_{2}), (Θ_{Three}, Φ_{Three}), (Θ_{Four}, Φ_{Four}) And (θ_{Five}, Φ_{Five}), Where θ_{1}Is located somewhere in the range of 0 ° to 180 °. 1 and 8 can be viewed as a coplanar arrangement of the illustrated elements or a projection of a threedimensional situation onto a particular planar subspace.
[0051]
The general definition of a spherical harmonic begins with a Legendre polynomial, which is defined as
From these, we can define the associated functions of Legendre, which are defined as follows:
Where P_{0}(Cos θ) = 1, P_{1}(Cos θ) = cos θ, P_{1} ^{1}(Cos θ) = − sin θ. Both Legendre polynomials and related functions are orthogonal (but not orthonormal). These authors gave these specific definitions because some authors defined them somewhat difficult. If one of the alternative definitions is used, the following equation must be changed accordingly.
[0052]
Although these are polynomials, they can be changed to periodic functions by the following permutations.
μ≡cosθ (13)
From these, expansion of functions in polar coordinates can be performed as follows.
Function P_{n}(Cosθ), cosmφP_{n} ^{m}(Cosθ), and sinmφP_{n} ^{m}(Cos θ) is called a spherical harmonic function. This expansion is equivalent to the Fourier series of equation (1), but in practice it is relatively tedious to derive. One approach is to fix the value of θ, ie 90 °. The remaining terms are summed up to be equal to the Fourier sine and cosine series. Coefficient (A_{n}, A_{nm}, B_{nm}) Is the coefficient (a) in equation (1)_{0}, A_{m}, B_{m}) For n ≠ 0.
[0053]
For the function just defined on the circle, there are 1 + 2T coefficients for the series containing harmonics of order 0 to T. For spherical harmonic expansion, if harmonics up to order T are included, the total number of coefficients is (T + 1)^{2}Since a sphere is a twodimensional surface, a square appears. Thus, to preserve harmonics up to the first order, the three terms a_{0}, A_{1}And b_{1}Instead of four terms A_{0}, A_{1}, A_{11}And B_{11}Is now required.
[0054]
When applied to sound, this can be viewed as the sound pressure on the surface of a microscopic sphere at a point in space concentrated at the listener's position. This extension can be used as a guideline through the generation of pan matrix and microphone processing for sounds that can come from any direction around the listener.
[0055]
As in the 2D discussion, the function on the sphere to be approximated is the direction to the listener (θ_{0}, Φ_{0}) And the additional coordinate θ is now explicit. Μ for compactness_{0}Is defined as follows.
μ_{0} ≡cosθ_{0} (15)
The extension of the unit impulse in that direction can be calculated as follows:
Many different positions (θ_{0}, Φ_{0}For multipoint sources or nonpoint sources, the function is replaced by the sum of these points or the integral of the distribution, respectively.
[0056]
The discussion here has been made using 3D harmonics originating from spherical coordinates, but other orthogonal function sets in 3D could be used as well. The corresponding orthogonal function will be used instead in equation (16) and other equations. For example, if the 3D loudspeaker placement geometry of the listening area fits into a specific coordinate system, or the microscopic surface around the point corresponding to the listener is nonspherical due to the microphone placement or characteristics If present, one of the spheroid coordinate systems and the corresponding orthogonal expansion will be used.
[0057]
Returning to FIG. 1, (θ_{1}, Φ_{1}), (Θ_{2}, Φ_{2}), ..., (θ_{N}, Φ_{N}) At N angles, but N = 5 exemplary values and each is θ_{i}= 90 ° is no longer used. Gain to each speaker, g_{i}The resulting sound field around a point at the center is the desired sound field (f_{0}(Θ, φ) is required to be as good as possible as above. These gains are obtained by minimizing the integrated square difference between the resulting sound field and the desired sound field. The result of this optimization is the following matrix equation, which generalizes equation (2) by switching between the left and right sides.
BG = S (17)
Where G is a column vector of speaker gain.
G^{T} = [G_{1} ... g_{N} ] (18)
The components of matrix B can be calculated as follows:
further,
S = [b_{Ten}... b_{N0}]^{T} (20)
[0058]
Equation (19) is the term (1)^{m}Note that is similar to the expansion in equation (16) for unit impulses in a certain direction. The initial sum is written without an upper bound, but in practice this is a finite sum. The rank of the matrix B depends on how many expansion terms are stored. If the 0th and 1st terms are preserved, the rank of B will be 4. If another term is taken, the rank will be nine. The rank of B also determines the minimum number of speakers required to match that many expansion terms.
[0059]
Any number of speakers can be used, but if the number of speakers is a perfect square number (T + 1) corresponding to the Torder harmonic^{2}Otherwise, the simultaneous equations are undetermined. There are various ways to solve the undetermined system. One way is to solve the system using the pseudo inverse of matrix B. This is equivalent to choosing a minimum norm solution, resulting in a completely acceptable solution. Another method is to augment the system with equations that null out some higher order harmonics. This includes taking the minimum number of rows of B that store the rank and then adding a row of the form
[Ρ_{n + 1}(μ_{1}) ... Ρ_{n + 1}(μ_{N} ]] = [0] (21a)
Or
[Cosφ_{1} Ρ^{m} _{n + 1}(μ_{1}) ... cosφ_{N} Ρ^{m} _{n + 1}(μ_{N} ]] = [0] (21b)
Or
[Sinφ_{1} Ρ^{m} _{n + 1}(μ_{1}) ... sinφ_{N} Ρ^{m} _{n + 1}(μ_{N} ]] = [0] (21c)
These equations are generalizations of the process used to change equation (3) above to equation (4). Strictly speaking, which of these is taken has no significant effect. Each additional row increases the rank of the matrix until the maximum number of ranks is reached.
[0060]
Thus, the inventor has derived the matrix equation necessary to create speaker gain for panning a single (monophonic) sound source to multiple speakers that accurately preserves several spatial harmonics in three dimensions.
[0061]
FIGS. 3 and 4 illustrate the mastering and reconstruction process in the case of the coplanar example of two mono sources mixed into five signals, which are spatial harmonics via the first order. And finally matrixed into a set of modified signals. As mentioned there, the common multichannel configuration is the 5.1 format for movie and home cinema soundtracks, so it is convenient to select 5 signals for recording and 5 modified signals for output. Any of these specific choices could be varied. Alternative multichannel recording and playback methods are described in, for example, James A. et al. No. 09 / 505,556 “CD Playback Enlargement”, filed Feb. 17, 2000, by Moore, which is incorporated herein by reference.
[0062]
The arrangements of FIGS. 3 and 4 are applied to include threedimensional harmonics, the main change is that if harmonics up to T are preserved, instead of (1 + 2T) signals, this time (T + 1 )^{2}The signal becomes the output of the harmonic matrix 51. Thus, to preserve harmonics via the first order, the three terms (a_{0}, A_{1}, B_{1}) Instead of four terms (A_{0}, A_{1}, A_{11}, B_{11})Is required. In addition, the control processor 59 now has individual azimuth angles φ_{i}And β_{j}Instead of an assumed speaker angle pair (θ_{i}, Φ_{i}) And the actual speaker angle pair (γ_{j}, Β_{j}) To calculate the gain from (γ_{j}, Β_{j}) Is again provided via the control panel 61. Finally, one convenient choice for 3D is to use a modified set of 6 signals S1S6 and 6 signals S1′S6 ′ in a noncoplanar case. In any case, the spherical harmonics require at least four noncoplanar speakers, just as the 2D case requires at least three noncollinear speakers. This is because in order to define a sphere, at least four noncoplanar points are required, and three noncollinear points define a circle on the plane.
[0063]
The reason that six speakers are a convenient choice is that it mixes four to five tracks recorded or transmitted on medium 15 for coplanar placement and the remaining two to one track from the plane. This is because it can be used for a speaker placed at a distance. This allows the listener to access and use four to five coplanar tracks without an elevated speaker or without a playback device for spherical harmonics, while providing a complete threedimensional for the listener. The remaining tracks along with playback capabilities are still available on the media. This is similar to the situation described above for the 2D case where two channels can be used in conventional stereo playback, but additional channels can be used for sound field playback. In the 6 channel 3D case, 2 channels are used for stereo mix, enhanced with 2 channels for 4 channel surround sound recording, and the last 2 channels further enhance playback via 6 channels. Can be used to provide a dimensional sound field. The listener will be able to access the required number of channels from the stored medium, for example as described in the copending application “CD Playback Enlargement” included by reference above.
[0064]
Returning to FIG. 3, the modification in this example is to include one additional amplifier for each mono source and one added to provide additional signal S6 to medium 15. In that case, the control board 29 also provides additional gain for each of the sources, all gain being derived not only from the azimuthal position of the assumed speaker arrangement, but also from the declination. Similarly in FIG. 4, each of the six signals S1S6 is fed to four amplifiers in matrix 51, and the four outputs in this example are created using the 0th and 1st harmonics, so that A_{0}, A_{1}, A_{11}, B_{11}(Or more generally, these four independent primary combinations) provide one for the four summing nodes. Matrix 53 has six amplifiers for each of these four harmonics, creating a set of six correction signals S1'S6 '. Again, not only the azimuth position of the actual speaker arrangement but also the declination is used. More generally, the control board 61 could also provide the control processor 59 with radius information for any speaker that is not on the same sphere as the other speakers. In that case, the control processor 59 could use this information matrix 53 to produce a corresponding correction signal and introduce a delay to compensate for different radii, wavefront spread compensation, or both.
[0065]
In this arrangement, the one corresponding to the above equation (6) is as follows.
Four of the speakers, S1S4, are placed in a typical coplanar arrangement parallel to the room floor, θ_{1}−θ_{Four}== 90 °, equation (6 ′) becomes fairly simple. Furthermore, by having a complete threedimensional representation, it is possible to realize a twodimensional projection onto any other surface of the listening area by fixing θs and φs.
[0066]
A standard directional microphone has a pickup pattern that can be expressed as zero order and first order spatial spherical harmonics. The equation for a standard sound pressure gradient microphone pattern is:
In the equation, Θ and Φ are angles in the spherical coordinates of the microphone main axis. That is, these are the directions in which the microphone is “pointing”. Equation (22) is a more general form of equation (9). Their equations are up to the overall factor of both microphone m_{1}, M_{2}Or m_{Three}Corresponds to equation (22) where C = 1/2, θ = Θ = 90 °, φ = ν, and Φ = α, 0, or −α. The constant C is called “directivity” of the microphone and is determined by the type of the microphone. C is for an omnidirectional microphone and zero for an “eight character” microphone. The intermediate values provide standard pickup patterns such as cardioid (1/2), hypercardioid (1/4), super cardioid (3/8), and subcardioid (3/4). With four microphones, the zeroorder and firstorder spatial harmonics of the 3D sound field can be recovered as follows.
This equation corresponds to the 2D zeroth order and first order spatial harmonics of equation (10). The spatial harmonic coefficients on the left side of this equation are sometimes referred to as W, Y, Z, and X in commercially available sound field microphones. The representation of the threedimensional sound field by these four coefficients is sometimes called “Bformat”. (This terminology is simply to distinguish it from a direct microfeed, sometimes called “Aformat”.)
[0067]
Term m_{1}..., m_{M}Is the angle (Θ_{1}, Φ_{1}), ..., (Θ_{M}, Φ_{1}) M sound pressure gradient microphones. The matrix D can be defined by the reciprocal as follows.
[0068]
Each row of this matrix is one direction pattern of the microphone. Four microphones unambiguously determine all the coefficients for the zeroth and first order terms of the spherical harmonic expansion. The microphone angle must be identifiable (no two microphones pointing in the same direction) and noncoplanar (since it provides information in only one angular dimension, not two). In these cases, the matrix is in good condition and has an inverse.
[0069]
Corresponding changes are also required in FIG. 5, FIG. 6, and FIG. In FIG. 5 and FIG. 6, the number of microphones is 4, and m in the equation (23)_{1}M_{Four}And four more harmonics (A_{0}, A_{1}, A_{11}, B_{11}, Or four independent linear combinations), the three terms (a_{0}, A_{1}, B_{1}). The number of output signals is also adjusted. In the example used above, S6 or S6 ′ is included. In addition, the alignment of each microphone is specified by a pair of parameters, principal axis angles (Θ, Φ), and each of the signals S1S6 or S1'S6 'has a declination as well as an azimuth. Yes. Correspondingly, the microphone matrix of FIG. 7 has four sets of four amplifiers.
[0070]
One possible arrangement of the four microphones of equations (23) and (24) is m as shown in FIG._{1}M_{Three}On the equator, m_{Four}Is placed on the north pole of the sphere. This is (Θ_{1}, Φ_{1}), (Θ_{Three}, Φ_{Three}) = (90 °, ± α), (Θ_{2}, Φ_{2}) = (90 °, 180 °), corresponding to Θ4 = 0 °. Another option is to place a microphone with two microphones facing back, as shown in FIG._{1}121 is (90 °, α), m_{2}123 is (90 ° + δ, 180 °), m_{Three}125 is (90 °, α), and m_{Four}126 is (90 ° −δ, 180 °). If α = δ = 60 °, a regular tetrahedral arrangement is obtained.
[0071]
In some applications, one of the microphones may be placed at various radii for practical reasons, in which case some delay or advance of the corresponding signal should be introduced. For example, the microphone m facing rearward in FIG._{2}Would move backwards, the recording would advance approximately 1 ms for every 1 foot moved to ensure propagation time.
[0072]
Equation (23) is valid for any set of four microphones, again assuming that only one of them is omnidirectional. Looking at this equation for two different sets of microphones, the directional pattern of the pickup can be changed by matrixing these four signals. The starting point is equations (23) and (24) and their corresponding matrix D for two different sets of microphones. The actual microphone and matrix are indicated by the letters m and D, along with the rematrixed “virtual” quantity shown by Tilde.
[0073]
Given the formulation of equations (23) and (24), these microphone supplies can be converted into a set of “virtual” microphone supplies as follows:
[0074]
Matrix D (note: tilde D) represents the directionality and angle of the “virtual” microphone. The result of this is a sound that would have been recorded if a virtual microphone was present in the recording unit instead of the microphone used. This allows recording using "general" sound field microphones and then later matrixing them into any set of microphones. For example, the first two virtual microphones, m_{1}(Note: Tilde m_{1}) And m_{2}(Note: Tilde m_{2}) Only, and they could be used as a stereo pair for standard CD recording. Next, m_{Three}(Note: Tilde m_{Three}) Used for full threedimensional realization in the abovementioned planar surround sound recording_{Four}(Note: Tilde m_{Four}) Could be added.
[0075]
Any nondegenerate transform of these four microphone feeds can be used to create any other set of microphone feeds, or any one that can accurately reproduce the zeroth and first order spatial harmonics of the original sound field. Can be used to generate speaker feeds for a few (four or more) speakers. In other words, the sound field microphone technology can be used to adjust the directional characteristics and angle of the microphone after recording is complete. Thus, by adding a third rear facing microphone for 2D and a fourth noncoplanar microphone for 3D, the microphone can be modified with a simple matrix operation. Regardless of whether the data is intended to be released in a multichannel format, recording of the third rearward channel increases the freedom of stereo release, and recording of the fourth noncoplanar channel enables stereo and Increases freedom in planar surround sound.
[0076]
To matrix the microphone feed into a large number of speakers, the right side of the matrix equation (17) is reformulated for panning as follows.
and
Matrix R_{1}Is simply the zerothorder and firstorder spherical harmonics evaluated at the speaker position. Term (1)^{m}Must be cautious to include. This is because it is a direct result of the least squares optimization necessary to derive these equations.
[0077]
Returning to the recording of the sound field, three to four channels of (preferably uncompressed) audio data corresponding to 2D and 3D sound fields, respectively, are stored on the disc or other medium, and then simply Rematrix to stereo or surround in a simple manner. With equation (25) (or its 2D reduction), there are countless lossless nondegenerate transformations of the four channels to the other four channels. Thus, instead of storing spatial harmonics, two channels store the appropriate stereo mix, the third stores the channel for 2D surround, and uses the fourth channel for 3D surround mix. It will be possible. In addition to audio, a matrix or its inverse can also be stored on the medium. For stereo performances, the player simply ignores the third and fourth channels of audio and plays the other two as left and right feeds. For 2D surround demonstrations, the inverse of the matrix is used to derive the 0th and 1st order 2D spatial harmonics from the first three channels. From the spatial harmonics, a matrix like the planar projection of equation (8) or equation (17) is formed and the speaker feed is calculated. For a 3D surround demonstration, 3D harmonics are derived from using all four channels, forming a matrix of equation (17) and calculating the speaker feed.
[0078]
While the various aspects of the present invention have been described with reference to preferred embodiments thereof, it will be understood that the invention is entitled to protection within the full scope of the appended claims.
[Brief description of the drawings]
FIG. 1 is a plan view of an arrangement of multiple loudspeakers surrounding a listening area.
FIG. 2 illustrates an acoustic spatial frequency of the sound reproduction arrangement of FIG.
FIG. 3 is a block diagram of a matrix system for determining the position of monaural sound.
FIG. 4 is a block diagram for rematrixing the signals matrixed in FIG. 3 in order to take into account speaker positions different from those assumed when matrixing the initial signals.
FIG. 5 is a block diagram illustrating an alternative arrangement for acquiring and playing sound from a multidirectional microphone.
FIG. 6 is a block diagram illustrating an alternative arrangement for acquiring and playing sound from a multidirectional microphone.
7 shows further details of the microphone matrix block of FIGS. 5 and 6. FIG.
FIG. 8 shows an arrangement of three microphones as audio signal sources for the systems of FIGS. 5 and 6.
FIG. 9 illustrates the arrangement of spherical coordinates.
FIG. 10 shows an angular arrangement for a threedimensional array of four microphones.
Claims (30)
 A method for processing a sound field for sound field reproduction over a predetermined frequency range via a surround sound system having at least four channels individually supplied to one of at least four speakers, comprising:
Acquiring a plurality of signals of the sound field, and sending the acquired sound field signals to individual channels of the plurality of channels with a set of relative gains for the entire frequency domain;
The set includes a relational expression including a position term that is a selected speaker position around the listening area but is not constrained to the same planar arrangement , and substantially represents each of the plurality of threedimensional spatial harmonics of the sound field. Is determined by solving to save
Thereby, the sound field reproduced from the speaker located at the selected position substantially reproduces a plurality of threedimensional spatial harmonics of the acquired sound field.  The method of claim 1, wherein the plurality of substantially conserved threedimensional spatial harmonics includes only zero and first harmonics.
 The method of claim 1, wherein the plurality of substantially conserved threedimensional spatial harmonics includes zeroth to nth harmonics, wherein n is an integer less than or equal to an integer less than the square root of the number of speakers.
 The step of acquiring the plurality of signals of the sound field includes acquiring a plurality of monaural signals of the sound to be placed at a specific position around the listening area, and the relational expression includes such a specific position, whereby the speaker The method of claim 1, wherein the sound field reproduced from further comprises a monaural sound at the specific location.
 The method of claim 1, wherein obtaining a plurality of signals of a sound field includes positioning a plurality of directional microphones in a sound field.
 The method of claim 1, wherein the set of relative gains is determined at least in part by a relational expression that includes an assumed speaker position around a listening area.
 The method of claim 1, wherein the set of relative gains is determined at least partially at a location adjacent to the listening area by a relational expression that includes actual speaker positions around the listening area.
 The method of claim 1, wherein the set of relative gains is further determined by what substantially aligns the velocity and power vectors.
 The method of claim 1, wherein the set of relative gains is further determined by one that minimizes second or higher order of the plurality of threedimensional spatial harmonics.
 10. A method according to any of claims 1 to 9, wherein the surround sound system has exactly 6 channels, each of which feeds to a separate one of exactly 6 speakers.
 The method of claim 10, wherein at least one of the just six speakers is arranged to be noncoplanar with the other of the just six speakers.
 A method for simulating a desired apparent threedimensional position of a sound in a multichannel surround sound system, comprising:
Including monophonically acquiring a sound to be simulated for a threedimensional position, and sending the acquired monaural sound to a plurality of individual channels with a set of relative gains;
The set is a set of positions corresponding to the expected position of a speaker driven by each of the multichannel signals extending around the point and a declination and azimuth of the desired apparent position of the sound. respect, as if monaural sound when the sound is reproduced through the loudspeaker at the expected position is actually present at the position of the apparent, at least zero order and first order of the threedimensional harmonic sounds substantially Method determined by solving to save .  The speakers are actually arranged so that at least one of them has an actual position that is different from the expected position, and a modified set of relative gains is obtained by solving a second relational expression that includes the actual position of the speaker. Calculating at least zero and first order threedimensional harmonics of the sound as if the monaural sound was actually present at the apparent position when the sound was played through the speaker at the expected position. The method according to claim 12, wherein the method is stored.
 14. A method according to any of claims 12 or 13, wherein the set of relative gains is further determined by what substantially aligns the velocity and power vector of the sound field reproduced through the speaker.
 14. The relative gain set is further determined by one that minimizes second and higher order threedimensional spatial harmonics of the sound field reproduced through the speaker. the method of.
 The method according to claim 12 or 13, wherein the number of channels is 4 or more.
 The method according to claim 12 or 13, wherein the number of channels is exactly six.
 The method of claim 16, wherein at least one of the predicted positions of the speaker is noncoplanar with other predicted positions of the speaker.
 A method of reproducing a threedimensional sound field through four or more speakers arranged around a listening area,
Obtaining a plurality of electrical signals representing a sound field;
Processing the plurality of electrical signals to generate signals of at least zero and first order threedimensional spatial harmonics of the sound field, and obtaining a relational expression including an actual position term of the speaker ; Solve to substantially preserve at least the zeroth and first order threedimensional harmonics of the sound field reproduced through the loudspeaker, corresponding to the zeroth and first order threedimensional harmonics of the generated sound field, respectively. it allows including the step of threedimensional space harmonics processed to determine the relative gain applied to the individual loudspeakers,
Method.  The method of claim 19, further comprising recording and reproducing a plurality of electrical signals representative of the sound field.
 The method of claim 19, further comprising recording and reproducing a sound field harmonic signal.
 The method according to any one of claims 19 to 21, wherein the sound field is reproduced via exactly six speakers.
 23. The method of claim 22 , wherein at least one of the just six speakers is arranged to be noncoplanar with the other of the just six speakers.
 An input unit for receiving at least four audio signals of an original sound field intended to be reproduced by each of at least four speakers at a specific assumed position surrounding the listening area; and a listening area different from the assumed position A sound reproduction system having an output for driving at least four loudspeakers in a specific actual position to surround,
In response to the input actual speaker position including the declination and azimuth and the input actual speaker position including the declination and azimuth and the assumed speaker position. An electronically executed matrix that provides other signals from the input signal to the output,
The output unit drives a speaker to reproduce a sound field by a plurality of threedimensional spatial harmonics, and the harmonics are individually and substantially each of the same number of threedimensional spatial harmonics in the original sound field. Match the system.  The matrix includes a first part that generates individual signals corresponding to a plurality of threedimensional spatial harmonics, and a threedimensional spatial harmonic signal and actual speaker position information from the assumed speaker position information and the input signal. 25. The sound system of claim 24, further comprising a second portion that produces individual signals for actual speakers.
 26. A sound system according to any of claims 24 or 25, wherein the plurality of matched threedimensional spatial harmonics includes zero and first harmonics.
 26. A sound system according to any of claims 24 or 25, wherein the plurality of matched threedimensional spatial harmonics includes only zero and first harmonics.
 26. A sound system according to any of claims 24 or 25, wherein the plurality of speakers at actual speaker locations comprises exactly six.
 26. The sound system of claim 25, wherein at least one of the actual speaker positions is arranged to be noncoplanar with the other of the actual speaker positions.
 A sound system including an input unit that receives an audio signal of an original threedimensional sound field and an output unit that drives at least four loudspeakers at specific actual positions surrounding the listening area to reproduce the sound field. And
An input unit that accepts information about the actual position of the speaker, and an electronically executed matrix that provides signals to the output unit in response to the input actual speaker position information and the input signal;
The output unit drives a speaker to reproduce a sound field by a plurality of threedimensional spatial harmonics, and the harmonics substantially correspond to the same number of threedimensional spatial harmonics in the original sound field. A system that matches individually.
Priority Applications (3)
Application Number  Priority Date  Filing Date  Title 

US09/552,378  20000419  
US09/552,378 US6904152B1 (en)  19970924  20000419  Multichannel surround sound mastering and reproduction techniques that preserve spatial harmonics in three dimensions 
PCT/US2000/027851 WO2001082651A1 (en)  20000419  20001006  Multichannel surround sound mastering and reproduction techniques that preserve spatial harmonics in three dimensions 
Publications (2)
Publication Number  Publication Date 

JP2003531555A JP2003531555A (en)  20031021 
JP4861593B2 true JP4861593B2 (en)  20120125 
Family
ID=24205070
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

JP2001578151A Active JP4861593B2 (en)  19970924  20001006  Multichannel surround sound mastering and playback method for preserving 3D spatial harmonics 
Country Status (6)
Country  Link 

EP (1)  EP1275272B1 (en) 
JP (1)  JP4861593B2 (en) 
CN (1)  CN1452851A (en) 
AU (1)  AU8003000A (en) 
CA (1)  CA2406926A1 (en) 
WO (1)  WO2001082651A1 (en) 
Cited By (1)
Publication number  Priority date  Publication date  Assignee  Title 

JP2016509819A (en) *  20130207  20160331  クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ  Determining the renderer for spherical harmonics 
Families Citing this family (23)
Publication number  Priority date  Publication date  Assignee  Title 

GB2379147B (en) *  20010418  20031022  Univ York  Sound processing 
JP4016681B2 (en)  20020318  20071205  ヤマハ株式会社  Effect imparting device 
SE0400997D0 (en) *  20040416  20040416  Cooding Technologies Sweden Ab  Efficient coding of multichannel audio 
US9015051B2 (en)  20070321  20150421  FraunhoferGesellschaft Zur Foerderung Der Angewandten Forschung E.V.  Reconstruction of audio channels with direction parameters indicating direction of origin 
US8908873B2 (en)  20070321  20141209  FraunhoferGesellschaft Zur Foerderung Der Angewandten Forschung E.V.  Method and apparatus for conversion between multichannel audio formats 
US8290167B2 (en)  20070321  20121016  FraunhoferGesellschaft Zur Foerderung Der Angewandten Forschung E.V.  Method and apparatus for conversion between multichannel audio formats 
TWI455064B (en)  20071220  20141001  Thomson Licensing  Method and device for calculating the salience of an audio video document 
WO2009093867A2 (en)  20080123  20090730  Lg Electronics Inc.  A method and an apparatus for processing audio signal 
EP2083584B1 (en)  20080123  20100915  LG Electronics Inc.  A method and an apparatus for processing an audio signal 
KR101024924B1 (en) *  20080123  20110331  엘지전자 주식회사  A method and an apparatus for processing an audio signal 
PL2154677T3 (en) *  20080813  20131231  Fraunhofer Ges Forschung  An apparatus for determining a converted spatial audio signal 
GB2478834B (en) *  20090204  20120307  Richard Furse  Sound system 
ES2472456T3 (en) *  20100326  20140701  Thomson Licensing  Method and device for decoding a representation of an acoustic audio field for audio reproduction 
MX338525B (en)  20101203  20160420  Fraunhofer Ges Zur Förderung Der Angewandten Forschung E V  Apparatus and method for geometrybased spatial audio coding. 
CN105792086B (en)  20110701  20190215  杜比实验室特许公司  It is generated for adaptive audio signal, the system and method for coding and presentation 
US8996296B2 (en) *  20111215  20150331  Qualcomm Incorporated  Navigational soundscaping 
CN102695116B (en) *  20120530  20150603  蒋憧  Sound acquisition, processing and recurrence method 
CN102752701B (en) *  20120710  20140917  武汉大学  Device and method for testing direction perception sensitivity of threedimensional space 
US9288603B2 (en)  20120715  20160315  Qualcomm Incorporated  Systems, methods, apparatus, and computerreadable media for backwardcompatible audio coding 
US9473870B2 (en)  20120716  20161018  Qualcomm Incorporated  Loudspeaker position compensation with 3Daudio hierarchical coding 
CN102932730B (en) *  20121108  20140917  武汉大学  Method and system for enhancing sound field effect of loudspeaker group in regular tetrahedron structure 
EP2733964A1 (en) *  20121115  20140521  FraunhoferGesellschaft zur Förderung der angewandten Forschung e.V.  Segmentwise adjustment of spatial audio signal to different playback loudspeaker setup 
US9959875B2 (en)  20130301  20180501  Qualcomm Incorporated  Specifying spherical harmonic and/or higher order ambisonics coefficients in bitstreams 
Family Cites Families (5)
Publication number  Priority date  Publication date  Assignee  Title 

GB9103207D0 (en) *  19910215  19910403  Gerzon Michael A  Stereophonic sound reproduction system 
US5757927A (en) *  19920302  19980526  Trifield Productions Ltd.  Surround sound apparatus 
JPH08130793A (en) *  19941101  19960521  Mitsubishi Electric Corp  Acoustic reproducing device 
JPH1118199A (en) *  19970626  19990122  Nippon Columbia Co Ltd  Acoustic processor 
AU6400699A (en) *  19980925  20000417  Creative Technology Ltd  Method and apparatus for threedimensional audio display 

2000
 20001006 CA CA 2406926 patent/CA2406926A1/en not_active Abandoned
 20001006 CN CN 00819561 patent/CN1452851A/en not_active Application Discontinuation
 20001006 JP JP2001578151A patent/JP4861593B2/en active Active
 20001006 AU AU8003000A patent/AU8003000A/en active Pending
 20001006 EP EP00970687A patent/EP1275272B1/en active Active
 20001006 WO PCT/US2000/027851 patent/WO2001082651A1/en active Application Filing
Cited By (1)
Publication number  Priority date  Publication date  Assignee  Title 

JP2016509819A (en) *  20130207  20160331  クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ  Determining the renderer for spherical harmonics 
Also Published As
Publication number  Publication date 

CA2406926A1 (en)  20011101 
AU8003000A (en)  20011107 
EP1275272B1 (en)  20121121 
CN1452851A (en)  20031029 
JP2003531555A (en)  20031021 
WO2001082651A1 (en)  20011101 
EP1275272A1 (en)  20030115 
Similar Documents
Publication  Publication Date  Title 

US20160174012A1 (en)  System and method for processing an input signal to produce 3d audio effects  
RU2719283C1 (en)  Method and apparatus for reproducing threedimensional sound  
US9374641B2 (en)  Device and method for driving a sound system and sound system  
US20170358308A1 (en)  Sound system  
US20150294672A1 (en)  Method And Device For Decoding An Audio Soundfield Representation For Audio Playback  
JP5431249B2 (en)  Method and apparatus for reproducing a natural or modified spatial impression in multichannel listening, and a computer program executing the method  
US9014404B2 (en)  Directional electroacoustical transducing  
EP2868119B1 (en)  Method and apparatus for generating an audio output comprising spatial information  
JP2015530825A (en)  System for rendering and playing back audio based on objects in various listening environments  
US5870484A (en)  Loudspeaker array with signal dependent radiation pattern  
US4817149A (en)  Threedimensional auditory display apparatus and method utilizing enhanced bionic emulation of human binaural sound localization  
Lipshitz  Stereo microphone techniques: Are the purists wrong?  
DE69726262T2 (en)  Sound recording and playback systems  
US8345899B2 (en)  Phaseamplitude matrixed surround decoder  
US7215782B2 (en)  Apparatus and method for producing virtual acoustic sound  
TW447223B (en)  Apparatus and method for synthesizing pseudostereophonic outputs from a monophonic input  
AU2003275290B2 (en)  System and method for integral transference of acoustical events  
US20130236039A1 (en)  Method and apparatus for playback of a higherorder ambisonics audio signal  
JP4505058B2 (en)  Multichannel audio emphasis system for use in recording and playback and method of providing the same  
Berge et al.  High angular resolution planewave expansion  
US5459790A (en)  Personal sound system with virtually positioned lateral speakers  
US20160119737A1 (en)  Arrangement and method for reproducing audio data of an acoustic scene  
US4524451A (en)  Sound reproduction system having sonic image localization networks  
CN100588286C (en)  Device and method for producing a lowfrequency channel  
US6021206A (en)  Methods and apparatus for processing spatialised audio 
Legal Events
Date  Code  Title  Description 

A621  Written request for application examination 
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20070910 

A131  Notification of reasons for refusal 
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20100601 

A711  Notification of change in applicant 
Free format text: JAPANESE INTERMEDIATE CODE: A711 Effective date: 20100623 

A521  Written amendment 
Free format text: JAPANESE INTERMEDIATE CODE: A821 Effective date: 20100623 

A521  Written amendment 
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20100831 

A521  Written amendment 
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20100830 

A131  Notification of reasons for refusal 
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20110324 

A521  Written amendment 
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20110623 

A131  Notification of reasons for refusal 
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20110715 

A521  Written amendment 
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20111004 

TRDD  Decision of grant or rejection written  
A01  Written decision to grant a patent or to grant a registration (utility model) 
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20111031 

A01  Written decision to grant a patent or to grant a registration (utility model) 
Free format text: JAPANESE INTERMEDIATE CODE: A01 

A61  First payment of annual fees (during grant procedure) 
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20111107 

R150  Certificate of patent or registration of utility model 
Free format text: JAPANESE INTERMEDIATE CODE: R150 Ref document number: 4861593 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 

FPAY  Renewal fee payment (event date is renewal date of database) 
Free format text: PAYMENT UNTIL: 20141111 Year of fee payment: 3 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 

R250  Receipt of annual fees 
Free format text: JAPANESE INTERMEDIATE CODE: R250 