EP2208369A2 - Installation de projecteur sonore - Google Patents
Installation de projecteur sonoreInfo
- Publication number
- EP2208369A2 EP2208369A2 EP08844786A EP08844786A EP2208369A2 EP 2208369 A2 EP2208369 A2 EP 2208369A2 EP 08844786 A EP08844786 A EP 08844786A EP 08844786 A EP08844786 A EP 08844786A EP 2208369 A2 EP2208369 A2 EP 2208369A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- sound
- maximum correlation
- channel
- angle
- sound channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 91
- 238000012360 testing method Methods 0.000 claims abstract description 90
- 230000000875 corresponding effect Effects 0.000 description 20
- 230000000694 effects Effects 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 230000010076 replication Effects 0.000 description 8
- 230000005236 sound signal Effects 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000003111 delayed effect Effects 0.000 description 5
- 230000002596 correlated effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000001934 delay Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000414 obstructive effect Effects 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2203/00—Details of circuits for transducers, loudspeakers or microphones covered by H04R3/00 but not provided for in any of its subgroups
- H04R2203/12—Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2205/00—Details of stereophonic arrangements covered by H04R5/00 but not provided for in any of its subgroups
- H04R2205/022—Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
Definitions
- the present invention concerns a device including a plurality of acoustic transducers.
- the device is capable of producing beams of audible sound.
- the invention is directed towards such devices when used for home entertainment or professional sound reproduction applications.
- the invention concerns methods and systems useful for configuring (i.e. setting up) such devices.
- WO 01/23104 WO 2/078388, WO 03/034780, WO 03/059005, WO 03/071827, WO 2004/075601, WO 2005/027514, WO 2005/086526, WO 2006/003528,
- WO 2007/007083 the disclosure of each of which is hereby incorporated by reference, describe systems comprising an array of transducers and their use to achieve a variety of sound effects.
- These documents describe methods and apparatus for taking an input signal, replicating it a number of times and modifying (e.g. delaying) each of the replicas before routing the modified replicas to respective output transducers such that a desired sound field is created.
- This sound field may comprise, inter alia, a directed, steerable, beam, a focussed beam or a simulated origin.
- Sound Projector The teachings of the above-mentioned documents may be used to implement aspects of the present invention.
- a surround sound system employs a left, centre and right speaker located in the front half space and two rear speakers located in the rear half space.
- “rear” are used relative to the listener's position and orientation.
- a subwoofer is also often provided, and it is usually specified that the subwoofer can be placed anywhere in the listening environment.
- These known surround sound systems receive input audio information and decode the information according to known standards (such a Dolby Surround 5.1 or Digital Theatre Sound - DTS).
- the decoded information provides output audio streams for the various audio channels, with each audio channel usually being emitted through one loudspeaker or a combination of two speakers.
- the audio information can itself comprise the information for each of the several channels (as in Dolby Surround 5.1) or for only some of the channels, with other channels being simulated (as in Dolby Pro-Logic systems).
- WO 01/23104 and WO 02/078388 problems were addressed in WO 01/23104 and WO 02/078388 by the provision of a Sound Projector that can generate a surround-sound environment using only a signal standalone bank of output transducers.
- the described system uses an array of output transducers to simultaneously direct a plurality of sound beams, with each beam representing one of the surround sound channels, with such sound beams being reflected from surfaces such as the ceiling and walls back to the listener.
- the listener perceives the sound beam as if it was emitted from an acoustic mirror image of a source located at or behind the spot where the last reflection took place.
- the surround sound effect is dependent upon the beams reaching the listener from the correct direction. This in turn is dependent on the room geometry and so it is necessary with this system for the apparatus to be correctly set up such that the various sound beams are emitted in the correct directions to achieve the surround sound effect. Furthermore, it is important in a surround sound system that temporally corresponding parts of the different channels reach the user at the same time. As the sound beam channels follow different path lengths, it is necessary to relatively delay some channels compared to others in order to ensure proper temporal correspondence at the user's position (see WO 02/078388). Accordingly, it was necessary in the prior art for the Sound Projector systems to be programmed by experts to ensure that the geometry of the particular room was taken into account.
- WO 2004/066673 discloses two methods for setting up a Sound Projector system. The methods disclosed rely on initially using the Sound Projector to provide a model of the room environment. Once the room environment model has been calculated, choices can be made about where to direct the sound beams such that they reach the listener from the appropriate direction and at the appropriate time.
- the Sound Projector is provided with a microphone located at the centre of the Sound Projector array. Pulses of sound are emitted from the array in chosen directions and the microphone is used to detect the reflected, refracted and diffracted return sounds. The time is measured between emitting a pulse and receiving the first reflected pulse back.
- This embodiment works on the principal that all objects, whatever their orientation, will reflect some of the sound pulse precisely back in the direction from which it came. Accordingly, an indication of the distance to the nearest object in each direction can be obtained. This is used to provide a computer model of the room in which the Sound Projector is located.
- the second disclosed embodiment of WO 2004/066673 uses a microphone at the listening position and uses one of the transducers in the array to output a sound pulse. No sound beams are formed in this embodiment.
- a fuzzy clustering algorithm is used to estimate the room geometry. -A-
- Both of the embodiments of WO 2004/066673 suffer from the problem that the set-up procedure is predicated on estimating the geometry of the room (i.e. the position of the walls, ceiling etc). Any error in determining the geometry of the room can have highly adverse consequences for the surround sound produced by the resulting system. For example, if a wall is mis-positioned by even just a few centimetres, or is positioned correctly but deemed to have an angle (with respect to the Sound Projector) that is inaccurate by a few degrees, then the sound beams can take a very different path than predicted, hi general, all of the walls and ceiling positions/orientations predicted by the method will have some error and these errors will accumulate to provide that the sound beams do not, in general, follow the intended path. Furthermore, creating and storing a computer model of the listening environment requires high processing power and a lot of memory, especially given the numerous different shapes of listening environment that are possible.
- the present invention stems in part from the realisation that better Sound Projector set-up can be achieved without creating a model of the listening environment. Accordingly, the invention provides that an input transducer
- the listening position e.g. a microphone
- the listening position is used to "listen" to a test signal as it is swept in a beam through a range of angles. From this information the system can determine those angles where the sound beam reaches the input transducer with maximum intensity. Furthermore, the path length of the sound beams can be determined by measuring the time from emitting the test signal to receiving the maximum reflection of the test signal. Methods and apparatus are disclosed for identifying the appropriate reflection from a received signal that can be the result of a very complicated superposition of a lot of reflections arriving at different times.
- the resulting surround sound system provides a good surround sound effect at the listening position. Furthermore, the computational complexity of calculating and storing a model of the listening space is avoided and the errors that this method can introduce are thereby also avoided.
- the method and system of the present invention also allows the path length of the various beams to be determined, thereby assisting in selecting which channels are directed in which directions and also assisting in providing the data necessary for the Sound Projector to ensure that, in use, the channels reach the listening position time- synchronously.
- the path length information is also extremely useful (preferably in conjunction with the angle and maximum correlation data) in selecting the most reliable possible beam directions.
- the method and apparatus of the present invention need not concern itself with extraneous information that the skilled person would learn is important from WO 2004/066673, such as the geometry of the room, the nature of the reflective surfaces in the environment (e.g. the wall materials, ceiling materials and coverings), the locations of sound reflective and/or sound obstructive obstacles in the environment (e.g. furniture) or the locations of sound "apertures" (such as doors and windows).
- the present invention takes the listening environment as it is and identifies the paths that are the best. Accordingly, all of the above factors are automatically taken into account without requiring any special processing.
- the set-up method of the invention can use an input transducer, such as a microphone, at the listening position which detects specific test signals emitted from the Sound Projector.
- the method can use the detected signals to compute the optimum beam parameters, allowing optimum set-up of the Sound Projector, for example for surround sound listening.
- the invention provides a method for use in setting up a speaker system that comprises an array of output transducers capable of generating plural directed beams of sound, said method comprising: emitting a test signal in the form of a directed sound beam from the speaker system into a room; detecting a reflection of said sound beam test signal at a location in said room; and determining a time of a maximum correlation between said emitted sound beam test signal and said reflected sound beam test signal.
- the sound beam is preferably emitted at an angle into the room.
- the determined time of maximum correlation is preferably used to calculate a path length for said directed sound beam.
- the time of maximum correlation is preferably obtained by convolving the reflected test signal detected at said location in the room with the emitted test signal and determining the time at which said maximum correlation occurs.
- the method can advantageously be repeated for a plurality of angles and the magnitude of the maximum correlation between said emitted sound beam test signal and said reflected sound beam test signal is determined for each of said plurality of angles.
- the test signal is usefully in the form of a directed sound beam that is swept through a range of angles.
- the method preferably further comprises storing, for each emission angle of said directed sound beam test signal, the time of said maximum correlation and preferably also the magnitude of said maximum correlation.
- Angles are generally measured with respect to a line passing through the centre of said speaker system and being perpendicular to the front face of said speaker system.
- the method may further comprise initially determining the position of said location in said room relative to said speaker system by detecting direct, non-reflected, test signals at said location in said room.
- a sound beam direction for a centre channel is preferably selected, which beam direction points from said speaker system directly towards said location in said room.
- a sound beam direction for a centre channel is preferably selected to be the direction for which a maximum correlation exists between said emitted sound beam test signal and a directly received sound beam test signal at said location in said room. Those angles for which a peak of maximum correlation exists are preferably identified.
- a peak is typically defined as existing for an angle at which the maximum correlation is greater than the maximum correlation for either of the two neighbouring angles.
- the determined time of maximum correlation can be used to select the angle of at least one of the surround sound channels.
- a sound beam direction for each of a left sound channel and a right sound channel is typically selected, which direction corresponds to an angle at which there is a peak magnitude of maximum correlation to the left and right of the centre channel, respectively.
- a sound beam direction for a left sound channel is usefully selected, which direction corresponds to the angle at which there is the largest peak magnitude of maximum correlation out of all of the interrogated angles to the left of the centre channel.
- a sound beam direction for a right sound channel is usefully selected, which direction corresponds to the angle at which there is the largest peak magnitude of maximum correlation out of all of the interrogated angles to the right of the centre channel.
- the method preferably determines if the peak magnitude of maximum correlation for the left sound channel is smaller than a predetermined percentage of the peak magnitude of maximum correlation for the centre channel, and then switches the speaker system to "direct mode".
- the method preferably determines if the peak magnitude of maximum correlation for the right sound channel is smaller than a predetermined percentage of the peak magnitude of maximum correlation for the centre channel, and then switches the speaker system to "direct mode".
- the selected sound beam direction for the left surround sound channel preferably corresp ⁇ >to an angle at which there is a peak magnitude of maximum correlation to the left of the centre channel and which direction has a smaller beam angle and larger time of maximum correlation than the selected direction for the left sound channel.
- the selected sound beam direction for the right surround sound channel preferably corresponds to an angle at which there is a peak magnitude of maximum correlation to the right of the centre channel and which direction has a smaller beam angle and larger time of maximum correlation than the selected direction for the right sound channel.
- the method preferably further comprises determining a plurality of candidate beam directions for the surround sound channels and assigning a quality coefficient to each candidate.
- the quality coefficient is initially preferably determined by multiplying the maximum correlation value for the beam angle by the beam path length although the path length used when calculating the quality coefficient is preferably limited to some multiple of the path length calculated for the left sound channel or right sound channel, respectively.
- the method may include disregarding candidate beams.
- candidate beams can be disregarded, preferably by setting the quality coefficient to be zero, if any of the following conditions (a) to (e) exist:
- the candidate beam has a larger angle and a larger time of maximum correlation than an already selected left or right sound channel
- the candidate beam has a smaller angle and a smaller time of maximum correlation than an already selected left or right sound channel
- the candidate beam has a time of maximum correlation within a predetermined amount (e.g. 100 - 500 ⁇ s, preferably 200 ⁇ s) and an emission angle within a predetermined amount (e.g. 5 - 25°, preferably 15°) of an already determined candidate beam;
- a predetermined amount e.g. 100 - 500 ⁇ s, preferably 200 ⁇ s
- an emission angle within a predetermined amount (e.g. 5 - 25°, preferably 15°) of an already determined candidate beam
- the candidate beam is directed in a direction where the path length for adjacent angles differs by a predetermined amount from the candidate beam path length.
- the candidate beam direction having the largest quality coefficient is selected for a surround sound channel.
- the focal length for the centre channel can be advantageously set to be a fixed negative number, such as -0.5 m.
- the focal length for the left sound channel and/or the right sound channel can preferably be set based on the path lengths for those channels, preferably being set at one half of the path length.
- a correlation can be calculated only for those sound beams having a path length larger than the distance from the speaker system to the location in said room.
- a "corner mode” can be set for the speaker system.
- the left sound channel and right sound channel can be emitted in the "direct mode", hi the "direct mode", the left sound channel and right sound channel can be directed in the same direction as the centre sound channel and a subset of the transducers in the array can be used to emit the left sound channel and right sound channel.
- the focal length of the left sound channel and the right sound channel can advantageously be set to be a fixed negative number, such as -0.5m.
- the test signal comprises a maximum length sequence.
- the correlation is usefully calculated in real time, in the time between samples of the test signal.
- the reflection of said sound beam test signal is preferably detected at the listening position by an omnidirectional microphone.
- the invention provides in another aspect a method of outputting surround sound, said method comprising: performing the method described above; determining beam emission angles for a centre, left, right, surround left and surround right channel respectively; and emitting sound beams representing said respective channels in directions given by said respective angles.
- the invention includes apparatus for outputting surround sound that is constructed and arranged to perform the method described.
- the invention provides apparatus for outputting sound, said apparatus comprising: an array of output transducers; an input transducer; and a controller programmed: to cause said array to emit a test signal in the form of a directed sound beam into a room; to receive a signal from said input transducer that corresponds to at least one reflection of said sound beam test signal; and to determine a time of a maximum correlation between said emitted sound beam test signal and said received reflected sound beam test signal.
- the input transducer is typically an omnidirectional microphone for placement at the listening position.
- the controller is preferably arranged to calculate a path length for said directed sound beam.
- the controller is preferably arranged to sweep said directed sound beam through a range of angles and to determine the magnitude of the maximum correlation between said emitted sound beam test signal and said reflected sound beam test signal for each angle in said range.
- the apparatus preferably comprises a memory for storing, for each emission angle of said directed sound beam test signal, the time of said maximum correlation and preferably also the magnitude of said maximum correlation.
- the controller is preferably arranged to initially determine the position of said location in said room relative to said speaker system by receiving signals from said input transducer relating to direct, non-reflected, test signals.
- the controller is preferably arranged to select a sound beam direction for a centre channel, which beam direction points from said speaker system directly towards said location in said room.
- the controller is preferably arranged to select a sound beam direction for a centre channel, which beam direction is set to be the direction for which a maximum correlation exists between said emitted sound beam test signal and a directly received sound beam test signal by said input transducer.
- the controller is preferably arranged to select an angle for at least one of the surround sound channels based on the determined time of maximum correlation.
- the controller is preferably arranged to identify those angles for which a peak of maximum correlation exists.
- the controller is preferably arranged to select a sound beam direction for a left sound channel, which direction corresponds to the angle at which there is the largest peak magnitude of maximum correlation out of all of the interrogated angles to the left of the centre channel.
- the controller is preferably arranged to select a sound beam direction for a right sound channel, which direction corresponds to the angle at which there is the largest peak magnitude of maximum correlation out of all of the interrogated angles to the right of the centre channel.
- the controller is preferably arranged to switch the speaker system to "direct mode" if the peak magnitude of maximum correlation for the left sound channel is smaller than a predetermined percentage of the peak magnitude of maximum correlation for the centre channel.
- the controller is preferably arranged to switch the speaker system to "direct mode" if the peak magnitude of maximum correlation for the right sound channel is smaller than a predetermined percentage of the peak magnitude of maximum correlation for the centre channel.
- the controller is preferably arranged to select a sound beam direction for the left surround sound channel, which direction corresponds to an angle at which there is a peak magnitude of maximum correlation to the left of the centre channel and which direction has a smaller beam angle and larger time of maximum correlation than the selected direction for the left sound channel.
- the controller is preferably arranged to select a sound beam direction for the right surround sound channel, which direction corresponds to an angle at which there is a peak magnitude of maximum correlation to the right of the centre channel and which direction has a smaller beam angle and larger time of maximum correlation than the selected direction for the right sound channel.
- the apparatus preferably further comprises a candidate beam determining means for determining a plurality of candidate beam directions for the surround sound channels and assigning a quality coefficient to each candidate.
- the quality coefficient is preferably determined by multiplying the maximum correlation value for the beam angle by the beam path length.
- the candidate beam determining means can be arranged to disregard a candidate beam under certain circumstances. This is done preferably by setting the quality coefficient to be zero for that beam. Any of the following conditions (a) to (e) may be used to disregard a beam:
- the candidate beam has a larger angle and a larger time of maximum correlation than an already selected left or right sound channel
- the candidate beam has a smaller angle and a smaller time of maximum correlation than an already selected left or right sound channel
- the candidate beam has a time of maximum correlation within, say, 200 ⁇ s and an emission angle within, say, 15° of an already determined candidate beam;
- the candidate beam is directed in a direction where the path length for adjacent angles differs by a predetermined amount from the candidate beam path length.
- the controller is preferably arranged to select the candidate beam direction having the largest quality coefficient for a surround sound channel.
- the controller is preferably arranged to set a "corner mode" for the speaker system if the angle between the identified left sound channel and identified right sound channel is smaller than a predetermined angle. In such cases, the controller is arranged to emit the left sound channel and right sound channel in "direct mode".
- the controller can be arranged to direct the left sound channel and right sound channel in the same direction as the centre sound and to use a subset of the transducers in the array to emit the left sound channel and right sound channel.
- the controller in addition, in direct mode, can be arranged to set the focal length of the left sound channel and the right sound channel to be a fixed negative number, such as -0.5m.
- the controller is preferably arranged to calculate the correlation in real time, in the time between samples of the test signal.
- Figure 1 shows a front view of a Sound Projector according to an embodiment of the invention comprising a two-dimensional array of output transducers
- Figure 2 shows a front view of a Sound Projector according to another embodiment of the present invention, comprising a line array of output transducers
- Figure 3 schematically depicts exemplary signal paths for the Sound Projector of the present invention
- Figure 4 shows a plan view of a room environment and three exemplary sound beams produced by the Sound Projector of the present invention
- Figure 5 shows a sonogram useful in explaining the present invention
- Figure 6 shows a plan view of a room environment and five exemplary sound beams that create a full Surround Sound effect
- Figure 7 is a graph showing the maximum magnitude of the correlation between emitted and received test signals together with the corresponding time delay between emitting and receiving the test signals, for the room environment of Figure 6;
- Figure 8 shows a plan view of a room environment and five exemplary sound beams that create a full Surround Sound effect
- Figure 9 is a graph showing the maximum magnitude of the correlation between emitted and received test signals together with the corresponding time delay between emitting and receiving the test signals, for the room environment of Figure 8;
- Figure 10 shows a plan view of a room environment and three exemplary sound beams when the Sound Projector is located in a corner
- Figure 11 is a graph showing the maximum magnitude of the correlation between emitted and received test signals together with the corresponding time delay between emitting and receiving the test signals, for the room environment of Figure 10;
- Figure 12 is a flow diagram showing how the beam directions for the left and right sound channels can be selected in accordance with the invention
- Figure 13 is a flow diagram showing how quality coefficients can be determined for particular candidate beam directions in accordance with the present invention.
- FIG. 1 shows a typical Sound Projector system to which the present invention may be applied.
- the system 100 comprises a front panel 101 that is generally planar and rectangular and upon which 254 individual acoustic output transducers 111 are mounted in an array 110.
- a front panel 101 that is generally planar and rectangular and upon which 254 individual acoustic output transducers 111 are mounted in an array 110.
- differently delayed versions of the same signal may be applied to the output transducers, so as to produce a sound beam that travels in a particular predetermined direction relative to the speaker system.
- the output transducers 111 extend both horizontally and vertically. Accordingly, this Sound Projector has a capability to steer sound beams in both the left/right and the up/down directions.
- Figure 2 shows an alternative embodiment in which the casing 101 comprises 16 output transducers 111 arranged in a horizontal line.
- This embodiment is capable of directing beams only in the left/right direction. Although more limited, this construction is nevertheless well adapted for providing surround sound.
- transducers used in the Sound Projector are not critical, provided enough are present to ensure that the sound beams can be adequately directed in useful directions.
- the transducers 111 are most preferably arranged such that sound beams can be directed at least in the left/right direction. However, embodiments (such as that shown in Figure 1) where sound beams are also capable of being directed in the up/down direction are included within the scope of the invention.
- FIG. 3 shows a simplified block diagram of the signal path through the Sound Projector.
- An input audio signal 190 is fed to a decoder 195.
- the audio signal may be a Dolby Digital 5.1 audio signal from a DVD, television decoder, Blu-Ray disk or any other type of audio signal source.
- the decoder 195 may be any type of conventional decoder, such as a Dolby 5.1 decoder.
- the decoder 195 outputs in this embodiment five separate audio signals, 200, 202, 204, 206, and 208.
- the audio signals each correspond to a respective channel in a Dolby 5.1 system, namely left 200, right 202, centre 204, left surround 206 and right surround 208.
- the subwoofer channel is not shown although this channel may usually also be decoded by the decoder 195 and sent to a conventional subwoofer for output.
- Each channel signal 200, 202, 204, 206, 208 is provided to a separate delay element 210 that imposes a channel-specific delay on the signal. This channel-specific delay will be related to the distance that sound waves for that channel must travel before they reach the user. In general, longer delays will be provided for channels that travel a shorter distance through the air. This helps to ensure that each channel reaches the listening position (where the user is) substantially simultaneously.
- Each delayed channel is then preferably fed to a replication and delay means 220.
- the replication and delay means 220 replicates (that is to say copies) the input signal a certain number of times (five times in Figure 3) and provides a selected delay to each replica input signal.
- the delay is selected in accordance with the direction in which the sound beam for that channel is to be directed. In this embodiment, the delay is also selected bearing in mind the position of the particular output transducers 250.
- each replication and delay means 220 produces five individually delayed replicas, one for each of the output transducers 250. Accordingly, in this embodiment, all of the output transducers 250 are used to output each of the channels 200, 202, 204, 206 and 208. hi some other embodiments, it is possible to use a subset of the output transducers 250 to output certain of the channels and not all transducers 250 need to be used to output all of the channels. In general, there will be more than five output transducers and in such cases, each replication and delay means 220 can have more than five outputs.
- Each output transducer 250 has associated with it an amplifier 240 and a summing means 230 which receives a plurality of delayed replicas from the replication and delay means 220.
- each summing means 230 receives a replica from each of the replication and delay means 220.
- each summing means 230 in this embodiment outputs the sum of five different signals.
- the summing means 230 for a particular output transducer 250 will receive a signal from a replication and delay means 220 in accordance with the particular choice of transducers to be used to output a particular channel.
- Figure 4 shows a plan view room of a room 300 in which a Sound Projector 100 has been placed. The position marked "X" is the listening position for the user. In an ideal configuration, the user will receive five channels, each corresponding to one of the surround sound channels: left, right, centre, surround left and surround right. Ideally, the sound will reach the listener from the intended direction for each channel.
- the arrow marked Bl in Figure 4 shows the centre channel path.
- This channel can be created by energising each transducer in the array with a signal corresponding to the centre channel (with no relative delay between any of the transducers).
- the replication and delay means 220 for the centre channel merely replicates the signal and does not apply any relative delay amongst the signals for each of the output transducers 250.
- a sound beam can be produced and the sound beam can be focussed, as described in the above mentioned prior art documents. When a focussed sound beam is output, there will be a relative delay amongst the transducers for the centre channel.
- transducers near the middle of the array will be delayed more than transducers near the edge to provide a focussed beam.
- a negative focus can be provided, which creates a more widely dispersed sound beam.
- the arrow marked B2 in Figure 4 shows a typical path for a right surround sound channel. As can be seen, this sound beam is reflected from the right sidewall and the back wall so as to arrive at the listening position from behind. Sound for this channel travels the longest distance.
- the arrow referenced B3 in Figure 4 shows a typical sound path for the left channel. As can be seen, the sound beam is reflected once from the left sidewall to arrive at the listening position substantially from a left-hand direction.
- Figure 4 shows only 3 sound beams, in most practical embodiments of the Sound Projector there will be five sound beams. As well as the left channel sound beam B3, there will also be a mirror-image right channel sound beam and as well as the right surround sound beam B2, there will be a mirror image left surround sound beam. Accordingly, a full surround sound experience can be provided for the user using a single device 100.
- the delay means 210 will typically apply the longest delay to the centre channel 200 (represented by Bl in Figure 4) because this channel travels the shortest distance.
- a medium-length delay can be applied to the left and right channels 202, 204 (represented by B3 in Figure 4) because these channels travel the second shortest distance.
- No delay (or perhaps a very small delay) can be applied to the surround sound channels 206, 208 (represented by B2 in Figure 4) because these channels usually travel the longest distance.
- the delays are preferably chosen such that temporally corresponding portions of the various channels reach the listening position "X" at the same time.
- an appropriate delay may be applied to the video signal to ensure that the video displayed also corresponds temporally with the sound reaching the user at any point in time.
- the left and right channels are directed through a larger angle (as measured from a line perpendicularly extending from the front of the Sound Projector) than the surround sound channels.
- the path length for the left and right channels is in Figure 4 shorter than the path length for the surround sound channels. Ln general, an optimum set-up for the system will be when the left and right channels each have a shorter path length and wider angle than the surround sound channels.
- An optional first step for the set-up system includes locating the position of the microphone in the room relative to the Sound Projector.
- the user may simply input the location of the microphone to the Sound Projector, perhaps by inputting the distance from the Sound Projector to the microphone and perhaps also the angle of the microphone relative to the Sound Projector.
- the Sound Projector may locate the microphone by triangulation, as disclosed in WO 01/23104.
- at least three of the output transducers are used to output three test signals and the time taken for the test signals to arrive at the microphone from the Sound Projector is measured. The recorded times can be used to calculate the distance between each output transducer and the microphone and, from this, the position of the microphone relative to the Sound Projector can be determined.
- the test signals may be output simultaneously, in which case it is preferable that they are distinguishable from one another (for example by being located in different frequency ranges or by using different pseudorandom noise or MLS codes). Alternatively, the test signals can be emitted sequentially.
- the following method is used to determine the location of the microphone, which is preferably omnidirectional.
- the Sound Projector is preferably programmed to first output a sound beam in the straight- ahead direction (i.e. perpendicular to the major plane of the Sound Projector) and with a focal length of -0.5 m. Setting the focal length to a negative value gives a broadly dispersing beam.
- the signal output preferably corresponds to a maximum length sequence (MLS) signal.
- the maximum length sequence test signal has a length of N samples. Preferably, 1023 samples are used. This gives a good balance between being able to well correlate the MLS test signal and performing the method in a short period of time.
- the sound received at the microphone is sampled, preferably at the same sampling rate as the MLS sequence, and correlated with the MLS signal.
- the correlation is preferably performed in real time every sample period between the last N samples received at the microphone and the MLS test signal. Preferably, such correlation is carried out in real time every sample period, until enough time has passed for sound that has travelled a maximum expected direct microphone-Sound Projector distance to reach the microphone.
- This maximum direct microphone distance can be set to, say, 6 m or 7 m. It can be user definable if desired. For example, when the maximum length sequence has a length of 1023 bits and when the sample rate is 48 kHz, and assuming a speed of sound of 340 m/s, a distance of 7 m corresponds to a time for sound to travel that distance of about 20.6 ms which corresponds to 988 sample periods. Accordingly, the sampled signal from the microphone is correlated to the original MLS test signal at every sample period between 1023 and 2011 samples. After this, 988 values of the correlation, each corresponding to a slightly different moment in time will be available.
- the value of this correlation that is highest is identified and the system can thereby determine that this value corresponds to the reception by the microphone of the MLS signal as it travels directly from the Sound Projector to the microphone.
- reflected signals will have a lower correlation due to their increased attenuation relative to the direct signal.
- the point in time of this maximum correlation can be determined and used to determine the distance (direct path length) from the Sound Projector to the microphone. For example, if the maximum correlation occurs after 1500 sample periods, then this means that the sound has taken 477 sample periods to travel from the Sound Projector to the microphone. This corresponds to a time delay of 9.9 ms, which in turn corresponds to a distance of 3.37 m. Accordingly, under such a circumstance, the direct distance between the Sound Projector and the microphone would be determined as 3.37m.
- the measured time delay i.e. the time between emitting the MLS and detecting the maximum correlation at the microphone
- This system delay can be quantified and represented as a constant time offset. This time offset can then be subtracted from the measured delay values to give a more accurate time of flight measurement for the audio signal.
- the relative angle between the microphone and the Sound Projector can then be determined, hi this document, all angles are expressed relative to a line extending perpendicularly out from the centre of the Sound Projector. This perpendicular line is denoted by an angle of 0°. Sound beams that travel to right of this line (as viewed by the user) are denoted by positive angles and sound beams that travel to the left of this line (as viewed by the user) are given negative angles. A full sweep of the beam through the half space in front of the Sound Projector from left to right will therefore be denoted as a sweep from -90° to +90°.
- the Sound Projector is arranged to repetitively output a sound beam in a sweep from the left to the right.
- This sweep may encompass the full half plane of -90° to +90°, but preferably covers a smaller range of angles than this because it is unlikely that the microphone will be located at extreme angles relative to the Sound Projector.
- the sweep is from -76° to +76°.
- the focal length of the sound beam used is set to be the same as the distance between the microphone and the Sound Projector determined in the first stage. Accordingly, when a beam having an angle that directly points to the microphone is output, it will be focussed on the microphone and the microphone will receive an intense signal.
- an MLS test signal having N samples is output as a sound beam directed at a desired angle (for example the first beam will be output at -76°).
- the MLS test signal is followed by silence.
- the signal received at the microphone is correlated with the original MLS signal every sample period, until the maximum expected direct microphone distance is reached.
- This maximum expected direct microphone distance can, once again, be set to 6 m or 7 m, although it is preferably related to the distance determined in the first stage, for example by multiplying that distance by 1.5 to give a margin of error.
- an expected microphone distance of no more than 5 m (1.5 x 3.37) may be used in the second stage. This corresponds to 706 sample periods. Accordingly, the correlation between the emitted sound beam test signal and the received sound beam test signal will be performed for each of 706 sample periods following the emission of the last bit of the MLS.
- the time of maximum correlation is determined. This can be determined as a number of samples, as a period of time in seconds or as a sound beam path length.
- the magnitude of the maximum correlation is also stored. This is repeated for each angle within the swept range of interest. Typically, the angles are incremented in one degree steps from -76° to +76°.
- the speaker system will be in possession of the magnitude of maximum correlation and the corresponding time (or, equivalently, path length or number of samples) for each angle within the swept range of angles. The system can then determine that the angle corresponding to the overall largest magnitude of maximum correlation is the angle of the microphone relative to the Sound Projector. As a check, the distance between the Sound Projector and microphone is preferably calculated again at this point and compared with that obtained or input previously.
- the next stage of the set-up procedure can be performed.
- a sweep is once again performed through all angles of interest, although this time with a view to receiving only reflections of the sound beam.
- a maximum expected travelling distance for the sound beam is set to be much greater, typically 3, 4, 5, 6 or 7 times greater than the distance from the Sound Projector to the microphone.
- the system may assume a sufficiently high fixed number, for example 24 m, for the expected travelling distance of sound beams from the Sound Projector to the microphone, which will in general travel via a reflection from one or more of the walls/ceiling/floor etc.
- a beam is output at each angle within the range of angles, for example -76° to +76°.
- an MLS test signal is used and correlated with the received microphone signal every sample period. The correlation continues for enough time to account for all signal paths of interest, for example enough time to account for 24 m of sound beam propagation at each angle. This corresponds to approximately 70 ms or 3300 sample periods (at a sampling frequency of 48kHz) at each angle.
- the focal length of the beam is set to be 1-2 times the distance from the Sound Projector to the microphone, for example 1.3 to 1.8 times, more preferably 1.5 times this distance. Accordingly, the microphone will receive maximum power when receiving a reflected signal that travels a distance roughly corresponding to the focal length.
- alias beams will in general be created (otherwise known as side lobes). These alias beams are generally stronger for larger deflection angles than smaller deflection angles. In some cases, the side lobe energy received at the microphone direct from the Sound Projector can significantly exceed the energy of the useful reflected beam incident on the microphone.
- a time mask can be applied to signals that are directed in a range outside of a small 20° window around the centre channel. No time mask will be applied for signals directed straight at the microphone and for signals directed 10° either side of the microphone. However, for signals directed at angles outside of this range, the correlation calculation is preferably started only after a certain time has passed. This time will preferably be chosen such that it exceeds the time that it would take for signals to travel directly from the Sound Projector to the microphone. A margin of error can be implemented to ensure that no direct signals at all are received by the microphone when beams are directed outside of the central window. Accordingly, if the microphone is a certain time has passed. This time will preferably be chosen such that it exceeds the time that it would take for signals to travel directly from the Sound Projector to the microphone. A margin of error can be implemented to ensure that no direct signals at all are received by the microphone when beams are directed outside of the central window. Accordingly, if the microphone is a certain time has passed. This time will preferably be chosen such that it exceeds the time that it
- the system can be programmed to disregard all signals having a travelling distance of 4 m or less that are directed outside the 20° zone pointing at the microphone. This ensures that alias signals travelling along a direct path from the Sound Projector to the microphone are not taken into account.
- This time mask can be represented as:-
- ⁇ c is the angle of the microphone
- Cl 0 is the distance to the microphone in metres.
- m( ⁇ ) is expressed in metres. So for angles outside of a small 20° window around the centre channel, correlations are only performed for distances 0.7 m greater than the direct path length.
- Correlations are performed every sample period between the emitted MLS test signal and the reflected test signal received at the microphone. This is repeated for every angle within the swept range.
- the above processing is preferably all carried out in a real time at a sample rate of 48 KHz. This can be achieved by using a typical low-cost audio signal processing device.
- Figure 5 shows a sonogram summarising the signals detectable at the microphone.
- the swept angles are shown on the x-axis, the distance (path length) is shown on the y-axis and the value of the correlation (whereby a correlation of 0 dB corresponds to a perfect correlation) is denoted by the intensity of shading in the Figure.
- Correlations that are only partial have negative values measured in dB in Figure 5.
- a correlation value of "1 " corresponds to 0 dB in Figure 5.
- path lengths greater than 2.8 m and smaller than about 11.3 m are shown. Several peaks of correlation can be identified as dark points on the sonogram.
- a well-correlated signal is received by the microphone at an angle of 0°, with path length of approximately 3.2 m.
- This is denoted on the sonogram by the reference numeral 10.
- This is the signal corresponding to the direct travel of the sound beam from the speaker system to the microphone.
- Reference numeral 12 shows a slightly longer path length and this peak of correlation is created by sound beams reflecting once from the floor before reaching the microphone.
- Reference numeral 14 shows a longer path length still and this relates to sound beams that have reflected once from the ceiling before reaching the microphone.
- Reference numeral 20 is a yet longer path length and this could relate to a signal that has reflected from both the floor and the ceiling or from some piece of furniture such as the seat.
- Numeral 30 identifies the correlation obtained from a sound beam that has travelled to the back of the room and directly bounced back from the rear wall.
- Correlations 32 and 34 are at angles of approximately ⁇ 30° and have a path length of 9.2 m, indicating that these correlations are for sound beams that have bounced once from the left/right wall and once on the rear wall (see the beam B2 in Figure 4).
- Correlations at positions 36 and 38 are at wide angles of ⁇ 65° and are likely to be triple bounces from the left and right walls.
- the following method is utilised to, preferably automatically, place the 5 audio channels along particular beam paths that will provide an optimum surround sound effect in a robust manner.
- Figures 6 and 7 show an example that will be explained in greater detail.
- Figure 6 shows a plan view of a typical listening room 300, in which the microphone is placed on a seat located 4.09 m from the front wall of the room, and where the sound projector 100 is placed centrally on the front wall 302.
- the desired sound beams to achieve a surround sound effect are illustrated in Figure 6.
- Figure 7 shows two graphs.
- the darker line illustrates the magnitude of maximum correlation for each angle in the range -76° to 76°.
- the grey line shows the time that that maximum correlation took place. This time is directly related to the path length that the sound beam takes.
- the Sound Projector stores in memory a value for the maximum correlation and a value for the corresponding time in respect of each angle in the swept range. This limits the amount of data that needs to be stored to only two data words per angle. If the entire sonogram of Figure 5 were stored, this would involve storing over 500,000 data words.
- the Micronas MSP-M device does not have enough memory to store the entire sonogram whereas it does have enough memory to store the data of Figure 7.
- the data of Figure 7 i.e. the smoothed magnitude of maximum correlation data for each angle and the time data (or path length data) for each angle
- the darker line in Figure 7 is related to the sonogram in Figure 5.
- the maximum value of correlation (wherever it occurs) is identified and stored.
- the distance (or, equivalently, the time) at which this maximum value of correlation occurs is also stored.
- a graph of the maximum correlation with respect to angle can then be drawn (see the darker line in Figure 7).
- a graph of the distance that the sound beam travels to achieve this maximum correlation can also be drawn (see the grey line in Figure 7).
- the data of Figures 7, 9 and 11 have been normalised but this is not necessary to carry out the invention.
- the time data (or equivalently path length data) is not usually smoothed.
- the speaker system memory has stored therein two items of data associated with each angle in the swept range. This can be stored as two vectors, one for the correlation data and one for the time data. As explained above, this data can be obtained in real time in a matter of seconds or minutes using the Micronas MSP platform.
- the data can then be manipulated in a post-processing procedure to determine where to place the sound channels.
- an algorithm is used to choose the beam angles for each of the audio channels. Parts of the algorithm are summarised in Figures 12 and 13. The algorithm needs to be robust to account for cases where the Sound Projector is placed in a poor listening environment or in a position that makes surround sound reproduction difficult (such as in the comer of a room).
- the first step in the procedure is to place the centre channel. This will usually be placed along the direct path from the speaker system to the microphone and will correspond to the angle for which the maximum correlation is the largest across the whole range of swept angles, hi Figures 6 and 7, the centre channel is directed at approximately +2.5°.
- the next stage is to identify the best beam candidates lying to the left and to the right of the centre channel respectively.
- all the potential candidate beams are determined. This is achieved by determining all of the angles for which a peak of maximum correlation exists. A peak is defined as existing at an angle where the maximum correlation value at that angle is larger than the correlation values lying at the two neighbouring angles.
- the already determined centre channel is not considered to be a "peak" in this analysis and, preferably, any peak located within 12° of the centre channel are not included.
- all of the peaks present are identified although it may save on processing if a finite number of the largest peaks are identified, for example 6 peaks on either side of the centre channel, hi Figure 7, six peaks have been identified each side of the centre channel.
- These candidate beams are numbered 50-61 respectively as shown in Figure 7.
- the candidate beams are then ranked in order of maximum correlation value. Two lists of candidate beams are thus obtained, one for the left and one for the right, hi the example of Figure 7, the ranking is as follows for the left side:
- the system starts by looking at the angle subtended between the two largest candidate beams.
- Candidate beam 51 lies at an angle of -45° whereas candidate beam 59 lies at an angle of 47°.
- the angle subtended between the two beams is thus 92°. It is then determined whether this subtended angle is greater than or less than a predetermined threshold angle. Good results have been found when this predetermined threshold angle is set to be 60°, although other angles such as 50° or 70° could be used. If the angle between the two candidate beams having the largest peak correlation is smaller than the threshold (which is not the case in Figures 7 or 9 but is the case in Figure 11), then the system determines that the Sound Projector is probably located in the corner of a room. A flag is then set to indicate that the Sound Projector is to be operated in "corner mode".
- This identification of whether the Sound Projector is located in a corner is a very useful function, because otherwise it would be necessary to request that the user indicates whether the Sound Projector is in the corner. Accordingly, this facility allows the set-up procedure to be conducted with a high degree of autonomy.
- the Sound Projector would at this stage allocate the left sound channel (referenced B 3 in Figure 4) to the candidate beam 51 and allocate the right sound channel to the candidate beam 59.
- the candidate beam has a maximum correlation value that is at least 10% of the correlation value for the direct sound beam from the Sound Projector to the microphone; and (b) the candidate beam has a larger angle than the already selected beam for the left or right sound channel; and (c) the candidate beam has a maximum correlation value that is within 3 db of the maximum correlation value for the already selected left or right sound channel; and (d) the candidate beam has a shorter path length than the beam already selected for the left or right channel
- this checking procedure is to ensure that, in the case when a surround sound beam is the strongest, this is not accidentally chosen for the left or right sound channels.
- the left or right sound channels will have a greater angle than the surround sound channels and will have comparable or greater strengths than the surround sound channels. Also, they will in general have a shorter path length than the surround sound channels.
- the four questions above ensure that these conditions are met.
- the quality coefficient for a candidate beam is made to be equal to the value of the correlation multiplied by the path length for that beam.
- the path length is limited at two times the path length chosen for the left or right channel beams, respectively.
- the left channel beam has already been selected as candidate beam 51.
- the path length for this candidate is shown in Figure 7 as having a relative magnitude of 0.31. This path length is multiplied by two to obtain the maximum path length (0.62) that will be used when calculating the quality coefficient for the other beams.
- the candidate beam 66 was chosen to be the left channel beam, it can be seen that this has a path length of 0.29 and so the maximum path length used in calculating the quality coefficient would be 0.58. Thus, when calculating the quality coefficient for the candidate beam 62, a path length of 0.58 would be used, even though this beam has an actual path length of 0.72.
- the quality coefficient for each of the beams can be calculated. The following values are obtained for each of the beams on the left side.
- Figure 13 shows a series of conditions that the candidate beam must meet in order to be considered for the surround sound channel. If the candidate beam does not meet each of the conditions then it is disregarded. The candidate beam is preferably disregarded in practice by setting its quality coefficient to be zero. In the first question of Figure 13, it is determined whether the maximum value of correlation for that candidate beam is smaller than some percentage of the channel centre correlation. In Figure 7, the centre correlation is 1.
- a threshold value of 7.5% can be selected, although other thresholds may be used, such as 5%, 10%, 15% etc.
- the correlation is at least 0.075.
- a normalised threshold may instead be used, for example by determining (C - C min )/( C centre - C min ) ⁇ 0.05 where C is the amplitude of the maximum correlation for the candidate beam, C centre is the amplitude of the maximum correlation for the centre channel beam and C min is the smallest maximum correlation from all of the angles.
- both the angle and path length are greater than the angle and path length for the respective left or right beam. If the angle and path length are both greater, then this can indicate a double bounce off the left or right walls or even a triple or quadruple bounce. It can be seen from Figure 7 that candidate beams 50 and 61 have a greater angle and longer path length than the left channel beams 51 and 59 respectively. Accordingly, the quality coefficient for the candidate beams 50 and 61 is set to be zero.
- both the angle and the path length is smaller than the left or right sound channels, respectively. If the angle is smaller and the path length is smaller, then this could indicate that the beam is taking a more direct route, for example by being bounced from the floor or ceiling before reaching the microphone. None of the beams of Figure 7 are disregarded due to this test.
- the candidate beam angle is within a certain angle range from a previously considered candidate beam and it is determined whether the path length is within a certain path length range from that other candidate beam. If two candidate beams are within 15° of one another and have a path length within ten samples of one another, then it is determined that the two candidates are actually part of one and the same beam and the quality coefficient for the candidate having the lower correlation is set to be zero.
- candidate beam 53 is within 15° of already considered candidate beam 52 and has a path length within ten samples. Accordingly, the quality coefficient for candidate beam 53 is set to zero.
- the threshold values of 15° and ten samples may of course be varied in accordance with the circumstances.
- the threshold value for angle is selected from the range 5° to 25°.
- it may be measured in time (seconds) or distance (metres). Samples, time and distance are all interchangeable in this respect.
- the threshold value is selected such as to detect time differences between path lengths in the range 100-500 ⁇ s.
- the candidate beams are "robust". If the candidate beams lie at an angle where there are great changes in path length for angles very close to the candidate beams, then this tends to indicate that the candidate beams are quite unstable, perhaps because they are directed at some piece of furniture having an unusual shape. It is firstly checked whether the path length for the angles either side of the candidate beam is within 20% of the candidate beam path length. So, for example, taking candidate beam 54 in Figure 7, it can be seen that this candidate beam lies at an angle of -27°. The path length here is 0.49. This path length is compared to the path length at -26° and -28°. In this example, these path lengths are also 0.49. When the adjacent path lengths are within a certain threshold of the candidate beam path length, for example within 20%, then it is determined that the candidate beam is a good candidate. Candidate beam 54 in Figure 7 passes this test.
- this test determines whether the path length for each of the two angles either side of the candidate beam is within ten samples of the candidate path length. So, if the candidate beam lies at +40°, the path lengths at +38°, +39°, +41 ° and +42° are checked. If all four path lengths are within 10 samples of the candidate beam path length, the candidate beam passes the test. Accordingly, this test can be seen as determining whether the path length fulfils either of the following two conditions: (a) the path length for the two angles that are each directly either side of the candidate beam is within 20% of the candidate beam path length; or
- the candidate beam is not disregarded due to this test. If both conditions are not fulfilled, then the candidate beam fails the test and the quality coefficient for that candidate beam is set to zero.
- the left and right surround sound channels are selected to be directed at an angle corresponding to the candidate beam having the highest quality coefficient for that side, hi Figure 7, the left surround sound channel will accordingly be directed at -27° (candidate beam 54) and the right surround sound channel will be directed at 24° (candidate beam 56).
- the surround sound channels can be steered at the same angle as the respective left or right sound channels. Accordingly, if on the left side there are no candidate beams having a non-zero quality coefficient, the left surround sound channel will be steered in the same direction as the left sound channel.
- a final check is done once the left sound and left surround sound channels have been selected. If the angle of the left surround sound channel is larger than the angle of the left sound channel, the two channels are swapped. Similarly, if the angle of the right surround sound channel is larger than the angle of the right sound channel, these two channels are swapped. This ensures that the left and right sound channels have a greater angle than the left and right surround sound channels, respectively.
- Figures 8 and 9 show another example where the sound projector is not centrally located in the listening room, hi this case, as shown in Figure 8, sound beams directed to the right generally have a longer path length than sound beams directed to the left. As shown in Figure 8, this also tends to mean that the channels directed to the right will have wider angles than the channels directed to the left.
- candidate beams 66 and 71 are initially selected as the left sound channel and right sound channel respectively. No other beam meets the requirements of Figure 12 and so these beams remain the left and right sound channels even after the process of Figure 12 has been carried out.
- Candidate beams 62, 63 and 64 have a longer path length and wider angle than candidate beam 66 and so are each disregarded. Similarly, candidate beams 72 and 73 are also disregarded for the same reason. These candidate beams likely relate to double or triple bounces from the left and right sidewalls.
- Candidate beam 65 is disregarded as having the same path length and being within 15° of candidate beam 66.
- Candidate beam 67 is the only remaining beam on the left-hand side having a non zero quality coefficient and thus is selected for the left surround channel.
- the three candidate beams 68, 69, 70 each have significantly different path lengths to one another, and the path length is in all three cases greater than the path length for the selected right channel. Furthermore, the three beams are each directed at an angle that is quite robust, because the path length for angles either side of the candidate beams are quite similar. Accordingly, the candidate beam having the largest quality coefficient will be selected, which in this case is beam 68.
- Figures 10 and 11 show an example where the Sound Projector is located in a corner of a room.
- the largest peaks of maximum correlation either side of the centre beam are at -9° (beam 79) and 25 ° (beam 80). Accordingly, the angle between these two candidate beams (34°) is smaller than the predetermined threshold (60°) and thus the Sound Projector determines that it has been located in the corner of the room. It accordingly automatically switches its mode to "corner mode".
- the left and right sound channels are directed in the same direction as the centre channel, preferably using a subset of the transducers located on the left-side or right-side of the Sound Projector. This is called "direct mode".
- direct mode For example, in the case where the Sound Projector is configured as shown in Figure 2, the leftmost six transducers can be used to output the left sound channel and the rightmost six transducers can be used to output the right sound channel.
- the method of Figure 13 is then be used to determine the candidate beams for the surround sound channels.
- the result of this method is that candidate beams 79 and 80 are used for the left surround sound and right surround sound channel respectively.
- the left and right sound channels may also be output directly, even when it is determined that the Sound Projector is not located in the corner of the room, hi this so-called "direct mode", the left and right sound channels are output directly by subsets of the transducers, in the manner shown in Figure 10.
- This "direct mode” will generally be used when all of the beam candidates are very weak (for example when all of the beam candidates have a maximum correlation magnitude smaller than 10% of the correlation magnitude for the centre channel). In such cases, it is better to use the reflected beams for the surround sound channels and operate the (more important) left and right sound channels in "direct mode".
- the focal length for each of the left sound channel and the right sound channel is set to be -0.5 m (to create a diffuse sound field) and the steering angle is set equal to the centre channel angle.
- different subsets of transducers are used for the left sound channel and the right sound channel.
- the six left-most transducers can be used for the left sound channel and the six right-most transducers can be used for the right sound channel.
- direct mode is used when the system identifies that it has been placed in the comer of the room or when the correlation for all of the candidate beams is very small. In direct mode, the candidate beam nevertheless having the largest correlation is chosen for the two surround sound channels and the focal length of each is set to be one half of the corresponding path length.
- the centre channel is set to have a focal length of -0.5 m and the other channels are set to have a focal length corresponding to one half of their path length. This ensures a reasonable degree of dispersion at the listening position.
- the system can be implemented using a Micronas MSP-M processor working on a sample by sample basis.
- a sampling rate of 48 kHz can be used because this is consistent with the usual sample rate that is used for audio reproduction. This gives a sample period of roughly 20.5 ms.
- the processing relating to the correlation calculation is carried out during this sample period (i.e. in the time between samples). Also during each sample period, the necessary computation for forming a sound beam is carried out for the next output audio sample and the signals are generated to drive the transducers of the array.
- the Micronas platform has a IK analysis buffer and this can be used to receive the audio sample from the microphone in real time to allow the correlation to be carried out.
- the system is advantageously configured as a state machine working on a sample basis although other configurations, for example by working on a block basis, are possible.
- the state machine routines for the microphone based audio set-up (MBAS) system are included below.
- MBAS_SteerBeam Calculate the beam forming coefficients for the given angle and focal length.
- Two data vectors, correlation amplitudes and path lengths, are ready for post processing.
- Random bit sequences can be generated by the recurrence relation:
- the bit sequence is generated by taking the successive values of s n Sc I .
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Stereophonic System (AREA)
- Circuit For Audible Band Transducer (AREA)
- Push-Button Switches (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
- Geophysics And Detection Of Objects (AREA)
- Adhesives Or Adhesive Processes (AREA)
- Amplifiers (AREA)
- Packaging Of Special Articles (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB0721313.5A GB0721313D0 (en) | 2007-10-31 | 2007-10-31 | Microphone based auto set-up |
PCT/GB2008/003707 WO2009056858A2 (fr) | 2007-10-31 | 2008-10-31 | Installation de projecteur sonore |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2208369A2 true EP2208369A2 (fr) | 2010-07-21 |
EP2208369B1 EP2208369B1 (fr) | 2011-05-04 |
Family
ID=38834548
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP08844786A Active EP2208369B1 (fr) | 2007-10-31 | 2008-10-31 | Installation de projecteur sonore |
Country Status (5)
Country | Link |
---|---|
EP (1) | EP2208369B1 (fr) |
AT (1) | ATE508594T1 (fr) |
DE (1) | DE602008006761D1 (fr) |
GB (1) | GB0721313D0 (fr) |
WO (1) | WO2009056858A2 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023274499A1 (fr) * | 2021-06-29 | 2023-01-05 | Huawei Technologies Co., Ltd. | Système et procédé de reproduction sonore |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4609502B2 (ja) * | 2008-02-27 | 2011-01-12 | ヤマハ株式会社 | サラウンド出力装置およびプログラム |
RU2576343C2 (ru) | 2010-05-20 | 2016-02-27 | Конинклейке Филипс Электроникс Н.В. | Оценка расстояния с использованием звуковых сигналов |
JP6186436B2 (ja) | 2012-08-31 | 2017-08-23 | ドルビー ラボラトリーズ ライセンシング コーポレイション | 個々に指定可能なドライバへの上方混合されたコンテンツの反射されたおよび直接的なレンダリング |
CN114639210B (zh) * | 2022-01-27 | 2024-02-23 | 杭州华橙软件技术有限公司 | 声波警告方法、系统、装置及存储介质 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB0301093D0 (en) * | 2003-01-17 | 2003-02-19 | 1 Ltd | Set-up method for array-type sound systems |
JP4127248B2 (ja) * | 2004-06-23 | 2008-07-30 | ヤマハ株式会社 | スピーカアレイ装置及びスピーカアレイ装置の音声ビーム設定方法 |
-
2007
- 2007-10-31 GB GBGB0721313.5A patent/GB0721313D0/en not_active Ceased
-
2008
- 2008-10-31 AT AT08844786T patent/ATE508594T1/de not_active IP Right Cessation
- 2008-10-31 DE DE602008006761T patent/DE602008006761D1/de active Active
- 2008-10-31 WO PCT/GB2008/003707 patent/WO2009056858A2/fr active Application Filing
- 2008-10-31 EP EP08844786A patent/EP2208369B1/fr active Active
Non-Patent Citations (1)
Title |
---|
See references of WO2009056858A2 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023274499A1 (fr) * | 2021-06-29 | 2023-01-05 | Huawei Technologies Co., Ltd. | Système et procédé de reproduction sonore |
Also Published As
Publication number | Publication date |
---|---|
WO2009056858A2 (fr) | 2009-05-07 |
GB0721313D0 (en) | 2007-12-12 |
ATE508594T1 (de) | 2011-05-15 |
EP2208369B1 (fr) | 2011-05-04 |
WO2009056858A3 (fr) | 2010-04-15 |
DE602008006761D1 (de) | 2011-06-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1584217B1 (fr) | Procede de mise en place pour un systeme de son de type reseau | |
JP5894979B2 (ja) | 音声信号を使用した距離推定 | |
US9107021B2 (en) | Audio spatialization using reflective room model | |
JP5728094B2 (ja) | 到来方向推定から幾何学的な情報の抽出による音取得 | |
US8325952B2 (en) | Directional speaker system and automatic set-up method thereof | |
US20090034756A1 (en) | System and method for extracting acoustic signals from signals emitted by a plurality of sources | |
EP1522868A1 (fr) | Système et méthode pour déterminer la direction d'une source de son | |
JP4175420B2 (ja) | スピーカアレイ装置 | |
EP2208369B1 (fr) | Installation de projecteur sonore | |
JP5915206B2 (ja) | 音場制御装置 | |
EP1651007A2 (fr) | Appareil de reproduction sonore multicanaux et procédé de réglage du son multicanaux | |
JP5543106B2 (ja) | 空間オーディオ信号再生装置及び空間オーディオ信号再生方法 | |
JP2006340303A (ja) | スピーカアレイ装置 | |
Calamia et al. | Diffraction culling for virtual-acoustic simulations | |
EP3016413A1 (fr) | Système et procédé pour détecter des caractéristiques acoustiques d'un système de haut-parleurs dans un environnement limité | |
Canclini et al. | A robust geometric approach to room compensation for sound field rendering | |
Roper | A room acoustics measurement system using non-invasive microphone arrays | |
JP2011233997A (ja) | オーディオ再生装置 | |
Islam | Estimating Source and Surface Locations in a Room Using Ambisonics | |
Embrechts et al. | Real-time auralization of industrial noise | |
Seo et al. | Impact of early reflection on real-time estimation of direction of arrival in different building environments | |
JP6003409B2 (ja) | 放音装置および音響ビームの放音方向の調整方法 | |
SARTI et al. | A Robust Geometric Approach to Room Compensation for Sound Field Rendering | |
Lesoinne et al. | 3D real-time auralization with separate rendering of direct sound, reflections and directional late reverberation | |
Peterson | Multiple source localization for real-world systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20100510 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA MK RS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 3/12 20060101ALN20101027BHEP Ipc: H04S 7/00 20060101AFI20101027BHEP |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 602008006761 Country of ref document: DE Date of ref document: 20110616 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602008006761 Country of ref document: DE Effective date: 20110616 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110905 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110804 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110904 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110815 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110805 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20120207 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111031 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20111123 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602008006761 Country of ref document: DE Effective date: 20120207 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602008006761 Country of ref document: DE Representative=s name: KRAMER - BARSKE - SCHMIDTCHEN, DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111031 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602008006761 Country of ref document: DE Representative=s name: KRAMER - BARSKE - SCHMIDTCHEN, DE Effective date: 20130430 Ref country code: DE Ref legal event code: R081 Ref document number: 602008006761 Country of ref document: DE Owner name: YAMAHA CORPORATION, JP Free format text: FORMER OWNER: CAMBRIDGE MECHATRONICS LTD., CAMBRIDGE, GB Effective date: 20130430 Ref country code: DE Ref legal event code: R081 Ref document number: 602008006761 Country of ref document: DE Owner name: YAMAHA CORPORATION, HAMAMATSU, JP Free format text: FORMER OWNER: CAMBRIDGE MECHATRONICS LTD., CAMBRIDGE, GB Effective date: 20130430 Ref country code: DE Ref legal event code: R082 Ref document number: 602008006761 Country of ref document: DE Representative=s name: KRAMER BARSKE SCHMIDTCHEN PATENTANWAELTE PARTG, DE Effective date: 20130430 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110804 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: TP Owner name: YAMAHA CORPORATION, JP Effective date: 20130606 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20121031 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20121031 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20130718 AND 20130724 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110504 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231020 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231024 Year of fee payment: 16 Ref country code: DE Payment date: 20231020 Year of fee payment: 16 |