EP2944098B1 - A sound-field control method using a planarity measure - Google Patents
A sound-field control method using a planarity measure Download PDFInfo
- Publication number
- EP2944098B1 EP2944098B1 EP14700441.0A EP14700441A EP2944098B1 EP 2944098 B1 EP2944098 B1 EP 2944098B1 EP 14700441 A EP14700441 A EP 14700441A EP 2944098 B1 EP2944098 B1 EP 2944098B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- sound
- zone
- target
- acc
- planarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 34
- 239000013598 vector Substances 0.000 claims description 12
- 230000006870 function Effects 0.000 description 24
- 238000013459 approach Methods 0.000 description 16
- 238000009826 distribution Methods 0.000 description 14
- 238000005457 optimization Methods 0.000 description 14
- 239000011159 matrix material Substances 0.000 description 10
- 230000015572 biosynthetic process Effects 0.000 description 7
- 238000003786 synthesis reaction Methods 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 6
- 230000004044 response Effects 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 241000196324 Embryophyta Species 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000009472 formulation Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 230000005404 monopole Effects 0.000 description 3
- 238000001308 synthesis method Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 241001085205 Prenanthella exigua Species 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013329 compounding Methods 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000009396 hybridization Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000026683 transduction Effects 0.000 description 1
- 238000010361 transduction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
- H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
- H04R2201/401—2D or 3D arrays of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
Definitions
- the present invention relates to a method of controlling sound output to a space and in particular sound output to a space having a target zone and a dark zone, where a predetermined audio signal is desired in the target zone but no sound (or another audio signal) is desired in the dark zone.
- the energy in the zones can be controlled, either via a beam forming approach, or using an energy cancellation based optimization approach.
- brightness control is an optimized beam former for focusing the energy in a particular direction
- acoustic contrast control (ACC) is an energy cancellation method creating a region of significant attenuation.
- ACC acoustic contrast control
- Various applications of this work have been investigated for the personal audio scenario, including for PC users, aircraft passengers, and users of mobile devices.
- An alternative cancellation method known as acoustic energy difference maximization (AEDM) was proposed with a modified cost function to avoid the matrix inversion and allow for adjustment of the array control effort.
- AEDM acoustic energy difference maximization
- the energy cancellation methods can produce excellent acoustic contrast (cancellation) between the zones, offering great potential for sound zone reproduction, yet the phase in the target zone is uncontrolled. Consequently, multiple plane wave components can impinge on the zone from various directions, which may create highly self-cancelling waves or other undesirable audio artefacts.
- the synthesis approaches are able to resolve this issue, but often at the cost of some contrast performance and with high array effort. Furthermore, in being strictly specified, these methods are subject to spatial aliasing problems as frequency increases, with an upper frequency bound for accurate reproduction (which is required for good cancellation).
- hybrid methods that attempt to recreate a plane wave in the target zone whilst using an energy cancellation approach for the dark zone.
- a constrained optimization approach has been devised based on PM to constrain the dark zone energy without specifying a desired field for that zone, and similar formulations have been proposed based on hybridisations of PM with ACC and AEDM.
- Such hybrids have been shown to be effective at relatively low frequencies for reproducing planar sound fields with good contrast between the zones. In each case, the target sound field must be specified by the designer.
- plane wave sound fields exhibit other advantageous properties such as good homogeneity of sound level across the zone and the avoidance of self cancellation properties.
- Plane wave reproduction has commonly been regarded as the best way to create a target zone with these properties (due to the potential of reproducing any given sound field by superposition), and synthesis approaches have been adopted for sound zone problems where this is a particular requirement.
- Other methods have considered the manipulation of intensity in a single zone (with no corresponding cancellation region). For instance, the intensity based on adjacent microphone responses in a zone may be spatially averaged, optimized and controlled, or a plane wave may be reproduced by focusing the plane wave energy towards a point in the wavenumber domain.
- the intensity is estimated using a superdirective beam forming approach that could be applied to any microphone array (rather than depending on a particular geometry), is spatially averaged, and applied to the target zone meanwhile a dark zone is also created.
- the present invention generally relates to a novel cost function 'planarity control' for sound zone optimization, where the incoming plane wave direction with respect to the target zone is constrained over a range of angles, rather than a single one (or unconstrained). In this way, a planar sound field can be reproduced (alongside excellent cancellation) but the optimization is free to find the best plane wave direction within the specified range.
- the invention relates to a method according to claim 1.
- the properties optimized may be an optimization of a contrast between the dark zone and the target zone.
- another parameter may be that no propagation takes place via the dark zone.
- Another parameter may be the control effort which is a measure of the power required by the loudspeakers in order to create the desired sound. Excessive sound power leads to degradations in reproduced performance due to non-linear transduction, an increased effect of room reflections and increased sensitivity to errors in the system.
- a Direction of Arrival is a direction seen from the target zone.
- a direction is determined by an angle from a predetermined direction, such as a direction perpendicular to a line through a centre of the target zone and a centre of the dark zone.
- the range of Directions of Arrival is 120-240 degrees to a direction perpendicular to an axis intersecting centres of the target and dark zones.
- the same ⁇ is used for all frequencies.
- step c) further requires that no sound must propagate across the dark zone.
- a sound zone system comprises an array of loudspeakers and a number of microphones sampling the sound field in each zone.
- the vectors of pressures at the microphones in each zone can likewise be written.
- p M [ p 1 p 2 ...p N ] T
- p B [ p 1 p 2 ...p N ] T
- p m is the complex pressure at the m th microphone in zone A
- p n is the complex pressure at the n th microphone in zone B.
- zone A the plant matrix containing the transfer functions between the loudspeakers and the microphones in zone A is defined as G A , and the equivalent notation is used for G B .
- Acoustic contrast is a spatially averaged summary measure for sound zone performance, and is commonly used in the cancellation literature to describe system performance.
- zone A defined by M microphones
- the pressures p B and p SPL B can be obtained.
- the planarity of the sound field is a physical measure for assessing the extent to which a reproduced sound field resembles a plane wave.
- the reproduction error often used in the sound field synthesis literature to quantify the performance of sound field synthesis methods, may rate a highly planar sound field very poorly if the plane wave direction does not coincide with the specified sound field.
- the absolute angle of the incoming plane wave is not important and the planarity property has been designed to test each plane wave component impinging on the microphone array.
- the energy distribution at the microphone array over each incoming plane wave direction w [ w1 ...
- planarity A ⁇ i w i u i . u i ⁇ i w i
- u i the unit vector associated with the ith component's direction
- u ⁇ i the sum of all components in the th direction
- ⁇ ⁇ arg max i w i
- the ACC cost function represents the energy cancellation approach.
- the minimum can be found by taking the eigenvector corresponding to the minimum eigenvalue of G b H G b ⁇ 1 G d H G d + ⁇ 2 I , which is equivalent to taking the eigenvector corresponding to the maximum eigenvalue of G d H G d + ⁇ 2 I ⁇ 1 G b H G b .
- the regularization term ⁇ 2 therefore regularizes both the control effort and the numerical conditioning of the inversion of G H d G d .
- any phase distribution can be specified for PM.
- a complex pressure is specified at each microphone; in this case, a plane wave is specified propagating through the target zone, and a pressure amplitude of zero is specified for the dark zone positions.
- the proposed cost function optimizes the acoustic planarity by modification of the ACC cost function stated in Eq. (7).
- h i argmax S i H S i + ⁇ h I ⁇ 1 P i H P i , and each row of H b is populated by the corresponding h i .
- H b represents a mapping between the complex pressures at the microphones and the reproduced plane wave energy distribution over azimuth, as previously introduced in Eq. (4). Therefore, it presents us with an opportunity to include it in the cost function for the sound zone optimization, and achieve some control of the plane wave energy in the target zone. In order to do this, a weighting must be applied based on the acceptable range of incoming plane wave directions.
- the optimal source weights can be found to be the eigenvector corresponding to the maximum eigenvalue of G d H G d + ⁇ 2 I ⁇ 1 G b H H b H ⁇ H b G b .
- the optimization is thus constrained to maximise the sound energy in the target zone from among the potential incoming azimuths allowed by ⁇ .
- the selection of ⁇ is clearly a significant factor. If the vector is filled with ones, then the cost function in Eq. (18) is no different from the contrast control formulation in Eq. (7) and identical performance is achieved. If, on the other hand, the vector is populated with zeros apart from a single target direction, a plane wave impinging from the specified direction should be reproduced.
- planarity optimization algorithm The operation and performance of the planarity optimization algorithm is demonstrated in the following by means of simulations.
- Figure 1 illustrates a geometry of the sound zone system, where zone A is the target zone and zone B is the dark zone.
- the outer (dashed) circle represents the loudspeaker array, and the inner circle the reproduction radius with respect to the aliasing limit of the synthesis methods.
- the directions of plane wave incidence with respect to zone A are indicated.
- the simulations were conducted in Matlab, simulating a free-field lossless anechoic environment, with each source modelled as an ideal monopole.
- the test geometry comprised a circular array with 48 equally spaced loudspeakers, of radius 1.2m (see Fig. 1 ), and 156 omnidirectional microphones in each zone spaced at 2.1cm and arranged to sample 30cm diameter circles.
- the microphones used for calculating the sound zone filters (setup) and those for obtaining predictions (playback) were kept spatially distinct or mismatched in order to assess a slightly wider spatial region than the specific points sampled for setup (becoming more independent with increasing frequency).
- the target sound pressure level was set to 76dB SPL (achieved by scaling the prototype source weight vector q ), which has been shown to be a comfortable listening level and has been used during listening tests based on the sound zone interference situation. This imposes an upper limit on the achievable contrast scores as we do not allow sound pressure levels below 0dB, although we consider 0dB to be measurable (there is no noise floor imposed).
- the minimum regularization parameter component ⁇ min was set to enforce a maximum matrix condition number of 10 10 , and the effort regularization parameter component ⁇ eff adjusted, where necessary, to enforce a maximum effort of 20dB, with reference to a single monopole on the radius of the circle ( q r , Eq. (6)).
- planarity control method was applied to the array and the results obtained under the evaluation metrics introduced in section 2.1.
- Figure 2 shows the method's performance over frequency, alongside those obtained for ACC, PM and ACC-PM under the same conditions.
- the contrast performance is very good and very consistent across the extended midrange band of 50-7000Hz.
- the term responsible for cancellation in the proposed planarity control (Eq. (18)) is unchanged from that in the ACC cost function (Eq. (7)) and the dark zone creation is therefore similar in each case, resulting in perfect cancellation as for ACC, and outperforming PM and ACC-PM.
- control effort performance tends towards that of ACC, which gives preferable performance by a small margin across the whole range, outperforming the planarity control by up to 6dB at the lowest frequencies but generally being within 3dB. Nonetheless, the effort is below 0dB for much of the frequency range, and it is consistently preferable to PM and ACC-PM under the same conditions. Finally, there is a good planarity performance across frequency. Under this metric, the synthesis metrics PM and ACC-PM naturally produce optimal scores for significant portions of the frequency range.
- planarity scores are similar to PM and ACC-PM, and greatly improved from ACC, as the DOA constraint has removed the self cancellation artefacts from the reproduced sound field.
- planarity control is its robustness as a function of frequency.
- PM and ACC-PM suffer from well documented limitations to the upper frequency of accurate reproduction, depending on the loudspeaker spacing and array radius, the planarity control is able to operate well above this limit.
- the aliasing problems for PM and ACC-PM can be observed in relation to each of the evaluation metrics: from the contrast the effect of aliasing lobes passing through the dark zone can be observed, and the corresponding control effort response noted.
- the planarity response is interesting. because a planar target field is still reproduced. Even under this metric, however, these methods falter around the aliasing frequency. As there is little to distinguish between the performance characteristics of PM and ACC-PM apart from the slight improvement in contrast, ACC-PM is taken forward for further simulations.
- the target (left) and dark zones are indicated by the white circles.
- the properties of the sound field reproduced by the planarity control method are of some interest to potential users.
- ACC-PM The synthesis adopted in ACC-PM can be seen to successfully place the plane wave propagation to the specified direction, with a wider lobe at low frequency due to the poor beam former resolution (c.f. planarity scores for PM at low frequency in Fig. 2 ), and the higher frequency aliasing effects noticeable as side lobes.
- ACC produces plane wave energy from a wide range of azimuths as well as self-cancellation patterns. It is likely that such a field would result in an unpleasant listening experience.
- the distribution of plane wave energy directions over frequency for planarity control can be noted to conform, for the most part, to the target range, with side lobes emerging at higher frequencies above the array aliasing limit.
- Figure 4 illustrates energy distributions over azimuth for PC (top), ACC (middle) and ACC-PM (bottom), plotted at 100Hz intervals from 100-7000Hz.
- the bold dot-dash line in the uppermost plot indicates the specified window along the diagonal of ⁇ , and the directions 90_ and 180_ correspond to incoming plane wave directions of west-east and north-south, respectively, in relation to Fig. 3 .
- Figure 5 illustrates target vs. achieved energy distribution over azimuth at 1kHz, using planarity control to specify the DOA, for 90° (west-east) (top), 180° (north-south) (middle) and 146° (optimal) (bottom). Maximum contrast is achieved in each case. Energy reproduced by PM is included for reference (dot-dash line)
- the window was set to allow a single azimuth (with a raised-cosine weighting to smooth the transition), and the direction varied.
- Three significant results are plotted in Fig. 5 for specified directions of 90°, 146° (the optimal case for this frequency) and 180°, at 1kHz.
- the planarity control method can be seen to accurately place the plane wave to arrive from the required direction (corresponding to north-south in Fig. 3 ), and for the optimal case this is achieved with additional side lobe suppression, although the width of the energy lobe for PM is slightly narrower.
- Figure 6 illustrates target vs. achieved energy distributions over azimuth with lines plotted over frequency, for low (top), mid (middle) and high (bottom) frequency bands, using planarity control to constrain the DOA to a window around the optimal angle of 146_. Maximum contrast is achieved in each case.
- the compounding of poor beam former resolution for both setup and evaluation results in very wide lobes, at mid frequencies up to the spatial aliasing limit (approximately 2kHz) the placement is satisfactory, and at high frequencies the behaviour is rather similar to that of ACC-PM, where side lobes emerge. Even so, the main energy components remain close to the specified window and good contrast and planarity are still achieved.
- a method for optimizing the planarity in the target zone, as well as producing significant cancellation between zones, has been proposed.
- the method has been shown to be comparable to the well-established acoustic control method in terms of contrast and control effort, and superior for creating a planar field in the target zone. It also outperforms the pressure matching approach and a state of the art hybrid between pressure matching and acoustic contrast control in terms of contrast and control effort, and particularly in terms of its ability to produce a good cancellation region above the spatial aliasing region, and a planar field around this limit.
- the resolution of the microphone array beam former limits planarity performance at low frequencies below 400Hz. Definition of the weighting matrix ⁇ is very important for good performance.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Circuit For Audible Band Transducer (AREA)
Description
- The present invention relates to a method of controlling sound output to a space and in particular sound output to a space having a target zone and a dark zone, where a predetermined audio signal is desired in the target zone but no sound (or another audio signal) is desired in the dark zone.
- In the mid 1990s, Druyvesteyn proposed the concept of personal sound zones. Since then, array signal processing for the creation of sound zones has emerged as a key sub-topic of sound field control. Fundamentally, at least two kinds of region must be created by the loudspeaker array - the target zone, where the sound pressure reaches a certain target level, and the dark zone, a region of cancellation where the audio programme delivered to the target zone is attenuated. If desired, an independent audio system then can be created by superposition.
- The development of sound zones in the academic literature has seen the emergence of techniques which broadly fall in to two categories. One technique, with its heritage in wave-field synthesis, is to precisely specify the sound field controlled by the array. In this manner, a target sound field can be defined and the dark zone created by attenuating the sound pressure magnitudes over a region. Such control has been investigated both analytically based on sound field coefficient translation in 2D (using line sources) and 2.5D (using point sources), and by an optimized pressure matching (PM) to directly minimise the error between a discretised desired sound field and the field reproduced by the array. These methods often require many loudspeakers, although some recent work has given attention to this. Typically, a plane-wave is specified as the desired field, although any sound field could in principle be synthesised.
- Alternatively, the energy in the zones can be controlled, either via a beam forming approach, or using an energy cancellation based optimization approach. For instance, brightness control is an optimized beam former for focusing the energy in a particular direction, and acoustic contrast control (ACC) is an energy cancellation method creating a region of significant attenuation. Various applications of this work have been investigated for the personal audio scenario, including for PC users, aircraft passengers, and users of mobile devices. An alternative cancellation method known as acoustic energy difference maximization (AEDM) was proposed with a modified cost function to avoid the matrix inversion and allow for adjustment of the array control effort.
- Owing to the nature of the respective cost functions, they have distinctive performance characteristics. The energy cancellation methods can produce excellent acoustic contrast (cancellation) between the zones, offering great potential for sound zone reproduction, yet the phase in the target zone is uncontrolled. Consequently, multiple plane wave components can impinge on the zone from various directions, which may create highly self-cancelling waves or other undesirable audio artefacts. The synthesis approaches are able to resolve this issue, but often at the cost of some contrast performance and with high array effort. Furthermore, in being strictly specified, these methods are subject to spatial aliasing problems as frequency increases, with an upper frequency bound for accurate reproduction (which is required for good cancellation).
- Accordingly, recent advances in the literature have included hybrid methods that attempt to recreate a plane wave in the target zone whilst using an energy cancellation approach for the dark zone. A constrained optimization approach has been devised based on PM to constrain the dark zone energy without specifying a desired field for that zone, and similar formulations have been proposed based on hybridisations of PM with ACC and AEDM. Such hybrids have been shown to be effective at relatively low frequencies for reproducing planar sound fields with good contrast between the zones. In each case, the target sound field must be specified by the designer.
- Whilst in some cases reproduction of specific plane wave components may be necessary (e.g. for spatial audio), specifying a plane wave is by no means the only satisfactory propagation pattern that the array could achieve in the context of personal audio. However, plane wave sound fields exhibit other advantageous properties such as good homogeneity of sound level across the zone and the avoidance of self cancellation properties.
- Plane wave reproduction has commonly been regarded as the best way to create a target zone with these properties (due to the potential of reproducing any given sound field by superposition), and synthesis approaches have been adopted for sound zone problems where this is a particular requirement. Other methods have considered the manipulation of intensity in a single zone (with no corresponding cancellation region). For instance, the intensity based on adjacent microphone responses in a zone may be spatially averaged, optimized and controlled, or a plane wave may be reproduced by focusing the plane wave energy towards a point in the wavenumber domain.
- Here, the intensity is estimated using a superdirective beam forming approach that could be applied to any microphone array (rather than depending on a particular geometry), is spatially averaged, and applied to the target zone meanwhile a dark zone is also created.
- Technology of this type may be seen in Møller, Martin Bo et al.: "A hybrid method combining synthesis of a sound field control of acoustic contrast", AES CONVENTION 132, april 2012, (2012-04-26), XP040574591.
- The present invention generally relates to a novel cost function 'planarity control' for sound zone optimization, where the incoming plane wave direction with respect to the target zone is constrained over a range of angles, rather than a single one (or unconstrained). In this way, a planar sound field can be reproduced (alongside excellent cancellation) but the optimization is free to find the best plane wave direction within the specified range.
- The invention relates to a method according to
claim 1. - In this respect, the properties optimized may be an optimization of a contrast between the dark zone and the target zone.
- As is mentioned below, another parameter may be that no propagation takes place via the dark zone.
- Another parameter may be the control effort which is a measure of the power required by the loudspeakers in order to create the desired sound. Excessive sound power leads to degradations in reproduced performance due to non-linear transduction, an increased effect of room reflections and increased sensitivity to errors in the system.
- A Direction of Arrival is a direction seen from the target zone. Usually, a direction is determined by an angle from a predetermined direction, such as a direction perpendicular to a line through a centre of the target zone and a centre of the dark zone.
- In one embodiment, the range of Directions of Arrival is 120-240 degrees to a direction perpendicular to an axis intersecting centres of the target and dark zones.
- In a preferred embodiment, the same Γ is used for all frequencies.
- In a preferred embodiment, step c) further requires that no sound must propagate across the dark zone.
- In the following, preferred embodiments of the invention will be described with reference to the drawing, wherein:
-
figure 1 illustrates a geometry of a sound zone system, -
figure 2 illustrates the performance of planarity control (PC) with respect to ACC, PM and ACC-PM (a = 0:9), under the metrics of contrast (top), effort (middle) and planarity (bottom), -
figure 3 illustrates a sound pressure level (top row) and phase (bottom row) maps for PC, ACC and ACC-PM (a = 0:9), -
figure 4 illustrates energy distributions over azimuth for PC (top), ACC (middle) and ACC-PM (bottom), plotted at 100Hz intervals from 100-7000Hz, -
figure 5 illustrates target vs. achieved energy distribution over azimuth, and -
figure 6 illustrates target vs. achieved energy distributions. - A sound zone system comprises an array of loudspeakers and a number of microphones sampling the sound field in each zone. For a single frequency, the source weight vector is written as q = [ q1q2... qL ] T, where there are L sources and q 1 describes the lth loudspeaker's complex source strength. The vectors of pressures at the microphones in each zone can likewise be written. Here, we consider two zones, A and B; p A = [p 1 p 2 ... p M] T and pB = [p 1 p 2 ...p N] T , where there are M microphones in zone A and N in zone B, pm is the complex pressure at the mth microphone in zone A and and pn is the complex pressure at the nth microphone in zone B.
- For zone A, the plant matrix containing the transfer functions between the loudspeakers and the microphones in zone A is defined as G A , and the equivalent notation is used for G B . The pressure vectors for each zone are populated by the summation of the contribution of each loudspeaker at each microphone, written in vector notation as p A =G A q and p B =G B q for zones A and B, respectively.
- In this section, the three evaluation metrics used for evaluation of the novel cost function are introduced.
- These measure the achieved zone separation, the extent to which the target zone sound field exhibits characteristics of a plane wave, and the physical cost of cancellation.
- Acoustic contrast is a spatially averaged summary measure for sound zone performance, and is commonly used in the cancellation literature to describe system performance. For zone A defined by M microphones, the spatially averaged squared pressure is
-
- The planarity of the sound field is a physical measure for assessing the extent to which a reproduced sound field resembles a plane wave. The reproduction error, often used in the sound field synthesis literature to quantify the performance of sound field synthesis methods, may rate a highly planar sound field very poorly if the plane wave direction does not coincide with the specified sound field. For sound zone reproduction at a single frequency, the absolute angle of the incoming plane wave is not important and the planarity property has been designed to test each plane wave component impinging on the microphone array. The energy distribution at the microphone array over each incoming plane wave direction w = [w1 ... wi] is given by
- The elements of H could alternatively be calculated using a spatial Fourier decomposition approach. The planarity metric can now be defined as the ratio between the energy due to the largest plane wave component and the total energy flux of plane wave components:
-
- It is a necessity in any practical system to achieve a suitably low control effort. On the one hand, it is physically related to whether a set of source weights is realizable through loudspeakers. Yet in addition, limiting the control effort results in there being less sound energy overall in the environment, leading to improved robustness to reflections in reverberant rooms, and limits the white noise gain of the system, improving robustness to other kinds of errors such as measurement noise and non-linear distortion.
- To facilitate a comparison between the proposed cost function and existing sound field control methods, ACC, PM and their hybrid are formally introduced in the following sections.
- The ACC cost function, where the ratio of the spatially averaged sound pressure levels between the bright zone and the dark zone is maximised, represents the energy cancellation approach. The cost function can be written as a constrained optimization problem based on minimising the dark zone pressure and constrained by the bright zone pressure and control effort:
-
- The minimum can be found by taking the eigenvector corresponding to the minimum eigenvalue of
- As a sound field synthesis method, any phase distribution can be specified for PM. A complex pressure is specified at each microphone; in this case, a plane wave is specified propagating through the target zone, and a pressure amplitude of zero is specified for the dark zone positions. The optimization cost function is written to minimise the error e=Gq-d between the desired sound field d and reproduced sound field, with a control effort constraint which acts as Tikhonov regularization:
-
- For the hybrid solution combining PM and ACC, the PM portion of the cost function is restricted to the reproduction of the bright zone, whilst the contrast control formulation is used for cancellation. Here, for consistency with the other methods, we introduce Tikhonov regularization instead of using the pseudo-inverse as in the original work:
- The proposed cost function optimizes the acoustic planarity by modification of the ACC cost function stated in Eq. (7). The elements of H from Eq. (4) can be written in full, with respect to the microphones in the target zone, as
- H b represents a mapping between the complex pressures at the microphones and the reproduced plane wave energy distribution over azimuth, as previously introduced in Eq. (4). Therefore, it presents us with an opportunity to include it in the cost function for the sound zone optimization, and achieve some control of the plane wave energy in the target zone. In order to do this, a weighting must be applied based on the acceptable range of incoming plane wave directions. Such a weighting can be specified in terms of the desired normalised energy distribution over DOA by means of a diagonal matrix Γ comprising weights, typically set between zero and one:
- The optimization is thus constrained to maximise the sound energy in the target zone from among the potential incoming azimuths allowed by Γ. The selection of Γ is clearly a significant factor. If the vector is filled with ones, then the cost function in Eq. (18) is no different from the contrast control formulation in Eq. (7) and identical performance is achieved. If, on the other hand, the vector is populated with zeros apart from a single target direction, a plane wave impinging from the specified direction should be reproduced. The approach is somewhat similar to that employed in the prior art, where a mapping matrix was used to deactivate certain loudspeakers for efficient 3-D reproduction based on a non-uniform array, yet here the loudspeakers are not expressly deactivated as they may contribute to the cancellation region as well as the target plane wave direction. Nevertheless, the solution is highly efficient. When the range of allowable angles is suitably designed, the system is free to maximise the energy under this constraint, which is best achieved by the generation of a planar sound field, and thus the planarity is optimized. Furthermore, if Γ is kept identical over frequency, similarity between adjacent frequency bins can be achieved.
- The operation and performance of the planarity optimization algorithm is demonstrated in the following by means of simulations.
-
Figure 1 illustrates a geometry of the sound zone system, where zone A is the target zone and zone B is the dark zone. The outer (dashed) circle represents the loudspeaker array, and the inner circle the reproduction radius with respect to the aliasing limit of the synthesis methods. The directions of plane wave incidence with respect to zone A are indicated. - The simulations were conducted in Matlab, simulating a free-field lossless anechoic environment, with each source modelled as an ideal monopole. The free-field Green's function was used to populate the plant matrices, giving the transfer function at each microphone due to a loudspeaker at distance r:
- The test geometry comprised a circular array with 48 equally spaced loudspeakers, of radius 1.2m (see
Fig. 1 ), and 156 omnidirectional microphones in each zone spaced at 2.1cm and arranged to sample 30cm diameter circles. The microphones used for calculating the sound zone filters (setup) and those for obtaining predictions (playback) were kept spatially distinct or mismatched in order to assess a slightly wider spatial region than the specific points sampled for setup (becoming more independent with increasing frequency). The target sound pressure level was set to 76dB SPL (achieved by scaling the prototype source weight vector q), which has been shown to be a comfortable listening level and has been used during listening tests based on the sound zone interference situation. This imposes an upper limit on the achievable contrast scores as we do not allow sound pressure levels below 0dB, although we consider 0dB to be measurable (there is no noise floor imposed). -
Figure 2 illustrates the performance of planarity control (PC) with respect to ACC, PM and ACC-PM (a = 0:9), under the metrics of contrast (top), effort (middle) and planarity (bottom) - To set the regularization conditions, the minimum regularization parameter component λmin was set to enforce a maximum matrix condition number of 1010, and the effort regularization parameter component λeff adjusted, where necessary, to enforce a maximum effort of 20dB, with reference to a single monopole on the radius of the circle (qr, Eq. (6)).
- The plane wave for the PM and ACC-PM hybrid (herein simply referred to as ACC-PM) approaches was specified to travel from north to south (ψ = 180_, marked on
Fig. 1 ), and the weighting matrix Γ was set to constrain the incoming plane wave components between 120° and 240°. The weighting on the diagonal of Γ is indicated inFig. 4 (top). The weighting α for ACC-PM was set to 0.9 to encourage good contrast performance. - The planarity control method was applied to the array and the results obtained under the evaluation metrics introduced in section 2.1.
Figure 2 shows the method's performance over frequency, alongside those obtained for ACC, PM and ACC-PM under the same conditions. The contrast performance is very good and very consistent across the extended midrange band of 50-7000Hz. The term responsible for cancellation in the proposed planarity control (Eq. (18)) is unchanged from that in the ACC cost function (Eq. (7)) and the dark zone creation is therefore similar in each case, resulting in perfect cancellation as for ACC, and outperforming PM and ACC-PM. - Likewise, the control effort performance tends towards that of ACC, which gives preferable performance by a small margin across the whole range, outperforming the planarity control by up to 6dB at the lowest frequencies but generally being within 3dB. Nonetheless, the effort is below 0dB for much of the frequency range, and it is consistently preferable to PM and ACC-PM under the same conditions. Finally, there is a good planarity performance across frequency. Under this metric, the synthesis metrics PM and ACC-PM naturally produce optimal scores for significant portions of the frequency range. However, with the exception of the low frequency performance (due to poor resolution of the planarity steering matrix in this region) and a narrow notch at 3.6kHz, the planarity scores are similar to PM and ACC-PM, and greatly improved from ACC, as the DOA constraint has removed the self cancellation artefacts from the reproduced sound field.
- Perhaps the most striking characteristic of the planarity control method is its robustness as a function of frequency. Where PM and ACC-PM suffer from well documented limitations to the upper frequency of accurate reproduction, depending on the loudspeaker spacing and array radius, the planarity control is able to operate well above this limit. In fact, the aliasing problems for PM and ACC-PM can be observed in relation to each of the evaluation metrics: from the contrast the effect of aliasing lobes passing through the dark zone can be observed, and the corresponding control effort response noted. The planarity response is interesting.
because a planar target field is still reproduced. Even under this metric, however, these methods falter around the aliasing frequency. As there is little to distinguish between the performance characteristics of PM and ACC-PM apart from the slight improvement in contrast, ACC-PM is taken forward for further simulations. - The optimal contrast and planarity performance obtained using planarity control can be further clarified by studying the sound pressure level and phase maps shown in
Fig. 3. Figure 3 illustrates a sound pressure level (top row) and phase (bottom row) maps for PC, ACC and ACC-PM (a = 0:9). The target (left) and dark zones are indicated by the white circles. For the sound pressure level maps, white indicates a high sound pressure and black a low sound pressure at high frequencies for PM and ACC-PM, We can now confirm that the planarity control produces an ACC-like dark zone, yet replaces the north-south self-cancellation (visible across the whole of the bottom-middle plot) in the target zone with a planar field (indicated by the sharp transition in the phase response), and reduces the overall sound pressure in the environment as a consequence of the low effort score with relation to PM and ACC-PM (visible by comparing the amount of bright white in the top row ofFig. 3 ). - The properties of the sound field reproduced by the planarity control method are of some interest to potential users.
- First, we consider the energy distribution over azimuth (with respect to the target zone) obtained for the window function used for the simulations in section 4.2. We have seen from the planarity scores (
Fig. 2 , bottom) and the phase distributions in the enclosure (Fig. 3 , bottom) that the planarity control method is capable of creating highly planar fields in the target zone, for single frequencies. However, these plots do not give us an indication of the range of incoming plane wave directions as a function of frequency. Therefore, inFig. 4 the normalised energy distributions for multiple frequencies have been plotted across azimuth for planarity control, ACC and ACC-PM. This gives us a useful insight in to the planarity control's performance in relation to the existing methods. The synthesis adopted in ACC-PM can be seen to successfully place the plane wave propagation to the specified direction, with a wider lobe at low frequency due to the poor beam former resolution (c.f. planarity scores for PM at low frequency inFig. 2 ), and the higher frequency aliasing effects noticeable as side lobes. Conversely, ACC produces plane wave energy from a wide range of azimuths as well as self-cancellation patterns. It is likely that such a field would result in an unpleasant listening experience. The distribution of plane wave energy directions over frequency for planarity control can be noted to conform, for the most part, to the target range, with side lobes emerging at higher frequencies above the array aliasing limit. -
Figure 4 illustrates energy distributions over azimuth for PC (top), ACC (middle) and ACC-PM (bottom), plotted at 100Hz intervals from 100-7000Hz. The bold dot-dash line in the uppermost plot indicates the specified window along the diagonal of Γ, and the directions 90_ and 180_ correspond to incoming plane wave directions of west-east and north-south, respectively, in relation toFig. 3 . -
Figure 5 illustrates target vs. achieved energy distribution over azimuth at 1kHz, using planarity control to specify the DOA, for 90° (west-east) (top), 180° (north-south) (middle) and 146° (optimal) (bottom). Maximum contrast is achieved in each case. Energy reproduced by PM is included for reference (dot-dash line) - To test the ability of the planarity control to reproduce a specific incoming plane wave direction, the window was set to allow a single azimuth (with a raised-cosine weighting to smooth the transition), and the direction varied. Three significant results are plotted in
Fig. 5 for specified directions of 90°, 146° (the optimal case for this frequency) and 180°, at 1kHz. In the middle plot (180°), the planarity control method can be seen to accurately place the plane wave to arrive from the required direction (corresponding to north-south inFig. 3 ), and for the optimal case this is achieved with additional side lobe suppression, although the width of the energy lobe for PM is slightly narrower. Yet for directions perpendicular to this (west-east propagation shown), which would require a beam to be placed across the dark zone, a highly self-cancelling pattern is instead reproduced and the peak in this direction is unsatisfactory. There is no variation in the contrast between these cases and the effort difference is minimal, yet if PM had been applied, the cancellation would have been poor and the effort very high, albeit with the specified plane wave component reproduced. An interesting property of the planarity control cost function is therefore exposed: that producing high contrast is the priority of the optimization, and that where specification of the incident direction does not conflict with contrast performance, the energy can be placed precisely in the desired direction. The behaviour over frequency is clarified byFig. 6 for a constrained window (146° ±20° with a raised cosine weighting).Figure 6 illustrates target vs. achieved energy distributions over azimuth with lines plotted over frequency, for low (top), mid (middle) and high (bottom) frequency bands, using planarity control to constrain the DOA to a window around the optimal angle of 146_. Maximum contrast is achieved in each case. - At low frequencies, the compounding of poor beam former resolution for both setup and evaluation results in very wide lobes, at mid frequencies up to the spatial aliasing limit (approximately 2kHz) the placement is satisfactory, and at high frequencies the behaviour is rather similar to that of ACC-PM, where side lobes emerge. Even so, the main energy components remain close to the specified window and good contrast and planarity are still achieved.
- A method for optimizing the planarity in the target zone, as well as producing significant cancellation between zones, has been proposed. The method has been shown to be comparable to the well-established acoustic control method in terms of contrast and control effort, and superior for creating a planar field in the target zone. It also outperforms the pressure matching approach and a state of the art hybrid between pressure matching and acoustic contrast control in terms of contrast and control effort, and particularly in terms of its ability to produce a good cancellation region above the spatial aliasing region, and a planar field around this limit. The resolution of the microphone array beam former limits planarity performance at low frequencies below 400Hz. Definition of the weighting matrix Γ is very important for good performance. The ability of Γ to constrain incident plane wave directions over frequency has been demonstrated, and furthermore under the condition that it does not require propagation across the dark zone, a precise plane wave direction can be specified. The method therefore presents a compelling cost function for sound zones where the self-cancellation artefacts of energy cancellation approaches can be reduced whilst allowing more flexibility over the incident plane wave specification, yet with the potential to reproduce a wave from a single direction if required.
Claims (5)
- A method of controlling sound output to a space by a plurality of sound generators, the method comprising:1) optimizing properties of a target zone in the space, wherein also a dark zone is present, by selecting, for at least one frequency, a Direction of Arrival where the sound at that frequency in the target zone should be planar and homogenous across the target zone, and2) outputting a sound into the space by the sound generators, the sound being output on the basis of the optimized propertiescharacterized in that a plurality of Directions of Arrival are defined, within +- 20° of a target direction, and step 1 comprises minimizing a function J defined by :P b is the pressure at each of M microphones in the target zoneP d is the pressure at each of N microphones in the dark zone
- A method according to claim 1, wherein step 1) a) comprises for each of a multiple of frequencies requiring that the sound at that frequency in the target zone is planar and homogenous across the target zone, and requiring that the plane waves at the individual frequencies are at least substantially aligned to each other.
- A method according to any of the preceding claims, wherein the range of angles is 120-240 degrees to a direction perpendicular to an axis intersecting centres of the target and dark zones..
- A method according to any of the preceding claims, wherein the same r is used for all frequencies.
- A method according to any of the preceding claims, wherein step c) further requires that no sound must propagate across the dark zone.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DKPA201300015 | 2013-01-10 | ||
PCT/EP2014/050081 WO2014108365A1 (en) | 2013-01-10 | 2014-01-06 | A sound-field control method using a planarity measure |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2944098A1 EP2944098A1 (en) | 2015-11-18 |
EP2944098B1 true EP2944098B1 (en) | 2017-03-29 |
Family
ID=49958444
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14700441.0A Active EP2944098B1 (en) | 2013-01-10 | 2014-01-06 | A sound-field control method using a planarity measure |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP2944098B1 (en) |
DK (1) | DK2944098T3 (en) |
WO (1) | WO2014108365A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112224142B (en) * | 2020-09-03 | 2022-04-15 | 中国汽车技术研究中心有限公司 | Vehicle-mounted independent acoustic space system and vehicle |
-
2014
- 2014-01-06 WO PCT/EP2014/050081 patent/WO2014108365A1/en active Application Filing
- 2014-01-06 DK DK14700441.0T patent/DK2944098T3/en active
- 2014-01-06 EP EP14700441.0A patent/EP2944098B1/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
WO2014108365A1 (en) | 2014-07-17 |
DK2944098T3 (en) | 2017-05-22 |
EP2944098A1 (en) | 2015-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Coleman et al. | Optimizing the planarity of sound zones | |
Coleman et al. | Acoustic contrast, planarity and robustness of sound zone methods using a circular loudspeaker array | |
Coleman et al. | Personal audio with a planar bright zone | |
US9219974B2 (en) | Method and apparatus for simultaneously controlling near sound field and far sound field | |
US20150043736A1 (en) | Method of applying a combined or hybrid sound-field control strategy | |
Zotter et al. | A beamformer to play with wall reflections: The icosahedral loudspeaker | |
Poletti et al. | Higher-order loudspeakers and active compensation for improved 2D sound field reproduction in rooms | |
US10880648B2 (en) | Loudspeaker assemblies and associated methods | |
Rosen et al. | FIR-based symmetrical acoustic beamformer with a constant beamwidth | |
Simon Galvez et al. | Loudspeaker arrays for transaural reproduction | |
US10244317B2 (en) | Beamforming array utilizing ring radiator loudspeakers and digital signal processing (DSP) optimization of a beamforming array | |
EP2755405A1 (en) | Zonal sound distribution | |
EP2944098B1 (en) | A sound-field control method using a planarity measure | |
Marschall et al. | Robustness of a mixed-order Ambisonics microphone array for sound field reproduction | |
Fazi et al. | Stage compression in transaural audio | |
Dick et al. | A comparison of measured room acoustics metrics using a spherical microphone array and conventional methods | |
Yanagidate et al. | Car cabin personal audio: Acoustic contrast with limited sound differences | |
Oreinos et al. | Validation of realistic acoustic environments for listening tests using directional hearing aids | |
Gao et al. | Weighted loudspeaker placement method for sound field reproduction | |
Gallian et al. | Optimisation of the target sound fields for the generation of independent listening zones in a reverberant environment | |
Rasumow et al. | The impact of the white noise gain (WNG) of a virtual artificial head on the appraisal of binaural sound reproduction | |
Zotter et al. | Compact spherical loudspeaker arrays | |
Riedel et al. | Design, Control, and Evaluation of Mixed-Order, Compact Spherical Loudspeaker Arrays | |
Simon Galvez | Design of an array-based aid for the hearing impaired | |
Poletti et al. | Analysis of 2D sound reproduction with fixed-directivity loudspeakers |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20150810 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20160428 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20161013 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: MICHELI AND CIE SA, CH |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 880768 Country of ref document: AT Kind code of ref document: T Effective date: 20170415 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602014008052 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20170518 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170630 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170629 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20170329 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 880768 Country of ref document: AT Kind code of ref document: T Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170629 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170729 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602014008052 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
26N | No opposition filed |
Effective date: 20180103 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180106 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20180928 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180106 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180106 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20140106 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170329 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DK Payment date: 20230116 Year of fee payment: 10 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230703 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240123 Year of fee payment: 11 Ref country code: CH Payment date: 20240202 Year of fee payment: 11 Ref country code: GB Payment date: 20240122 Year of fee payment: 11 |