EP2648425A1 - Simulation et configuration d'un système acoustique - Google Patents

Simulation et configuration d'un système acoustique Download PDF

Info

Publication number
EP2648425A1
EP2648425A1 EP12163054.5A EP12163054A EP2648425A1 EP 2648425 A1 EP2648425 A1 EP 2648425A1 EP 12163054 A EP12163054 A EP 12163054A EP 2648425 A1 EP2648425 A1 EP 2648425A1
Authority
EP
European Patent Office
Prior art keywords
speaker
sound
venue
virtual space
sound pressure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12163054.5A
Other languages
German (de)
English (en)
Other versions
EP2648425B1 (fr
Inventor
Eric Van Duin
Kevin Ivan Kleine
Zeger Van der Voet
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dutch & Dutch BV
Original Assignee
RINNIC/VAUDE BEHEER BV
RINNIC VAUDE BEHEER BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by RINNIC/VAUDE BEHEER BV, RINNIC VAUDE BEHEER BV filed Critical RINNIC/VAUDE BEHEER BV
Priority to EP12163054.5A priority Critical patent/EP2648425B1/fr
Publication of EP2648425A1 publication Critical patent/EP2648425A1/fr
Application granted granted Critical
Publication of EP2648425B1 publication Critical patent/EP2648425B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R27/00—Public address systems

Definitions

  • the invention relates to simulating and/or configuring an acoustic system, and, in particular, though not exclusively, to methods for simulating and/or configuring an acoustic system comprising acoustic sources, an electronically configurable acoustic system, an electronically configurable acoustic source for use in such system and a software program product for simulating and/or configuring such electronically configurable acoustic sources.
  • Optimizing the configuration of an acoustic system comprising a plurality of speakers for a predetermined venue is complex process requiring know-how that is only available to an expert, e.g. a sound engineer.
  • a sound engineer needs to look on-site for an optimal speaker configuration wherein multiple speakers need to be positioned and, preferably, configured within the constraints of a particular three-dimensional (3D) space such that a desired sound quality is achieved.
  • Sound engineers who need to find an optimal configuration on the basis of multiple system parameters (e.g. source position and direction, delay between different sources, dimensions and acoustic properties of objects located within that space etc.), may use simulation tools and know-how from earlier, similar situations in order to find a best or at least an acceptable solution for a new situation.
  • simulation tools however only provide limited functionality in the sense that such they allow simulation and - in some cases - visualization of a particular configuration defined by a sound engineer. Examples of such conventional simulation systems are described in US2009/0144036 and US7206415 .
  • These conventional simulation tools do not provide and/or suggest a particular optimal speaker configuration in terms of location and parameter settings of the speakers.
  • these simulation tools do not provide functionality wherein a user may define and shape the acoustic characteristic of a 3D venue space bound by e.g. walls, a floor and a ceiling and comprising objects with specific acoustic properties. For example, if an outdoor concert is held at a venue close to a residential area, it may be desired "engineer" the 3D sound pressure distribution around the venue such that the sound level at the residential area is kept below a certain level. Similarly, in an indoor concert hall, it may be desired to define silent zones where people can talk to each other without the need to isolate that such area using e.g. sound absorbing or reflecting objects.
  • the invention may relate to a computer-implemented acoustic simulation method for generating one or more speaker set-ups for electronically configuring speakers in an acoustical system such that, when configured, said speakers produce a desired sound pressure distribution in an acoustical space of a venue, wherein said method may comprise the steps of: providing an at least a partially bounded virtual space on a graphical display, wherein at least part of said virtual space substantially matches the acoustic space of said venue; determining one or more sound spots by selection of one or more volumes at one or more sound spot locations in said virtual space and assigning said one or more sound pressure values to said one or more volumes; determining at least part of said speaker set-ups settings by executing an evolutionary algorithm, preferably a differential evolutionary algorithm.
  • said algorithm comprising the steps of: generating a population of speaker set-ups, preferably a speaker set-up including at least one of: speaker location information, speaker phase delay information and/or speaker amplitude information; and, optimizing said population of speaker set-ups on the basis of said one or more sound pressure values associated with said one or more sound spots.
  • method allows simple and intuitive modeling on a venue of arbitrary design and optimization of an acoustical system located in that virtual space of the modeled venue such that speaker set-ups are generated which match a desired 3D sound pressure distribution.
  • said algorithm may further comprise: creating a further speaker set-up on the basis one or more speaker set-ups in said population; and, optimizing said population of speaker settings by replacing a speaker set-up in said population with said further speaker set-up on the basis of a fitness function configured for comparing sound pressure values associated with said one or more sound spots with sound pressure values calculated on the basis of said further speaker set-up.
  • said algorithm may comprise: determining one or more speaker envelopes by selecting one or more volumes in said displayed virtual space, a speaker envelope defining a volume for locating one or more configurable speakers; and, generating said population of speaker settings on the basis of said one or more speaker envelopes.
  • defining said one or more speaker envelopes further may include: selecting a predetermined number of speakers; associating speaker locations to said predetermined number of speakers on the basis of said speaker envelopes.
  • optimizing said population of speaker set-ups may further comprise: checking whether the speaker locations associated with said further speaker setting are at least partly located within said one or more speaker envelopes.
  • determining one or more sound spots may include: selecting said one or more volumes in said virtual space using a graphical user interface; or, selecting one or more sound spots from a database comprising a list of stored sound spots.
  • determining one or more speaker envelopes may include selecting said one or more volumes in said virtual space using a graphical user interface or selecting one or more speaker envelopes from a database comprising a list of stored sound spots.
  • selecting said one or more sounds spots or one or more speaker envelopes may further include: rendering of a graphical selector, preferably a 3D wire-frame, in said displayed virtual space; selecting a sound spot or speaker envelope location by locating said graphical selector in said virtual space; selecting a sound spot or a speaker envelope volume by graphically modifying the 3D shape of said selector.
  • a graphical selector preferably a 3D wire-frame
  • said method may further comprise: selecting at least one speaker set-up from said optimized population of speaker set-up; sending said at least one speaker set-up, preferably said speaker comprising filter coefficients, to at least part of said electronically configurable speakers.
  • the invention may relate to an acoustic simulation system for configuring one or more electronically configurable speakers in an acoustical system such that said configurable speakers produce a desired sound pressure distribution in an acoustical space of a venue, said acoustic simulation system being configured for: providing an at least partially bounded virtual space on a graphical display, wherein at least part of said virtual space substantially matches the acoustic space of said venue; determining one or more sound spots by selecting one or more volumes at one or more sound spot locations in said virtual space and assigning said one or more sound pressure values to said one or more volumes; determining at least part of said speaker set-ups settings by executing an evolutionary algorithm, preferably a differential evolutionary algorithm, said algorithm comprising: generating a population of speaker set-ups, preferably said speaker set-ups including at least one of: speaker location information, speaker phase delay information and/or speaker amplitude information; and, optimizing said population of speaker set-ups on the basis of said one or more sound pressure values associated
  • the invention may relate to a configurable acoustical source comprising: a digital signal processor configured for receiving at least one optimized speaker set-up from an acoustical simulation system as described above, said digital signal process further being configured to configure at least part of a digital audio filter on the basis of filter coefficients in said speaker set-up; or, said digital signal processor further being configured for executing a computer-implemented acoustic simulation method as described above.
  • the invention may relate to graphical user interface for use in an acoustical simulation system as described above, wherein said graphical user interface may be configured for graphically defining boundary conditions for said evolutionary algorithm, wherein said graphical user interface may be configured for: displaying a partially bounded 3D virtual space on a graphical display, wherein at least part of said virtual space substantially matches the acoustic space of said venue; rendering of a 3D shaped graphical selector, preferably a 3D wire-frame, in said displayed virtual space, said 3D shaped graphical selector being associated with location coordinates in said virtual space and wherein said 3D shaped graphical selector comprising one or more shape manipulation markers wherein a user may interact with said 3D shaped graphical selector such that: a sound spot location is selected by locating said graphical selector at predetermined location coordinates in said virtual space; a sound spot volume is selected by graphically modifying the 3D shape of said selector so that it defines volume of a predetermined size in said virtual space.
  • the invention also relates to a computer program product, implemented on computer-readable non-transitory storage medium, wherein the computer program product may be configured for, when run on a computer, executing the method steps as described above.
  • the computer program product may be configured for, when run on a computer, executing the method steps as described above.
  • Fig. 1 schematically depicts an acoustic simulation system 102 according to an embodiment of the invention.
  • Fig. 1 depicts an acoustic simulation system comprising a 3D venue editor 104 for graphically building and modeling a virtual 3D model of a venue using a special graphical user interface (GUI), an acoustic simulation engine 106 and a filter generator 108.
  • the acoustic simulation system is configured to design, build and simulate a virtual 3D model of a real-life venue 114 and to use the simulation results to determine locations for configurable speakers in the real-life venue and to generate filter coefficients for the configurable speakers 118 1-4 .
  • a venue may be defined as a place wherein the acoustical system is installed or going to be installed, e.g. a concert hall, a shopping mall or simply a living hall at home.
  • the venue typically defines a 3D space, hereafter referred to as a (3D) venue space, which is bound or at least partially bound by structures, e.g. walls, floors and/or a ceiling, which may (partially) reflect, diffract and/or absorb sound (sound pressure waves) generated by the speakers. Sound may also be (partially) reflected, diffracted and/or absorbed by objects located in the venue, e.g. a stage 116, tribunes, people, etc.
  • the sound experienced by a person in the venue is determined by the acoustical properties of the above-mentioned structures and objects (including people) and the configuration of the acoustic system.
  • the sound pressure waves generated by the speakers will cause a complex sound pressure distribution in the 3D venue space, which may be manipulated by either changing the venue structure or by changing the locations and/or setting of the speakers.
  • an acoustic system should be able to deliver sound of a particular quality and/or level.
  • the sound pressure distribution depends on many parameters and is therefore too complex to determine. Therefore, typically, only sub-optimal configurations are realized using trail-and-error methods combined with specific know-how and/or software tools.
  • the simulation system comprises a 3D venue editor.
  • the editor is configured to render in a graphical user interface (GUI), which allows user interaction 112, in order to graphically define virtual structures and objects in 3D space.
  • GUI graphical user interface
  • the GUI may allow to define predetermined volumes - sound spots and/or listening planes - in the 3D venue space and assigning certain targeted acoustic pressure values to thereto.
  • the GUI of the 3D venue editor may further allow a user to select speaker envelopes, i.e. predetermined volumes wherein configurable speakers are allowed to be located. The sound spots, listening planes and speaker envelopes will be described hereunder in more detail.
  • the venue data may be stored in a database 110 for later use. Different pre-defined venues may be stored in the database allowing a user to select one and, optionally, to customize the venue via the GUI and to assign desired sound pressure levels to predetermined sound spots and listening planes.
  • the acoustic simulation engine may use the venue data to execute an algorithm which is configured to search for sound pressure distributions in the virtual venue space which closely match the targeted sound levels defined by sound spots and/or listening planes.
  • a sound pressure distribution may be calculated on the basis of speaker set-up information or - in short a speaker set-up - comprising e.g. speaker locations, and a speaker direction and audio filter coefficients for each speaker as optimization parameters.
  • speaker locations are located within predefined speaker envelopes.
  • the audio filter coefficients may be used for configuring an electronic filter in each of the speakers.
  • the audio filter coefficient may define a digital filter comprising a predetermined response in the audio spectrum in terms of phase delays and amplitudes.
  • the 3D venue editor may be implemented as a software program comprising a number of GUI functions for modeling structures and objects which are used to graphically design a (partially) bounded virtual 3D venue space (e.g. floor(s), walls, ceiling(s) etc.), and objects within that space (e.g. a stage or tribunes), wherein acoustic properties such as absorption and/or reflection coefficients may be assigned to these elements and objects.
  • the 3D venue editor thus allows the modeling of a virtual 3D venue space, which closely matches the acoustical behavior of the real-life venue space.
  • the GUI rendered by the 3D venue editor is designed to easily assign predetermined areas, e.g. volumes and/or planes, in the virtual 3D venue space to certain desired sound pressure levels using user-configurable 3D wire frames.
  • the GUI may display the wire frame and allow a user to manipulate it in 3D space using e.g. multi-touch gestures or mouse manipulations. This way, a user may manipulate a wire-framed 3D volume (e.g. a sphere or a cube) in order to define a sound spot in the venue space and to assign a certain desirable sound pressure or sound pressure range to it.
  • a wire-framed 3D volume e.g. a sphere or a cube
  • the (partially) bounded virtual 3D venue space and the sound spots and/or listening planes in that space define a set of boundaries and conditions for a complex equation wherein sound pressure waves are emitted by configurable speakers into the virtual 3D venue space and wherein interference of these transmitted, diffracted and reflected sound pressure waves forms a complex sound pressure distribution in the 3D venue space.
  • This sound pressure distribution may be calculated for different frequency bands in the acoustic spectrum and for different speaker set-ups. Solutions matching the conditions, which are set by the sound spots and/or listening planes, may be determined and presented as possible optimal solutions. Finding speaker set-ups for a certain desired 3D sound pressure distribution taking into account all frequencies in the audio spectrum is a very complex problem for which no analytical solution exists.
  • best solutions for the above described problem are determined using an evolutionally algorithm (EA).
  • the EA algorithm parameters may be configured such that a set of best solutions, i.e. best speaker configurations, may be determined very rapidly.
  • each speaker set-up comprising speakers located at different speaker positions, wherein each speaker has a predetermined audio filter which may be defined in terms of phase delays and amplitudes for different frequency bands in the sound spectrum.
  • the filter generator may generate a set of filter coefficients for each speaker in the speaker configuration.
  • the filter generator may comprise an interface, which is configured to communicate with speakers in the real life venue.
  • the speaker interface may send the filter coefficients in a suitable communication protocol via a (wireless) network 120 or the Internet and a gateway 122 to the speakers.
  • a configurable speaker 118 1-4 may comprise one or more electro-acoustical transducers 116, and a signal-processing module 118 for digitally processing an audio signal received from an external amplifier system into a signal, which is suitable for driving the transducers.
  • the electro-acoustical transducers may comprise actuators or drivers of a particular type (e.g. piezoelectric, magnetostrictive and/or electrostatic) for transforming the audio signal into sound waves in the audible frequency range (typically between 20 Hz and 20 KHz).
  • a driver may be configured to reproduce either the full or a part of the audio spectrum, e.g.
  • a speaker may be characterized by a speaker directivity function, i.e. a predetermined function determining in which directions a speaker transmits sound pressure waves of a particular frequency.
  • the acoustic source may be digitally configurable in that sense that the signal processing module comprise one or more configurable digital audio filters 120 for controlling part of the acoustic properties (in particular the gain and magnitude) of the acoustic source in one or more predetermined frequency bands.
  • the signal-processing module may further comprise a communication module 122 for communication with the acoustic simulation system via a network, e.g. a (W)LAN, WiFi-network, and/or the Internet using a suitable control e.g. Ethernet.
  • a network e.g. a (W)LAN, WiFi-network, and/or the Internet using a suitable control e.g. Ethernet.
  • an IP address may be assigned to the signal-processing module so that an acoustic source may be easily accessed and programmed by the acoustic simulation system via the network.
  • the acoustic simulation system may be implemented as one or more software programs executed on one or more a (mobile) computing devices, including a PC, a smart phone, an electronic tablet, etc.
  • the simulation system executed on the computing device may send filter coefficients associated with a calculated speaker set-up via a network and/or the Internet to electronically configurable speakers in the real-life venue.
  • the simulation system may be distributed, wherein e.g. a client on a (mobile) computing device may execute the 3D venue editor as an app, which may subsequently send a predetermined venue model of to a server hosting the acoustic simulation engine.
  • the acoustic simulation engine executes the evolutionary algorithm for determining one or more optimized speaker set-ups. Thereafter, the filter coefficients associated with an optimized speaker set-up may be sent to the configurable speakers in the real-life venue.
  • acoustic sources may each receive an individual IP address in order to communicate filter information and/or a venue model (generated by a user using the 3D venue editor) to each other and/or to calculate (fully or partly) the filter information based on the information in the user-generated venue model.
  • the acoustic sources and simulation system(s) may behave like a distributed computing system.
  • Fig. 2 schematically depicts a flow diagram for defining boundaries and conditions used by the acoustic simulation system for simulating a speaker set-up in a venue according to an embodiment of the invention.
  • Fig. 2 depicts a computer-implemented method, which may be executed by the 3D venue editor program as described with reference to Fig. 1 .
  • a GUI is rendered, which allows a user to easily and intuitively define desired boundaries and conditions of a particular virtual 3D venue space.
  • a user may define an at least partially bounded virtual space on a graphical display, which substantially matches at least part of an acoustic space of a (real-life) venue.
  • a user may locate one or more (virtual) objects, e.g. a stage, a wall, people, etc. in said virtual space (step 204).
  • a user may further set locations in said space to desired sound pressure values by defining sound spots in said virtual space (step 206).
  • a user may use the GUI to manipulate the location and size of a 3D wire frame defining a volume of a sound spot, which may be associated with at least one sound pressure value or a range of sound pressure values (e.g. a minimum and/or a maximum sound pressure value).
  • a user may set speaker envelopes, i.e. volumes in the virtual space in which speakers may be located (step 208).
  • the speaker envelope is larger than the dimensions of a speaker so that during simulation an optimal speaker location in a speaker envelope may be determined.
  • a speaker envelope may be determined in a similar way as sound spots, i.e. by manipulating a 3D wire frame defining a volume of the speaker envelope.
  • (virtual) speakers may be selected and located in one or more speaker envelopes (step 210).
  • the acoustic properties of a virtual speaker may be selected such that they substantial match the acoustic properties of predetermined (real-life) electronically configurable speakers.
  • the user may select speakers from a list of predefined speakers stored in a speaker library. Thereafter, if the sound spots and speaker envelops comprising speakers are properly defined, the simulation system may execute an evolutionary algorithm for optimizing speaker settings (i.e. the location of a speaker in the speaker envelope, and the phase delay and amplitude for frequencies in the acoustic spectrum) of the virtual speakers a on the basis of said sound pressure values in said sound spots and speaker locations in said speaker envelopes (step 212).
  • speaker settings i.e. the location of a speaker in the speaker envelope, and the phase delay and amplitude for frequencies in the acoustic spectrum
  • simulation system allows simple and intuitive modeling on a venue of arbitrary design. Further, it allows optimization of an acoustical system located in that virtual space of the modeled venue such that speaker set-ups are generated which match a desired 3D sound pressure distribution.
  • steps 202-210 The process of defining a virtual acoustic space and the objects, sound spots and speaker envelopes located (i.e. steps 202-210 ) is illustrated hereunder in more detail with reference to Fig. 3-8 .
  • Fig. 3 depicts a GUI for building a 3D venue space according to an embodiment of the invention.
  • the GUI may comprise one or more (touch-screen or mouse) buttons 302-306 for selecting a particular graphical tool for building and modeling the 3D venue space.
  • the display may comprise a stage button 302 for activating a stage building function, an object button 304 for activating an object building function and a plane button 306 for activating a plane building function.
  • a user may graphically define predetermined planes, objects and stages in a virtual 3D space and assign particular acoustic properties thereto so that the basic bounds of the venue are defined.
  • a user may select a coordinate system 300 (e.g. conventional Cartesian coordinate system) and the orientation of that coordinate system in the 3D space. Then, a user may activate the plane building function for defining a first plane 308 normal to the z-direction and determine this plane to match the properties of a floor material e.g. concrete.
  • a coordinate system 300 e.g. conventional Cartesian coordinate system
  • a user may activate the plane building function for defining a first plane 308 normal to the z-direction and determine this plane to match the properties of a floor material e.g. concrete.
  • the stage building function may allow a user to select a basic stage object 312 from a predetermined number of preconfigured stage objects stored in a database associated with the acoustic simulation system. After having selected a stage object, the displayed object may be dragged to a predetermined position in the 3D venue space. After positioning the stage object, its dimensions may be manipulated. In order to accurately manipulate the size of the 3D object, touch sensitive manipulation markers 314 1-2 may be used. The markers may be used to resize the stage object. During manipulation, an object label 316 may be displayed to the user indicating basic dimensions.
  • the plane-, stage- and object building functions may be adapted to create planes and objects (i.e. geometrical objects and/or people) in the 3D venue space and to manipulate the shape, size and angle of the planes and objects by means of a touch-sensitive GUI.
  • the manipulation of the planes and objects in 3D space may be assisted by manipulation markers 314 1-2 , e.g. in the form of a predetermined shape (e.g. a triangular shape) which may be manipulated by touch or a mouse.
  • manipulation markers may be aligned to the axis of the coordinate system and are designed such that the user may easily manipulate a complicated 3D object using e.g. known multi-touch gestures.
  • an object label 316 may be displayed to the user indicating the coordinates and dimensions 320 of the plane and, optionally, the material and/or acoustic properties 318 of the object.
  • a menu may appear allowing a user to assign certain properties to the plane, e.g. material and/or acoustic properties.
  • the program may link this material to predetermined acoustic characteristics, e.g. reflection and absorption coefficients.
  • a user may repeat the above-described process in order to define predetermined planes and stage objects and other objects in the 3D venue space. For example, planes 310 1-6 , which are positioned under a predetermined angle from the floor plane may represent a virtual 3D structure, which closely matches a tribune.
  • Fig. 4 depicts a GUI for building a 3D venue space according to another embodiment of the invention.
  • a user may select the object building function 402 in order to create and located one or more objects 404,406 in the 3D venue space.
  • a menu structure may appear, allowing a user to select from basic object structures, e.g. a cuboid or spherical 3D frame wire structure, which can be manipulated using manipulation markers 408 into an 3D object of any desired shape and properties.
  • basic object structures e.g. a cuboid or spherical 3D frame wire structure, which can be manipulated using manipulation markers 408 into an 3D object of any desired shape and properties.
  • a cuboid object 404 may be selected and stretched to a desired length on the basis of the dimensions displayed in the object label 410.
  • Fig. 5 depicts a GUI for building a 3D venue space according to yet another embodiment of the invention.
  • a plane 504 associated with part of the tribune may be selected using the plane button 502.
  • a menu may be displayed comprising - amongst others - a button to activate a listening plane 505, which is a relatively thin, planar volume parallel to the plane of the tribune.
  • the listening plane may be formed by a planar volume positioned at a predetermined height (e.g., 1.26 meters, the height of a sitting person) above the plane of the tribune.
  • Manipulation markers 506 may modify the height of the listening plane.
  • a sound level button 522 may be used to assign predetermined sound pressure values, e.g. a range comprising a minimum and maximum sound pressures (e.g. 96-100 dB) to the listening plane.
  • the sound pressure value may be values minimal and maximal values associated with one or more predetermined sound frequency bands.
  • the influence of the amount of people on the acoustic properties of the tribune structure may be determined using a crowd fill factor 520.
  • a concrete tribune structure is for 75% occupied with (sitting) people having an average listening height of 1.26 meter.
  • the object parameters e.g. listening height, area and material, can be view via the object label 508.
  • the listening plane thus defines a particular condition for the acoustic simulation engine which will look for speaker settings which produce a sound pressure distribution in the venue space which meets the listening plane conditions as close as possible.
  • Fig. 6 depicts a GUI for building a 3D venue space according to a further embodiment of the invention.
  • selecting the silent button 602 may activate a menu for creating sound spots, i.e. predetermined volumes in the 3D venue space associated with a predetermined desired (targeted) maximum sound pressure. If a sound spot defines a volume of relative low sound pressures, such sound spot may be referred to as silent spot). Different sound spots may be selected using an "add" button 618.
  • the GUI may render a 3D wire-framed sphere in the graphical display.
  • the wire-framed sphere may comprise a centre point and axis, which may be parallel to the central coordinate system of the venue.
  • One or more wire frame circles may visualize the radius of the spherical volume of the sound spot.
  • a radial manipulation marker i.e. a triangular-shaped marker 626 pointing toward the centre point of the spherical volume may be used to manipulate the volume of the sound spot.
  • a sound spot label 620 may display the location and the radius of the sound spot.
  • a user may assign a particular maximum sound pressure value to the sound spot 610.
  • a user may assign further parameters, which may be of importance for the acoustical properties of a sound spot e.g. temperature 604 and humidity 606 of the medium (air) in the sound spot.
  • Multiple sound spots may be defined in order to assign predetermined sound pressure levels to particular volumes in of the venue space. Similar to the listening planes, these sound spots define particular conditions for the acoustic simulation engine, which will look for speaker set-ups which meet these conditions as close as possible.
  • the acoustic simulation engine may look for solutions, which meet the conditions as set by the listening planes and the sound spots.
  • the algorithm will select a group of solutions, i.e. speaker set-ups associated with sound pressure distributions, which match the conditions set by the sound spots/listening planes as close as possible.
  • a user may influence the selection of the solutions by the algorithm using a selector, e.g. a slider and a sliding bar 628,630, in which one sound spot or listening plane may be prioritized over another sound spot or listening plane.
  • a selector e.g. a slider and a sliding bar 628,630
  • the algorithm may prioritize solutions in which the silent spots are matched best.
  • the acoustic simulation engine will look for solutions, i.e. speaker set-ups and associated 3D sound pressure distributions, which meet the conditions as set by the listening planes and the sound spots.
  • the algorithm will select a group of solutions, which match the conditions as close as possible.
  • a user may influence the selection of the solutions using a slider and a sliding bar 628,630 in which one or more sound spots and/or listening planes may be more prioritized over other sound spots and/or listening plances.
  • the algorithm may prioritize solutions in which the silent spots are matched best.
  • Fig. 7 depicts a GUI for building a 3D venue space according to yet a further embodiment of the invention.
  • a user may define speaker envelopes by selecting the envelope button 702.
  • a speaker envelope defines a volume in the virtual 3D venue space in which it is desired to position the speakers.
  • This button may open a menu for creating speaker envelopes 704 1-2 .
  • the menu may comprise first envelope creating button 718 for the creation of an envelope centered in the middle (under) the stage and second envelope creating button 720 for creation of two (symmetrical) envelopes which may be positioned left and right from the stage.
  • the GUI may allow a user to drag the envelope to a desired location.
  • the size of the envelopes may be manipulated on the basis of manipulation markers 708 in a similar way as described above with respect to the sound spots and listening planes.
  • the location 728, dimensions 726 and angular orientation 724 of a speaker may be displayed in a speaker label 722.
  • a speaker envelope may comprise one or more speakers up to a predetermined maximum 714. Further, different types of speakers may be selected. Preferably, a user may select predetermined speakers from a speaker library, wherein each speaker may be associated with a predetermined speaker directivity function.
  • Fig. 8 depicts a GUI for building a 3D venue space according to an embodiment of the invention.
  • This embodiment depicts a more complex speaker envelope configuration comprising a speaker envelope 800 associated with speakers 802 1-3 , e.g. sub-woofers, which are positioned under the stage, speaker envelopes 804 1-2 for speakers 806 1-2 which are positioned next to the stage and speaker envelopes 808 1-2 for speakers 8010 1-2 which are positioned above the stage.
  • This figure illustrates that the GUI may allow a user to easy model a real-life venue comprising a complex speaker configuration.
  • Fig. 3-8 illustrate that the GUI functions of the 3D venue editor allows a user, a layman in acoustics, to build complex venues and speaker configurations in a simple and intuitive way.
  • the acoustic simulation engine may execute a simulation algorithm, in particular an evolutionary algorithm, wherein the 3D model defines the boundaries and conditions in which an optimization program may calculate the best solution for the 3D model.
  • the algorithm may initialize and define the parameters with are used by the evolutionary algorithm.
  • the user may define the number of speakers and assign an initial location to the speakers.
  • the algorithm may use this information in order to reserve memory space for defining a predetermined number of speakers in terms of speaker positions and filter coefficients.
  • Fig. 9 schematically depicts a flow diagram of a computer-implemented acoustic simulation method according one embodiment of the invention.
  • a random initial population of virtual speaker set-ups is generated wherein each speaker set-up represents a predetermined set-up of speakers in the virtual space.
  • Each speaker in a virtual speaker set-up is associated with a speaker location and a predetermined audio filter.
  • the audio filter may be defined by a predetermined number of filter coefficients.
  • the population is optimized on the basis of an evolutionary algorithm, preferably a differential evolutionary algorithm (step 904).
  • the evolutionary algorithm will optimize the population of speaker set-ups by forming one or more new speaker set-ups on the basis of existing speaker set-ups in the population and by executing a cost function (sometimes also referred to as a fitness function), which assigns a penalty or a score to the new speaker setting (step 906).
  • the cost function may assign a penalty or a score for differences between calculated and targeted sound characteristics, such as sound pressure, in sound spots, wherein the calculated sound characteristics are calculated on the basis of the new speaker setting.
  • a penalty or score may be assigned to differences between calculated speaker locations, which deviate from the user specified and common-sense constraints, such as speaker locations outside the speaker envelopes.
  • the assigned penalties and scores may be added to a total cost value, which is then used to evaluate a new speaker set up with respect to an existing speaker-set up in the population. Better scoring solutions (i.e. low penalties, high score) are replaced with earlier solutions in the population. This optimization process may be repeated until a predetermined condition is met (e.g. max number of iterations or optimized solutions match the desired 3D sound distribution within a certain accuracy.
  • a predetermined condition e.g. max number of iterations or optimized solutions match the desired 3D sound distribution within a certain accuracy.
  • one or more optimized speaker set-ups may be selected (step 908 ).
  • These one or more selected optimized speaker set-ups include filter coefficients, which may be sent to the (real-life) electronically configurable speakers (step 910 ).
  • the filter coefficients may be used to configure a digital audio filter in each of the speakers such that the speakers produce a desired sound pressure distribution.
  • Filter coefficients may be determined on the basis of known filter algorithms. For example, in one embodiment, a filter coefficients may be generated on the basis of a digital FIR filter algorithm as described in the article by Karam et. al., "Design of optimal digital FIR filters with arbitrary magnitude and phase responses", ISCAS 1996 . This particular algorithm may allow design of optimal Chebyshev digital FIR filters with arbitrary magnitude and phase specifications and is thus particular suited for use with the acoustical optimization method as described with reference to Fig. 9 .
  • the computer-implemented acoustic simulation method will be discussed hereunder in more detail on the basis of parts of pseudo-codes of the evolutionary algorithm.
  • the algorithm may start with the initialization of some important simulation parameters including, the frequency bands in the sound spectrum for which the sound pressure distributions are determined, the maximum number of speakers used during the simulation and the positions and dimensions of the speaker envelopes.
  • a directionality function may be associated with one or more of the speakers.
  • a directionality function may be associated with a particular speaker design.
  • the directionality function defines the speaker directivity, i.e. a speaker characteristic determining in which directions sound pressure waves of a particular frequency are transmitted.
  • the cost function in the algorithm revolves around a so-called Transfer Function.
  • This is a mathematical function, which takes the speaker positions, their associated filter settings (coefficients) and a position of interest in the 3D venue space as input and returns a sound pressure integrated over the sound spectrum. This calculated value is then compared with the desired sound pressure (integrated over the sound spectrum).
  • the directionality function of a speaker may be part of this Transfer Function, as is any diffusion. Basically the Transfer Function describes anything between the electrical current into the speaker and the pressure wave at the spatial point of interest. This includes behavior of a speaker, the speaker cabinet, speaker directionality, the medium through which sound pressure waves propagate (e.g. air, which can be modeled as being linear or non-linear diffusive medium), reflections and/or interferences.
  • speaker envelopes may be defined in the 3D venue space. Defining speaker envelopes using the GUI of the 3D venue editor is described in detail with reference to Fig. 7 and 8 .
  • solutions associated with speakers positioned in the speaker envelopes are strongly preferred over those solutions where speakers are outside the envelopes.
  • a speaker envelope may comprise one speaker or, alternatively, it may comprise several speakers. To that end, speaker dimensions may be defined so that an initial distribution of the speakers over the envelopes may be calculated.
  • Predetermined desired sound pressures may be set by defining predetermined volumes in the venue space and assigning predetermined one or more sound pressure values to these volumes and/or planes.
  • so-called sound spots may be defined in the 3D venue space, which are associated with a particular sound pressure, e.g. in one embodiment sound pressures targets may be selected from a range between 0 and approx. 110 dB.
  • a sound spot may be used to define a "silent region" i.e. a region in which the sound pressure stays below a predetermined maximum value.
  • a listening plane i.e. a sound spot with an audible sound pressure amplitude at listening height
  • a desired sound pressure may be assigned to this plane.
  • the algorithm will seek for 3D sound pressure distributions for which the calculated sound pressure at the locations of sound spots and/or listening planes closely match the desired sound targets.
  • each sound spot may be divided in a grid of cubes each having a predetermined position.
  • the algorithm will calculate sound pressures for each cube and compare the calculated maximum and minimum sound pressure values with the sound pressure values associated with the sound spot.
  • the grid size may determine the accuracy of the algorithm. The smaller the grid size, the more accurate a solution may be determined. Typically, the grid size will be a compromise between desired accuracy and computational costs.
  • An importance factor may be assigned to a sound spot or a listening plane.
  • the importance factor may be used by the simulation algorithm in order to make a selection between two or more "conflicting" solutions, e.g. first speaker settings which produce a sound pressure distribution closely matching the conditions of a first sound spot and second speaker settings which procedure a sound pressure distribution closely matching a second sound spot.
  • first speaker settings which produce a sound pressure distribution closely matching the conditions of a first sound spot
  • second speaker settings which procedure a sound pressure distribution closely matching a second sound spot.
  • the 3D venue editor may allow a user to assign an importance factor to one or more sound spots so that the sound spots are prioritized allowing the algorithm to select a particular solution which closely matches the conditions of a sound spot with a high importance factor.
  • an evolutionary algorithm may be initialized.
  • a differential evolution algorithm may be used in order to calculate solutions, i.e. one or more speaker set-ups, each being associated with a 3D sound pressure distribution, which closely match a desired sound pressure distribution as defined by the sound spots and/or listening planes, and to determine the speaker locations and audio filters associated with these solutions.
  • the algorithm will seek for sound pressure distributions within the virtual 3D venue space, which match the conditions as set by the user.
  • the venue may be modeled by elements defining a bounded 3D space, e.g. floor(s), walls, ceiling(s) etc., and objects within that space, e.g. a stage or tribunes, wherein acoustic properties such as absorption and/or reflection coefficients may be assigned to these elements and objects.
  • Each sound pressure distribution represents a solution of a number of interfering sound pressure waves generated by a speaker set-up in the venue space.
  • a speaker set-up is associated with predetermined a number of speakers, each being associated with a speaker position and a (digital) audio filter.
  • a audio filter may include phase delay and/or amplitude information.
  • Sound pressure waves are generated by the speakers, travelling through the medium, typically air, of the 3D venue space and reflected and/or diffracted via the walls, floors, ceiling and/or objects in the virtual venue space wherein reflection and diffraction is determined by the acoustic properties of the walls, floor, ceilings and/or objects and the medium.
  • a complex 3D sound pressure distribution in the virtual 3D venue space will be formed. In certain areas constructive interference will result in high sound pressure areas and in other areas destructive interference will result in low sound pressure areas. The amount, size and locations of these high and low sound pressure areas in the venue space depend on the particular speaker set-up.
  • Finding solutions i.e. virtual speaker set-ups associated with desired 3D sound pressure distributions
  • the evolutionary algorithm will look for solutions by evaluating the evolution of a population of virtual speakers set-ups on the basis of a cost function.
  • the evolutionary algorithm uses this cost function to optimize the problem of finding specific solutions, i.e. speaker set-ups, which are associated with 3D sound pressure distributions which match the conditions set for a predetermined virtual 3D venue as close as possible.
  • the algorithm may execute the evolution of a population of candidate solutions, i.e.
  • candidate speaker set-ups by generating predetermined changes (mutations) in certain speaker set-ups and creating new set-ups (mutants) on the basis of existing speaker set-ups and the mutated speaker set-up according to a predetermined mutation/recombination scheme.
  • a score or fitness may be associated with the existing and new speaker set-up and those having the best score or fitness are selected and kept.
  • the evolutionary process of mutations, recombination and selection is iteratively repeated until the solutions are evolved within a predetermined number of recombinations (iterations) into solutions, which closely match desired conditions in terms of sound spots and/or listening planes.
  • a first initial random population comprising a number of NP solutions, i.e. speaker set-ups
  • the parameters defining a particular speaker set-up i.e. the locations of the speakers and the phase delay information and the amplitude information per frequency band for each speaker may be represented in the form of a vector m.
  • m may have n parameters (m 1 ,...,m n ) wherein n represents the dimensionality of the acoustic problem to be optimized.
  • the initial speaker locations may be selected on the basis of the speaker envelopes and the (maximum allowable) speakers in a speaker envelope such that the evolutionary process starts with a population of speaker set-ups having speaker locations somewhere within the envelopes. Further, during the initialization phase, the maximum number of recombination cycles gen_max may be determined.
  • the predetermined changes (mutations) in speaker set-ups and the creation of new speaker set-ups on the basis of existing speaker set-ups and the mutated speaker set-ups (mutants) is implemented using a differential evolutionary (DE) algorithm.
  • the DE algorithm uses mutations, which are generated on the basis of differences between randomly selected solutions in the population.
  • This process is repeated for each vector in the population until one or more vectors in the population generate a sound pressure distribution which matches a desired sound pressure distribution (in terms of sound spots and/or listening planes) within a certain acceptance range or until a maximum number of recombinations (iterations) is executed.
  • F cost represents a cost function, which is configured to evaluate a vector to what extend the sound pressure distribution matches or differs from the desired sound pressure distribution as defined by the sound spots and/or listening planes and to what extend the speaker locations associated with a vector are within the desired speaker envelopes.
  • the cost function is configured to calculate sound pressure distributions, in particular sound pressure in the sound spots and/or listening planes on the basis of a vector in the population.
  • the function takes into account as much factors as possible that significantly influence the sound pressure, including the location of the speakers, the direction in which the speakers transmit the sound pressure waves into the venue space and/or the sound pressure waves, which are reflected and/or diffracted by objects in said venue space and/or boundaries of said venue space. Thereafter, the cost function assigns a cost score to the vector.
  • the cost function is further configured to compare calculated sound pressures and desired sound pressure targets in the sound spot and to assign penalties or a score for differences and matches respectively between the calculated and desired (targeted) sound pressures in a sound spot and for maxima in the frequency domain. Further, it may assign an additional penalty for speaker locations, which do not conform to the constraints as defined by the user, e.g. speaker positions which lie (partially) outside the speaker envelopes. In an embodiment, a score may also be assigned to the number of speakers used and/or for the filter length in order to minimize both and push the optimization towards more economically attractive solutions for the end-user.
  • a speaker set-up may be evaluated graphically as shown in Fig. 10 showing a simple virtual space similar to the one depicted in Fig. 6 comprising a simple stage and two speaker modules located next to the stage.
  • the program may illustrate the sound pressure distribution 1002 at certain listening planes for a predetermined frequency 1004 (in this case 63 Hz).
  • a selection tool e.g. a mouse or touch screen button, which may be manipulated by sliding the area in which the frequency is displayed up or down, so that sound pressure distributions associated with a certain audio frequency band may be graphically displayed.
  • the program may allow to graphically visualizing at least part of an optimized speaker setting. For example, in Fig. 11 the amplitude settings for a configured virtual speaker are shown wherein e.g. in a certain frequency range the amplitude of the signal is amplified (amplitude coefficients 1002) and in other ranges the amplitudes may be attenuated (amplitude coefficients 1004).
  • a user may manipulate the amplitude settings by frequency rules 1108-1112 and amplitude rules 1106.
  • One embodiment of the disclosure may be implemented as a program product for use with a computer system.
  • the program(s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media.
  • the computer-readable storage media can be a non-transitory storage medium.
  • Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state nonvolatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory, flash memory) on which alterable information is stored.
  • non-writable storage media e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state nonvolatile semiconductor memory
  • writable storage media e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory, flash memory

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)
  • Circuit For Audible Band Transducer (AREA)
EP12163054.5A 2012-04-03 2012-04-03 Simulation et configuration d'un système acoustique Active EP2648425B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP12163054.5A EP2648425B1 (fr) 2012-04-03 2012-04-03 Simulation et configuration d'un système acoustique

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP12163054.5A EP2648425B1 (fr) 2012-04-03 2012-04-03 Simulation et configuration d'un système acoustique

Publications (2)

Publication Number Publication Date
EP2648425A1 true EP2648425A1 (fr) 2013-10-09
EP2648425B1 EP2648425B1 (fr) 2019-06-19

Family

ID=46025442

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12163054.5A Active EP2648425B1 (fr) 2012-04-03 2012-04-03 Simulation et configuration d'un système acoustique

Country Status (1)

Country Link
EP (1) EP2648425B1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9811721B2 (en) 2014-08-15 2017-11-07 Apple Inc. Three-dimensional hand tracking using depth sequences
US10048765B2 (en) 2015-09-25 2018-08-14 Apple Inc. Multi media computing or entertainment system for responding to user presence and activity
WO2023281899A1 (fr) * 2021-07-09 2023-01-12 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Procédé de traitement d'informations, dispositif de traitement d'informations, et programme

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1205863A1 (fr) * 2000-11-14 2002-05-15 Honda R&D Europe (Deutschland) GmbH Optimisation à objectifs multiples
US20030007648A1 (en) * 2001-04-27 2003-01-09 Christopher Currell Virtual audio system and techniques
US20050220309A1 (en) * 2004-03-30 2005-10-06 Mikiko Hirata Sound reproduction apparatus, sound reproduction system, sound reproduction method and control program, and information recording medium for recording the program
US7206415B2 (en) 2002-04-19 2007-04-17 Bose Corporation Automated sound system designing
US20090144036A1 (en) 2007-11-30 2009-06-04 Bose Corporation System and Method for Sound System Simulation
US20100305725A1 (en) * 2009-05-28 2010-12-02 Dirac Research Ab Sound field control in multiple listening regions

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1205863A1 (fr) * 2000-11-14 2002-05-15 Honda R&D Europe (Deutschland) GmbH Optimisation à objectifs multiples
US20030007648A1 (en) * 2001-04-27 2003-01-09 Christopher Currell Virtual audio system and techniques
US7206415B2 (en) 2002-04-19 2007-04-17 Bose Corporation Automated sound system designing
US20050220309A1 (en) * 2004-03-30 2005-10-06 Mikiko Hirata Sound reproduction apparatus, sound reproduction system, sound reproduction method and control program, and information recording medium for recording the program
US20090144036A1 (en) 2007-11-30 2009-06-04 Bose Corporation System and Method for Sound System Simulation
US20100305725A1 (en) * 2009-05-28 2010-12-02 Dirac Research Ab Sound field control in multiple listening regions

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9811721B2 (en) 2014-08-15 2017-11-07 Apple Inc. Three-dimensional hand tracking using depth sequences
US10048765B2 (en) 2015-09-25 2018-08-14 Apple Inc. Multi media computing or entertainment system for responding to user presence and activity
US10444854B2 (en) 2015-09-25 2019-10-15 Apple Inc. Multi media computing or entertainment system for responding to user presence and activity
US11561621B2 (en) 2015-09-25 2023-01-24 Apple Inc. Multi media computing or entertainment system for responding to user presence and activity
WO2023281899A1 (fr) * 2021-07-09 2023-01-12 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Procédé de traitement d'informations, dispositif de traitement d'informations, et programme

Also Published As

Publication number Publication date
EP2648425B1 (fr) 2019-06-19

Similar Documents

Publication Publication Date Title
JP4651710B2 (ja) グラフィック・ユーザ・インタフェースの手段により空間音響再生システムにおける音響効果を生成及び処理するための装置及び方法
CN105264915B (zh) 混合控制台、音频信号发生器、用于提供音频信号的方法
EP2232484B1 (fr) Modélisation de caractéristiques de propagation d'onde dans un environnement
CN103065355B (zh) 一种实现智慧楼宇三维建模的方法、装置
EP2648425B1 (fr) Simulation et configuration d'un système acoustique
EP4101182A1 (fr) Amélioration de source audio virtuelle à réalité augmentée
US20230306953A1 (en) Method for generating a reverberation audio signal
Raghuvanshi et al. Interactive and Immersive Auralization
TWI830989B (zh) 使用有效的中間繞射路徑來渲染一音訊場景的設備及方法
WO2023182024A1 (fr) Procédé de commande de signal acoustique, procédé de génération de modèle d'apprentissage, et programme de commande de signal acoustique
WO2023246327A1 (fr) Procédé et appareil de traitement de signal audio, et dispositif informatique
TWI640983B (zh) 室內聲響效應模擬方法
KR102032803B1 (ko) 멀티미디어 컨텐츠를 위한 입체 음향 효과 제공 방법 및 상기 방법을 수행하는 멀티미디어 디바이스
WO2024190172A1 (fr) Procédé de traitement acoustique, dispositif de traitement acoustique, et programme de traitement acoustique
US12063491B1 (en) Systems and methods for generating device-related transfer functions and device-specific room impulse responses
Vorländer et al. Virtual room acoustics
WO2024122398A1 (fr) Procédé de traitement acoustique, dispositif de traitement acoustique et programme de traitement acoustique
WO2024225428A1 (fr) Procédé d'aide à la conception de disposition pour équipement acoustique, dispositif de traitement d'informations et programme
WO2023214515A1 (fr) Procédé de commande de génération de son, dispositif de production de son et programme de commande de génération de son
Feistel et al. Acoustic Modelling–Basics
WO2024122307A1 (fr) Procédé de traitement acoustique, dispositif de traitement acoustique, et programme de traitement acoustique
JP2024062690A (ja) 表示方法、情報処理装置、およびプログラム
JP4255421B2 (ja) 住宅の音シミュレーションシステム
Wang Augmented Reality in Room Acoustics: A Simulation Tool for Mobile Devices with Auditory Feedback
Pool et al. Ceiling Sound Reflectors to Optimize Acoustic Performance Using Parametric Tools

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20140409

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

17Q First examination report despatched

Effective date: 20140724

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: DUTCH & DUTCH B.V.

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190107

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012061083

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1147049

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190715

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190920

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1147049

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191021

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191019

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200224

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012061083

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG2D Information on lapse in contracting state deleted

Ref country code: IS

26N No opposition filed

Effective date: 20200603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200430

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200430

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200430

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200403

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200430

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200403

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240426

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240429

Year of fee payment: 13