EP2823650B1 - Audiowiedergabesystem - Google Patents

Audiowiedergabesystem Download PDF

Info

Publication number
EP2823650B1
EP2823650B1 EP12753717.3A EP12753717A EP2823650B1 EP 2823650 B1 EP2823650 B1 EP 2823650B1 EP 12753717 A EP12753717 A EP 12753717A EP 2823650 B1 EP2823650 B1 EP 2823650B1
Authority
EP
European Patent Office
Prior art keywords
audio rendering
rendering system
information
loudspeakers
portable electronic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12753717.3A
Other languages
English (en)
French (fr)
Other versions
EP2823650A1 (de
Inventor
Yue Lang
David Virette
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP2823650A1 publication Critical patent/EP2823650A1/de
Application granted granted Critical
Publication of EP2823650B1 publication Critical patent/EP2823650B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation

Definitions

  • the present invention relates to an audio rendering system, in particular a stereo or multichannel rendering system for rendering room acoustics, a method for calibrating an audio rendering system and a portable electronic device, in particular a Smartphone or a Tablet PC usable for calibration of an audio rendering system.
  • the optimization of loudspeaker and/or room rendering is a technology which aims at improving and/or correcting the audio rendering of non-standard loudspeaker layouts. Some examples of such optimization can be found in JP2000261900 , FR2850183 and EP2378795 .
  • the methods are based on adaptive modifications, i.e. filtering, gain, delay, equalization, etc. of the audio signal computed and applied for each channel which are used for the optimization of the rendering.
  • the adaptation of the rendering is usually based on the measurement of the audio signal which is actually received at the listening position. This optimal position is usually called the "sweet spot". This can be done by a directional microphone system, i.e.
  • FIG. 9 illustrates a 5.1 multichannel loudspeaker system 900 according to the recommendation ITU-R BS 775-1.
  • Three front loudspeakers (left L, center C, right R) are combined with two rear/side loudspeakers (left surround L S , right surround R S ).
  • the left and right frontal loudspeakers are placed at the extremities of an arc subtending 60° at the reference listening point 901.
  • Both side/rear loudspeakers L S , R S should be placed within the sectors from 100° to 120° from the center front reference. Precise location is not necessary.
  • Side/rear loudspeakers should be not closer to the listener than the frontal loudspeakers, unless compensating time delay is introduced.
  • the frontal loudspeakers should ideally be at a height approximately equal to that of the listener's ears.
  • Figure 10 illustrates a non-standardized 5.1 multichannel loudspeaker system 1000 which requires rendering adaptation and correction.
  • the three front loudspeakers L, C, R are not placed on a straight line base and the two rear/side loudspeakers L S , R S are located in different distances from the reference listening point 1001. Both side/rear loudspeakers L S , R S are not placed within the sectors from 100° to 120° from the center front reference. The left and right frontal loudspeakers are not placed at the extremities of the arc subtending 6o° at the reference listening point 1001.
  • WO 2011 139 502 A1 provides a method to instruct a processing system to carry out the method, which includes applying corrective filters directly in a portable media device to correct, e.g., equalize for the overall system comprising the portable media device and the playback system to which it is attached.
  • WO 2007 017 809 A1 introduces a device for processing audio data, wherein the device comprises a first microphone adapted to detect first audio data indicative of the first acoustic waves and a second microphone located in the second ear canal of the second ear of the human being to detect second acoustic waves.
  • US 2010 284 544 A1 relates to an apparatus for reproducing sound from original source signals, having a control device, a first speaker, and a second speaker.
  • the present invention is defined by a portable electronic device according to independent claim 1, an audio rendering system according to independent claim 6, and a method for calibrating an audio rendering system for audio rendering room acoustics according to independent claim 13.
  • the invention is based on the finding that by using the available microphones (two or more) of a mobile device, e.g. a mobile device 1100 depicted in Fig. 11 or any other mobile device like a Smartphone or a Tablet PC for example, associated with an adapted user interface allows a flexible and adaptive multi-loudspeaker calibration of the audio rendering system comprising thereof.
  • a mobile device e.g. a mobile device 1100 depicted in Fig. 11 or any other mobile device like a Smartphone or a Tablet PC for example
  • the audio rendering calibration system based on the mobile device comprises a mobile device with at least two microphones, synchronization means between the rendering system and the mobile device, e.g.
  • WiFi, docking station, etc. rendering means for rendering of test or training signals
  • analysis means for analysis of the loudspeaker rendering system e.g. position, frequency response, etc, and/or room characteristics
  • compensation means for adaptive compensation of the audio rendering based on the analysis step performed by the analysis means.
  • the invention relates to an audio rendering system for audio rendering room acoustics according to claim 1.
  • the audio rendering system allows a dynamic calibration, in particular a calibration when the user moves or changes his listening position, based on the synchronization between the rendering system and the mobile device, synchronized by WiFi, docking station, etc., for example.
  • the audio rendering system provides a control interface to the user and allows the user to control the calibration of the rendering system.
  • the user can directly feedback on the performance of the calibration, modify the detected position of the loudspeaker and thus improve the performance of the audio rendering system.
  • control device is configured to start calibrating the audio rendering system by sending a training signal through the plurality of loudspeakers.
  • the calibration of the audio rendering is started at a predetermined point in time.
  • the mobile device is configured to record the training signal and to send the recorded training signal and/or information based thereupon to the control device.
  • the control device When the mobile device records the training signal and sends the recorded training signal and/or information based thereupon to the control device, the control device is able to improve its audio rendering by exploiting data from a position inside the room.
  • the room may be a closed room such as a theater, a concert hall or a small office room or it may be an open room such as an arena or a football stadium.
  • the recorded training signal is sent to the control device, complexity of the mobile device is low as processing is performed in the control device.
  • information based on the recorded training signal is sent to the control device, data being transmitted can be kept low, as pre-processing is performed in the mobile device and only key performance data is sent to the control device.
  • the information based on the recorded training signal comprises at least one of the following information: information on positions of the plurality of loudspeakers, information on a room characteristic, information on gains in signal paths between the plurality of loudspeakers and the at least two microphones, information on delays in signal paths between the plurality of loudspeakers and the at least two microphones and information on transfer function in signal paths between the plurality of loudspeakers and the at least two microphones.
  • control device is configured to adapt the audio rendering of the audio rendering system based on the recorded training signal and/or information based thereupon received from the mobile device.
  • Audio rendering is not limited to static environments, it performs well in dynamic environments by an adaptive rendering process based on the recorded training signal.
  • control device is located in a docking station of the mobile device.
  • the control device does not require a separate unit, it can be integrated in an existing unit such as a docking station. Implementation effort and costs can be kept low when an existing docking station is enhanced by control device functionality.
  • each of the at least two microphones is located in the middle of a different edge of the mobile device.
  • the invention relates to a portable electronic device according to claim 8.
  • the portable electronic device allows a dynamic calibration of an audio rendering system, in particular a calibration when the user moves or changes his listening position, based on the synchronization between the audio rendering system and the portable electronic device.
  • the user is allowed to control the calibration of the audio rendering system.
  • the user can directly feedback on the performance of the calibration, modify the detected position of the loudspeaker and thus improve the performance of the audio rendering system.
  • the processor comprises an analysis module configured for analyzing the recorded training signal to provide the information based on the recorded training signal.
  • the processor comprises an analysis module for analyzing the recorded training signal
  • analyzed information based on the recorded training signal can be sent to the control device, thereby keeping the load of the interface between portable electronic device and the control device low. Only key performance data found, by the analysis module is sent to the control device.
  • the analysis module is configured to provide as information based on the recorded training signal at least one of the following information: information on a room characteristic, information on room positions of the loudspeakers, information on gains in signal paths between the loudspeakers and the at least two microphones, information on delays in signal paths between the loudspeakers and the at least two microphones, and information on transfer function in signal paths between the loudspeakers and the at least two microphones.
  • the portable electronic device comprises a synchronization circuit configured to synchronize the recording of the training signal and the transmission of the recorded training signal and/or the information based thereupon with a control device initiating the calibration of the audio rendering system.
  • Synchronizing the calibration process improves the accuracy of audio rendering.
  • the portable electronic device comprises a graphical user interface configured for allowing a user to control the calibration of the audio rendering system by inputting information on a room characteristic used for adapting the audio rendering system.
  • the graphical user interface enables the direct interaction with the user who can indicate if a loudspeaker which is positioned on the front should be actually positioned on the rear. For instance, if the mobile device is equipped with only two microphones, the analysis system can only discriminate the position of the loudspeaker according to one direction, e.g. left/right or front/back depending on the position of the microphones on the mobile device. The final set of loudspeaker positions and necessary rendering adaptation are then determined according to the recording and additional user information. The user, however, cannot provide the sufficient information on potential delay and equalization between channels. Recording is always necessary to achieve the optimal rendering.
  • the portable electronic device may comprise an App, i.e., an application tool on a mobile device with a graphic user interface, which can be installed in the mobile OS (operational system) directly.
  • the user can directly get the feedback of the rendering system on the display, e.g. information on loudspeaker position, configuration of the room, etc. and control the adaptation of the rendering system without any difficulties.
  • the portable electronic device further comprises earphones, wherein the processor is configured to record the training signal received by microphones integrated in the earphones for binaurally capturing the training signal at an ear canal of a user using the portable electronic device or to record the training signal by a combination of the at least two microphones and the microphones integrated in the earphones.
  • audio rendering can directly exploit information of the sweet spot.
  • the audio rendering is improved and adapted to the listener's position.
  • the invention relates to a method for calibrating an audio rendering system for audio rendering room acoustics according to claim 14.
  • This direct interaction with the user allows to always providing the rendering sweet spot to the listeners. It improves the multichannel audio rendering at the listening position for applications like Home Cinema, Home Theater, sound bar, docking station etc. based on an easy to use calibration tool based on mobile device as analysis/control tool.
  • the method further comprises: initiating the calibrating by the mobile device by transmitting an initiation signal to the control device.
  • the calibration of the audio rendering is acknowledged by the mobile device and thus started at a predetermined point in time.
  • DSP Digital Signal Processor
  • ASIC application specific integrated circuit
  • the invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof.
  • the first one corresponds to the configuration depicted in Fig. 11 but the seoond microphone 1107 depicted in Fig. 11 is missing. It has only one single microphone 1113 which is used for communication and any other mono sound pickup. This first configuration is based on omnidirectional microphone and cannot provide any stereo image.
  • the other configuration is illustrated in Fig. 11 and uses two omnidirectional microphones, the main microphone 1113 and the auxiliary microphone 1107.
  • the main microphone 1113 is used for the sound pick up, i.e. for communication application as well as simple audio/video recording.
  • the auxiliary microphone 1107 is used for noise cancellation and gain control.
  • An omnidirectional microphone needs only one hole in the terminal as opposed to directional microphones. Indeed, the omnidirectional microphone offers a uniform directivity pattern in all the direction, equivalent to a sphere, and only one hole is then required for the microphone housing 1115. The size of the hole and the actual microphone housing 1115 will affect the directivity of the complete system, but a single hole is sufficient for perfect sound pickup. Two omnidirectional microphones mounted with a spacing of several centimeters can also be used in order to obtain a stereo recording.
  • Fig. 1 shows a schematic diagram of a basic audio rendering system 100 according to an implementation form.
  • a user or listener 109 sitting on a sofa holds a mobile device 105, e.g. a Smartphone or a tablet PC including at least two microphones 107.
  • the mobile device 105 is connected to a control device 103, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 103 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 100 comprises the following elements:
  • the analysis and control devices are implemented in separated devices. In an alternative implementation form, the analysis and control devices are implemented in the same device.
  • the adaptation of the multi-loudspeaker audio rendering can be directly calculated in the analysis device if the test signal is known a priori by this device.
  • the user 109 of the mobile device 105 performs the following steps which interact with the control device 103:
  • the mobile device 105 performs the computation of adaptation parameters such as delay, gains, filters, equalizer, etc. and sends these parameters to the control device 103.
  • adaptation parameters such as delay, gains, filters, equalizer, etc.
  • the computation of the adaptation parameters are then performed in the rendering system, i.e., in the control device 103 in order to select the most appropriate processing, e.g., post processing of the audio channel signal, or adaptation in the coded domain.
  • Fig. 2 shows a schematic diagram of an audio rendering system 200 with the user 209 being outside the system 200 according to an implementation form.
  • a user or listener 209 sitting on a sofa outside the audio system holds a mobile device 205, e.g. a Smartphone or a tablet PC including at least two microphones 207.
  • the mobile device 205 is connected to a control device 203, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 203 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 200 comprises the following elements: a plurality of loudspeakers 201, a mobile device 205 with at least two omnidirectional or directional microphones, a synchronization means between the control device 203 and the mobile device 205, an analysis device of the audio rendering system and/or room characteristics and a control device 103 for the adaptive compensation of the audio rendering based on the analysis of the analysis device.
  • the calibration is performed analogously to the procedure described with respect to Fig. 1 .
  • the audio rendering system 200 is able to calibrate the system if the user 209 is not located at the sweet spot of the audio rendering system even if the user 209 is sitting outside the audio system.
  • Fig. 3 shows a schematic diagram of an audio rendering system 300 with all loudspeakers being positioned in front of the user according to an implementation form.
  • the user 309 holds a mobile device 305, e.g. a Smartphone or a tablet PC including at least two microphones 307.
  • the mobile device 305 is connected to a control device 303, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 303 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 300 comprises the following elements: a plurality of loudspeakers 301 arranged in line in front of the user 309, a mobile device 305 with at least two omnidirectional or directional microphones 307, synchronization means between the control device 303 and the mobile device 305, analysis device of the audio rendering system and/or room characteristics and a control device 303 for the adaptive compensation of the audio rendering based on the analysis of the analysis device.
  • the calibration is performed analogously to the procedure described with respect to Fig. 1 .
  • the audio rendering system 300 is able to calibrate the system even if the loudspeakers 301 are arranged in front of the user 309 and if there are no surround loudspeakers available in the audio system.
  • Fig. 4 shows a schematic diagram of an audio rendering system 400 with calibration based on a docking station 401 according to an implementation form.
  • a user or listener 409 is sitting on a sofa behind a docking station 401 comprising at least two loudspeakers.
  • the user 409 holds a mobile device 405, e.g. a Smartphone or a tablet PC including at least two microphones 407.
  • the mobile device 405 is connected to a control device 403, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 403 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 400 comprises the following elements: loudspeakers integrated in a docking station 401 arranged in front of the user 409, a mobile device 405 with at least two omnidirectional or directional microphones 407, synchronization means between the control device 403 and the mobile device 405, analysis device of the audio rendering system and/or room characteristics and a control device 403 for the adaptive compensation of the audio rendering based on the analysis of the analysis device.
  • Docking station 401 and control device 403 are separate units.
  • the calibration is performed analogously to the procedure described with respect to Fig. 1 .
  • the audio rendering system 400 is able to calibrate the system even if there are only two loudspeakers integrated in the docking station 401 arranged in front of the user 409 and if there are no other front loudspeakers or surround loudspeakers available in the audio system.
  • Fig. 5 shows a schematic diagram of a microphone arrangement on a Tablet PC 500 according to an implementation form.
  • the Tablet PC 500 comprises a number of four microphones 507 arranged in the middle of each edge of the tablet in order to better discriminate or distinguish the directions of the sounds.
  • the tablet PC 500 corresponds to the mobile device described above with respect to Figures 1 to 4 .
  • the table PC 500 is adapted to perform the calibration analogously to the procedure described with respect to Fig. 1 .
  • Fig. 6 shows a schematic diagram of an audio rendering system 600 with calibration based on a docking station integrating a control device according to an implementation form.
  • a user or listener 609 sitting on a sofa holds a mobile device 605, e.g. a Smartphone or a tablet PC including at least two microphones 607.
  • the mobile device 605 is connected to a control device 603, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 603 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 600 comprises the following elements:
  • the user 609 of the mobile device 605 performs the following steps which interact with the control device in the docking station 601:
  • the user 609 of the mobile device 605 performs the following steps which interact with the control device in the docking station 601:
  • the user 609 of the mobile device 605 performs the following steps which interact with the control device in the docking station 601:
  • the user 609 of the mobile device 605 performs the following steps which interact with the control device in the docking station 601:
  • Fig. 7 shows a schematic diagram of an audio rendering system 700 with calibration based on microphones installed in headphones of a listener according to an implementation form.
  • a user or listener 709 is sitting on a sofa and holds a mobile device 705, e.g. a Smartphone or a tablet PC including at least two microphones 707.
  • the user 709 carries earphones with stereo microphones 713.
  • the mobile device 705 is connected to a control device 703, e.g. a set top box or an amplifier through a wireless or a wire line connection.
  • the control device 703 is performing the rendering adaptation, i.e., application of gains, delays, filters, etc.
  • the audio rendering system 700 comprises the following elements: front loudspeakers 701 and surround loudspeakers 711, a mobile device 705 with at least two omnidirectional or directional microphones 707, synchronization means between the control device 703 and the mobile device 705, analysis device of the audio rendering system and/or room characteristics and a control device 703 for the adaptive compensation of the audio rendering based on the analysis of the analysis device.
  • the calibration is performed analogously to the procedure described with respect to Fig. 1 .
  • the audio rendering system 700 is based on the use of stereo microphones 713 installed on the earphones or headphones and which are connected to the mobile device 705.
  • this alternative scenario is based on binaural recording made through earphones or headphones with microphones 713 mounted on each side.
  • Microphones 713 are directly integrated in the earphones/headphones capturing the audio signal directly at the ear canal.
  • the user 709 connects the earphones/headphones to the mobile device 705 and the sound recording is done by this binaural microphone 713.
  • This implementation form is advantageously based on recording which is directly representative of human perception of the multi-loudspeaker rendering system.
  • the rendering adaptation is directly based on the recording at the user's 709 ears which ensure that the adaptation, if done properly, will perfectly reconstruct the optimal multichannel signal at the ears of the listener709.
  • the analysis device combines the recording by the microphones 713 integrated to the earphones with the recording by the microphones 707 of the mobile device 705. This combined recording provides a larger area of adaptation and limits the over tuning of the calibration algorithm to the user ears position.
  • Fig. 8 shows a schematic diagram of a method 800 for calibrating an audio rendering system according to an implementation form.
  • the method 800 comprises recording 801 a training signal by at least two microphones of a mobile device being located in a room, the training signal being transmitted through a plurality of loudspeakers located in the room.
  • the method 800 comprises transmitting 803 the recorded training signal and/or information based thereupon to a control device configured for calibrating the audio rendering system.
  • the method 800 comprises initiating the calibrating by the mobile device by transmitting an initiation signal to the control device.
  • the present disclosure also supports a computer program product including computer executable code or computer executable instructions that, when executed, causes at least one computer to execute the performing and computing steps described herein.
  • the present disclosure also supports a system configured to execute the performing and computing steps described herein.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)

Claims (14)

  1. Tragbare elektronische Vorrichtung (705), die Folgendes aufweist:
    mindestens zwei Mikrofone (707);
    Ohrhörer (713); und
    einen Prozessor, ausgebildet zum Aufzeichnen eines Trainingssignals, gesendet durch mehrere Lautsprecher (701, 711) und empfangen durch eine Kombination der mindestens zwei Mikrofone (707) und von in die Ohrhörer (713) integrierten Mikrofonen, und zum Senden des aufgezeichneten Trainingssignals und/oder von darauf basierenden Informationen über eine Sendeschnittstelle zur Kalibrierung eines Audiowiedergabesystems (700).
  2. Tragbare elektronische Vorrichtung (705) nach Anspruch 1, wobei der Prozessor ein Analysemodul aufweist, das ausgebildet ist zum Analysieren des aufgezeichneten Trainingssignals zum Bereitstellen der Informationen basierend auf dem aufgezeichneten Trainingssignal.
  3. Tragbare elektronische Vorrichtung (705) nach Anspruch 1 oder Anspruch 2, wobei das Analysemodul ausgebildet ist zum Bereitstellen von Informationen basierend auf dem aufgezeichneten Trainingssignal, wobei die darauf basierenden Informationen zumindest eine der folgenden Informationen aufweisen:
    Informationen zu einer Raumcharakteristik,
    Informationen zu Raumpositionen der Lautsprecher (701, 711),
    Informationen zu Verstärkungen in Signalpfaden zwischen den Lautsprechern und den mindestens zwei Mikrofonen (707),
    Informationen zu Verzögerungen in Signalpfaden zwischen den Lautsprechern und den mindestens zwei Mikrofonen (707), und
    Informationen zu einer Übertragungsfunktion in Signalpfaden zwischen den Lautsprechern und den mindestens zwei Mikrofonen (707).
  4. Tragbare elektronische Vorrichtung (705) nach einem der Ansprüche 1 bis 3, eine Synchronisierungsschaltung aufweisend, die ausgebildet ist zum Synchronisieren der Aufzeichnung des Trainingssignals und der Übertragung des aufgezeichneten Trainingssignals und/oder von darauf basierenden Informationen mit einer Steuervorrichtung (703), die die Kalibrierung des Audiowiedergabesystems einleitet.
  5. Tragbare elektronische Vorrichtung (705) nach einem der Ansprüche 1 bis 4, umfassend eine grafische Benutzerschnittstelle, die dazu ausgebildet ist, einem Benutzer zu ermöglichen, die Kalibrierung des Audiowiedergabesystems zu steuern durch Eingeben von Informationen zur Raumcharakteristik, die zum Anpassen des Audiowiedergabesystems verwendet werden.
  6. Audiowiedergabesystem (700) zur Audiowiedergabe von Raumakustik, das Folgendes aufweist:
    die mehreren Lautsprecher (701, 711);
    die tragbare elektronische Vorrichtung (705) nach einem der Ansprüche 1 bis 5, wobei die tragbare elektronische Vorrichtung (705) an einer Raumposition befindlich ist; und
    die Steuervorrichtung (703), gekoppelt mit den mehreren Lautsprechern (701, 711) und gekoppelt mit der tragbaren elektronischen Vorrichtung (705), wobei die Steuervorrichtung (703) ausgebildet ist zum Kalibrieren des Audiowiedergabesystems (700) basierend auf der Raumposition der tragbaren elektronischen Vorrichtung (705) und basierend auf Raumpositionen der Lautsprecher (701, 711) durch Evaluieren von Informationen der mindestens zwei Mikrofone (707) der tragbaren elektronischen Vorrichtung (705).
  7. Audiowiedergabesystem (700) nach Anspruch 6, wobei die Steuervorrichtung (703) ausgebildet ist zum Starten der Kalibrierung des Audiowiedergabesystems (700) durch Senden des Trainingssignals durch die mehreren Lautsprecher (701, 711).
  8. Audiowiedergabesystem (700) nach Anspruch 7, wobei die tragbare elektronische Vorrichtung (705) ausgebildet ist zum Aufzeichnen des Trainingssignals und zum Senden des aufgezeichneten Trainingssignals und/oder von darauf basierenden Informationen an die Steuervorrichtung (703).
  9. Audiowiedergabesystem (700) nach Anspruch 8, wobei die Informationen basierend auf dem aufgezeichneten Trainingssignal zumindest eine der folgenden Informationen aufweisen:
    Informationen zu den Raumpositionen der mehreren Lautsprecher (701, 711), Informationen zur Raumcharakteristik,
    Informationen zu den Verstärkungen in den Signalpfaden zwischen den mehreren Lautsprechern (701, 711) und den mindestens zwei Mikrofonen (707),
    Informationen zu den Verzögerungen in den Signalpfaden zwischen den mehreren Lautsprechern (701, 711) und den mindestens zwei Mikrofonen (707), und
    Informationen zur Übertragungsfunktion in den Signalpfaden zwischen den mehreren Lautsprechern (701, 711) und den mindestens zwei Mikrofonen (707).
  10. Audiowiedergabesystem (700) nach Anspruch 8 oder Anspruch 9, wobei die Steuervorrichtung (703) ausgebildet ist zum Anpassen der Audiowiedergabe des Audiowiedergabesystems (700) basierend auf dem aufgezeichneten Trainingssignal und/oder von darauf basierenden Informationen, empfangen durch die tragbare elektronische Vorrichtung (705).
  11. Audiowiedergabesystem (700) nach einem der Ansprüche 6 bis 10, wobei die Steuervorrichtung (703) in einer Andockstation (601) der tragbaren elektronischen Vorrichtung (605) befindlich ist.
  12. Audiowiedergabesystem (700) nach einem der Ansprüche 6 bis 11, wobei jedes der mindestens zwei Mikrofone (707) in der Mitte einer anderen Kante der tragbaren elektronischen Vorrichtung (705) befindlich ist.
  13. Verfahren (800) zum Kalibrieren des Audiowiedergabesystems (700) nach einem der Ansprüche 6 bis 12 zur Audiowiedergabe von Raumakustik, wobei das Verfahren Folgendes aufweist:
    Aufzeichnen (801) des Trainingssignals durch die tragbare elektronische Vorrichtung (705) nach einem der Ansprüche 1 bis 5, befindlich in einem Raum, wobei das Trainingssignal durch die mehreren Lautsprecher (701, 711), befindlich in dem Raum, gesendet wird;
    Senden (803) des aufgezeichneten Trainingssignals und/oder von darauf basierenden Informationen an die Steuervorrichtung (703), ausgebildet zum Kalibrieren des Audiowiedergabesystems (700).
  14. Verfahren (800) nach Anspruch 13, das ferner Folgendes aufweist:
    Einleiten des Kalibrierens durch die tragbare elektronische Vorrichtung (705) durch Senden eines Einleitungssignals an die Steuervorrichtung (703).
EP12753717.3A 2012-08-29 2012-08-29 Audiowiedergabesystem Active EP2823650B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2012/066775 WO2014032709A1 (en) 2012-08-29 2012-08-29 Audio rendering system

Publications (2)

Publication Number Publication Date
EP2823650A1 EP2823650A1 (de) 2015-01-14
EP2823650B1 true EP2823650B1 (de) 2020-07-29

Family

ID=46785413

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12753717.3A Active EP2823650B1 (de) 2012-08-29 2012-08-29 Audiowiedergabesystem

Country Status (2)

Country Link
EP (1) EP2823650B1 (de)
WO (1) WO2014032709A1 (de)

Families Citing this family (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9084058B2 (en) 2011-12-29 2015-07-14 Sonos, Inc. Sound field calibration using listener localization
US9219460B2 (en) 2014-03-17 2015-12-22 Sonos, Inc. Audio settings based on environment
US9106192B2 (en) 2012-06-28 2015-08-11 Sonos, Inc. System and method for device playback calibration
US9690271B2 (en) 2012-06-28 2017-06-27 Sonos, Inc. Speaker calibration
US9690539B2 (en) 2012-06-28 2017-06-27 Sonos, Inc. Speaker calibration user interface
US9706323B2 (en) 2014-09-09 2017-07-11 Sonos, Inc. Playback device calibration
US9264839B2 (en) 2014-03-17 2016-02-16 Sonos, Inc. Playback device configuration based on proximity detection
EP2928216A1 (de) * 2014-03-26 2015-10-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und verfahren für bildschirmbezogene audioobjekt-neuabbildung
MX358769B (es) 2014-03-28 2018-09-04 Samsung Electronics Co Ltd Método y aparato para emitir una señal acústica, y medio de grabación legible en computadora.
US10051364B2 (en) * 2014-07-03 2018-08-14 Qualcomm Incorporated Single channel or multi-channel audio control interface
US9516444B2 (en) * 2014-07-15 2016-12-06 Sonavox Canada Inc. Wireless control and calibration of audio system
US9952825B2 (en) 2014-09-09 2018-04-24 Sonos, Inc. Audio processing algorithms
US9910634B2 (en) 2014-09-09 2018-03-06 Sonos, Inc. Microphone calibration
US9891881B2 (en) 2014-09-09 2018-02-13 Sonos, Inc. Audio processing algorithm database
US10127006B2 (en) 2014-09-09 2018-11-13 Sonos, Inc. Facilitating calibration of an audio playback device
EP3018917B1 (de) 2014-11-06 2016-12-28 Axis AB Verfahren und System zur Audiokalibrierung einer Audiovorrichtung
US20160239255A1 (en) * 2015-02-16 2016-08-18 Harman International Industries, Inc. Mobile interface for loudspeaker optimization
WO2016172593A1 (en) 2015-04-24 2016-10-27 Sonos, Inc. Playback device calibration user interfaces
US10664224B2 (en) 2015-04-24 2020-05-26 Sonos, Inc. Speaker calibration user interface
US10136234B2 (en) * 2015-05-14 2018-11-20 Harman International Industries, Incorporated Techniques for autonomously calibrating an audio system
US9538305B2 (en) 2015-07-28 2017-01-03 Sonos, Inc. Calibration error conditions
EP3351015B1 (de) 2015-09-17 2019-04-17 Sonos, Inc. Erleichtern der kalibrierung einer audiowiedergabevorrichtung
US9693165B2 (en) 2015-09-17 2017-06-27 Sonos, Inc. Validation of audio calibration using multi-dimensional motion check
US9743207B1 (en) 2016-01-18 2017-08-22 Sonos, Inc. Calibration using multiple recording devices
US11106423B2 (en) 2016-01-25 2021-08-31 Sonos, Inc. Evaluating calibration of a playback device
US10003899B2 (en) 2016-01-25 2018-06-19 Sonos, Inc. Calibration with particular locations
WO2017143067A1 (en) 2016-02-19 2017-08-24 Dolby Laboratories Licensing Corporation Sound capture for mobile devices
US11722821B2 (en) 2016-02-19 2023-08-08 Dolby Laboratories Licensing Corporation Sound capture for mobile devices
US9860662B2 (en) 2016-04-01 2018-01-02 Sonos, Inc. Updating playback device configuration information based on calibration data
US9864574B2 (en) 2016-04-01 2018-01-09 Sonos, Inc. Playback device calibration based on representation spectral characteristics
US9763018B1 (en) 2016-04-12 2017-09-12 Sonos, Inc. Calibration of audio playback devices
US10446166B2 (en) 2016-07-12 2019-10-15 Dolby Laboratories Licensing Corporation Assessment and adjustment of audio installation
US9860670B1 (en) 2016-07-15 2018-01-02 Sonos, Inc. Spectral correction using spatial calibration
US9794710B1 (en) 2016-07-15 2017-10-17 Sonos, Inc. Spatial audio correction
US10372406B2 (en) 2016-07-22 2019-08-06 Sonos, Inc. Calibration interface
US10459684B2 (en) 2016-08-05 2019-10-29 Sonos, Inc. Calibration of a playback device based on an estimated frequency response
US10200800B2 (en) 2017-02-06 2019-02-05 EVA Automation, Inc. Acoustic characterization of an unknown microphone
US10377486B2 (en) 2017-12-07 2019-08-13 Harman International Industries, Incorporated Drone deployed speaker system
US10225656B1 (en) 2018-01-17 2019-03-05 Harman International Industries, Incorporated Mobile speaker system for virtual reality environments
US10837944B2 (en) 2018-02-06 2020-11-17 Harman International Industries, Incorporated Resonator device for resonance mapping and sound production
US11206484B2 (en) 2018-08-28 2021-12-21 Sonos, Inc. Passive speaker authentication
US10299061B1 (en) 2018-08-28 2019-05-21 Sonos, Inc. Playback device calibration
WO2021010884A1 (en) * 2019-07-18 2021-01-21 Dirac Research Ab Intelligent audio control platform
US10734965B1 (en) 2019-08-12 2020-08-04 Sonos, Inc. Audio calibration of a portable playback device
JP7461771B2 (ja) * 2020-03-26 2024-04-04 株式会社ディーアンドエムホールディングス マルチチャンネルオーディオシステム、マルチチャンネルオーディオ装置、プログラム、およびマルチチャンネルオーディオ再生方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000261900A (ja) 1999-03-09 2000-09-22 Sony Corp 音場補正方法および音響装置。
FR2850183B1 (fr) 2003-01-20 2005-06-24 Remy Henri Denis Bruno Procede et dispositif de pilotage d'un ensemble de restitution a partir d'un signal multicanal.
WO2007017809A1 (en) * 2005-08-05 2007-02-15 Koninklijke Philips Electronics N.V. A device for and a method of processing audio data
KR100930835B1 (ko) * 2008-01-29 2009-12-10 한국과학기술원 음향 재생 장치
JPWO2010073336A1 (ja) 2008-12-25 2012-05-31 パイオニア株式会社 音場補正装置
JP2013530420A (ja) * 2010-05-06 2013-07-25 ドルビー ラボラトリーズ ライセンシング コーポレイション 可搬型メディア再生装置に関するオーディオ・システム等化処理

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP2823650A1 (de) 2015-01-14
WO2014032709A1 (en) 2014-03-06

Similar Documents

Publication Publication Date Title
EP2823650B1 (de) Audiowiedergabesystem
US9877133B2 (en) Sound collection and reproduction system, sound collection and reproduction apparatus, sound collection and reproduction method, sound collection and reproduction program, sound collection system, and reproduction system
US7123731B2 (en) System and method for optimization of three-dimensional audio
JP5526042B2 (ja) 音響システム及びサウンドを提供する方法
EP3619921B1 (de) Audio prozessor, system, verfahren und computer-programm für audio-wiedergabe
US8638959B1 (en) Reduced acoustic signature loudspeaker (RSL)
CN101129088B (zh) 具有增强的立体声图像的便携式设备
US20110091055A1 (en) Loudspeaker localization techniques
US20110188662A1 (en) Method of rendering binaural stereo in a hearing aid system and a hearing aid system
US20080199023A1 (en) Assembly, System and Method for Acoustic Transducers
EP3468228B1 (de) Binaurales hörsystem mit lokalisierung von schallquellen
AU2001239516A1 (en) System and method for optimization of three-dimensional audio
JP2007135094A (ja) 音場補正装置
EP2816823B1 (de) Audiosystem und Audiovorrichtung und Kanalmapping-Verfahren dafür
KR20170009650A (ko) 오디오 신호 처리 방법 및 장치
US10708691B2 (en) Dynamic equalization in a directional speaker array
US20190394603A1 (en) Dynamic Cross-Talk Cancellation
US20190394598A1 (en) Self-Configuring Speakers
US10511906B1 (en) Dynamically adapting sound based on environmental characterization
US10440495B2 (en) Virtual localization of sound
US20190394601A1 (en) Automatic Room Filling
JP2012227647A (ja) マルチチャンネル音響による空間音響再生システム
US11589180B2 (en) Electronic apparatus, control method thereof, and recording medium
US10440473B1 (en) Automatic de-baffling
CN111510847B (zh) 微型扬声器阵列、车内声场控制方法及装置、存储装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20141011

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20190111

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20200313

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1297246

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012071477

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200729

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1297246

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201030

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201029

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201029

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201130

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201129

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200829

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012071477

Country of ref document: DE

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20201029

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

26N No opposition filed

Effective date: 20210430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200929

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200829

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201029

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230705

Year of fee payment: 12