EP2763433B1 - Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung - Google Patents

Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung Download PDF

Info

Publication number
EP2763433B1
EP2763433B1 EP13153735.9A EP13153735A EP2763433B1 EP 2763433 B1 EP2763433 B1 EP 2763433B1 EP 13153735 A EP13153735 A EP 13153735A EP 2763433 B1 EP2763433 B1 EP 2763433B1
Authority
EP
European Patent Office
Prior art keywords
detection pulse
pulse
audio
shaped
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13153735.9A
Other languages
English (en)
French (fr)
Other versions
EP2763433A1 (de
Inventor
Jens Kristian Poulsen
Yong Zhang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BlackBerry Ltd
Original Assignee
BlackBerry Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BlackBerry Ltd filed Critical BlackBerry Ltd
Priority to EP13153735.9A priority Critical patent/EP2763433B1/de
Publication of EP2763433A1 publication Critical patent/EP2763433A1/de
Application granted granted Critical
Publication of EP2763433B1 publication Critical patent/EP2763433B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R29/00—Monitoring arrangements; Testing arrangements
    • H04R29/001—Monitoring arrangements; Testing arrangements for loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2420/00—Details of connection covered by H04R, not provided for in its groups
    • H04R2420/05—Detection of connection of loudspeakers or headphones to amplifiers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments

Definitions

  • Embodiments herein relate to electronic devices and audio accessories, and in particular to apparatus, systems and methods for identifying an audio accessory, such as a headset, coupled to an electronic device.
  • EP 2224749 A1 discloses a method for controlling a maximum signal level output to headphones of a wireless device. The method includes: determining an impedance of the headphones; determining a carrier specific maximum signal level for headphones having the impedance; and, adjusting an audio amplifier of the wireless device coupled to the headphones to restrict the maximum signal level output to the headphones to the carrier specific maximum signal level.
  • US 20080318629 A1 discloses an electronic device including a jack arranged to receive a peripheral device A/V plug, the A/V plug providing an end-terminal for a peripheral device to be connected to the jack, wherein the device is arranged to detect insertion of the A/V plug, and upon detection of insertion, discriminate the functionality provided by the peripheral device.
  • the present teaching provides a method as detailed in claim1. Also provided is a device according to claim 9.
  • Electronic devices including portable electronic devices like smart phones, tablet computers, media players, and so on have gained widespread use and may provide a variety of functions including playing media (e.g., music and movies), and in some cases telephonic services, text messaging, web browsing and other data applications.
  • media e.g., music and movies
  • telephonic services e.g., text messaging, web browsing and other data applications.
  • Electronic devices are often used with audio accessories such as headsets.
  • some electronic devices have audio jacks that are sized and shaped to receive a mating plug from a headset.
  • a user can connect the headset to the electronic device by inserting the plug on the headset into the audio jack on the electronic device. Once connected, audio can be output to the user via speakers on the audio accessory.
  • an audio accessory may incorporate a microphone to allow audio signals (e.g., speech) to be sent from the audio accessory to the electronic device. This may allow the user to make phone calls through the audio accessory, record voice memos, control the electronic device using voice commands, and so on.
  • audio signals e.g., speech
  • an audio accessory may include one or more buttons or other input devices to control the electronic device.
  • some embodiments as described herein may be implemented on electronic devices, which may include a wide range of portable devices that can be worn or carried by a human user, such as mobile phones, smart phones, personal digital assistants
  • PDAs notebooks, laptops, digital audio/video players, digital audio/video recorders, tablet computers, portable computers, music or media players, tablet computers, GPS devices and the like.
  • portable electronic devices may be handheld, that is, they may be sized and shaped to be held or carried in a human hand, and may be used while so held or carried.
  • electronic devices may also include devices that are normally not worn or carried by a human user, for example a desktop computer, a stereo system, a vehicle audio system, and so on.
  • computer resources e.g., memory capacity, processing power, and screen space
  • a smart phone may have a smaller display and less memory capacity than a desktop computer, which may have a larger display and more memory.
  • the electronic device may be a portable electronic device that has voice communication capabilities or data communication capabilities (or both), over one or more data connections (e.g., a wireless connection).
  • a method for identifying an audio accessory coupled to an electronic device includes: applying at least one detection pulse to the accessory over an audio jack, each detection pulse excluding: energies that are above a lower audible human threshold, and frequency components within the human audible frequency range, so as to be inaudible to a human; receiving at the electronic device at least one response signal from the audio accessory, each response signal corresponding to a detection pulse; wherein each detection pulse is sufficiently long such that the at least one response signal will settle to a stable current; measuring the at least one response signal after it has settled to determine an impedance of the audio accessory; and based on the determined impedance, identifying the audio accessory as a particular audio accessory.
  • a method for identifying an audio accessory coupled to an electronic device includes applying at least one detection pulse to the audio accessory, each detection pulse being spectrally shaped to be inaudible to a human user, receiving at least one response signal corresponding to each detection pulse that is indicative of the impedance of the accessory, and based on the impedance, identifying the accessory.
  • the response signal is settled before the impedance is measured.
  • the shaped detection pulse excludes energies that are above the lower audible human threshold, and frequency components within the human audible frequency range.
  • audio accessory may include any accessory (such as a supplemental device or add-on or other device that works in concert with the electronic device) and that manages, controls, processes or otherwise operates with audio signals.
  • audio accessories may include headphones, speakers, microphones, sound recorders, or accessories that incorporate one or more of such things.
  • an audio accessory may be coupled to an electronic device physically, electronically, communicatively, or some combination thereof.
  • insertion of a plug on a headset into an audio jack of a media player may physically, electronically and communicatively couple the media player and headset together, in that they can behave like a unified object, they can send or receive electrical signals with respect to one another, and they can communicate with one another.
  • references to the exclusion of energies above a lower audible human threshold and the like should be interpreted as referring to the total exclusion of energies above a lower audible human threshold as well as to the substantial exclusion of energies above a lower audible human threshold.
  • Figure 1 is a schematic diagram illustrating an electronic device 12 and an audio accessory 14 according to one embodiment.
  • the electronic device 12 may include any suitable electronic device, such as a portable smart phone having a display 13 and a physical keyboard 15 (as shown).
  • the electronic device 12 may include a touchscreen device, optionally with or without a keyboard.
  • the audio accessory 14 may include a microphone 30 for receiving audio signals (e.g., a user's voice) and for sending those audio signals to the electronic device 12.
  • the microphone 30 may be provided with the user control interface 20.
  • the microphone 30 may be provided at another location on the audio accessory 14.
  • the audio accessory 14 is connected to the electronic device 12 using a conventional audio plug on the audio accessory 14 that mates with a corresponding audio jack on the electronic device 12.
  • the plug and jack can be of the tip-ring-sleeve (TRS) variety, a tip-ring1-ring2-sleeve (TRRS) variety, or other various types.
  • TRS tip-ring-sleeve
  • TRRS tip-ring1-ring2-sleeve
  • some audio connectors are in the form of 3.5 mm (1/8") miniature plugs and jacks, or other sizes such as 2.5 mm connectors and 1/4" connectors. In headsets and other audio accessories, these audio connectors are generally used to carry audio signals and other information between the speakers 16, 18, the microphone 30 and the electronic device 12.
  • different audio accessories particularly audio accessories from different manufacturers, may have different pin configurations (e.g., TRS vs. TRRS), different control interfaces 20, may or may not have a microphone 30, may have varying audio capabilities (including volume ranges), or in general may have other differences that affect their performance and functionality when coupled to the electronic device 12.
  • Determining which particular audio accessory 14 is coupled to the electronic device 12 can allow the electronic device 12 to make changes to compensate for or take advantage of the differences between audio accessories. For instance, different functions on the electronic device 12 may be activated or deactivated depending on which audio accessory 14 is connected (e.g., whether or not a microphone 30 is present).
  • both a ground terminal placed at either RING2 or SLEEVE on a TRRS jack may be detected and supported to enable compatibility with a wider range of accessories.
  • different audio output profiles might be used for audio accessories with different audio capabilities (e.g., mono, stereo), and so on.
  • various detection techniques can be employed, which can include making one or more electrical measurements of the accessory.
  • the approach employed to identify audio accessories is to measure the impedance of the audio accessory. Impedance tends to vary between different audio accessories, but is usually relatively constant for a particular brand or type of audio accessory. Thus an impedance measurement can provide a relatively reliable "fingerprint" of the audio accessory type or brand (or both).
  • the impedance of the audio accessory 14 can be measured by applying a voltage (e.g., a detection pulse, such as a +100 mV pulse) to at least one of the pin connectors of the accessory while making a current return path available on at least one other pin.
  • a voltage e.g., a detection pulse, such as a +100 mV pulse
  • the response of the audio accessory to this voltage can then be measured and used to determine an impedance value for that audio accessory.
  • the measured impedance value can then be compared to a list of known impedances corresponding to particular audio accessories.
  • a table of audio accessories and their associated impedances could be stored on the electronic device 12 (e.g., in a memory or a database), downloaded from a webserver, and so on.
  • detection techniques are often repeated many times to validate the results. For example, depending on the desired level of certainty (e.g., less than a 0.01 % probability of a false detection result), a detection pulse may be applied to the audio accessory two, three or many more times in order to verify what particular audio accessory has been detected. In some cases, this repetition may be done by hardware (e.g., at chip level), by software, or some combination thereof.
  • the human auditory system is normally sensitive to frequencies from about 20 Hz to around 20,000 Hz (20 kHz), which is defined herein as the "human audible frequency range” (although this range can vary somewhat between different humans). Audio artifacts will generally only be audible to a human being when they fall within the human audible frequency range, or in the special case of amplitude limited pressure conditions which will not be considered here.
  • the impedance response of an audio accessory to this low frequency detection pulse can then be measured without normally introducing substantial audio artifacts.
  • this approach may involve using detection pulses with frequencies at around 10 Hz or less. This approach can also be referred to as using a "slow detection pulse”.
  • any detection pulse even a slow detection pulse, will have some spectral leakage above the 20 Hz lower limit of the human audible frequency range. Nevertheless, with a properly designed detection pulse it should be possible to control this spectral leakage and keep at least a substantial portion of the energy content below the human audible frequency range and thereby make a slow detection pulse at least substantially inaudible.
  • identifying an audio accessory using this approach can take a very long time (relatively speaking). For example, in some cases, it may take several seconds to accurately identify an audio accessory using a low frequency pulse, particularly when several detection pulses are sent to validate results. This delay can be noticeable to a user, and may lead to undesirable performance.
  • the audio accessory includes an AC-coupled load (e.g., a LINE IN connection, such as a capacitively connected input to an audio amplifier system)
  • an AC-coupled load e.g., a LINE IN connection, such as a capacitively connected input to an audio amplifier system
  • the detection of that load may be difficult (or even impossible) using low frequency detection pulses since impedance is measured in a frequency range where the load (i.e. the combination of an input capacitor and the input impedance of the amplifier itself) is very high. This tends to make the determination of the impedance inaccurate.
  • using a low frequency detection pulse may not always be suitable for detecting audio accessories and can lead to inaccurate results, particularly where it is not supplemented by other methods.
  • Figure 5 shows a sensitivity map of the human auditory response based on human hearing range research of the audible frequency range.
  • a lower audible threshold curve 80 defines the minimum audibility curve for an average human. Sounds below this curve 80 normally cannot be perceived (subject to variability for different individuals with very good or very poor hearing).
  • a second approach to avoiding audio artifacts therefore involves applying a detection pulses that are shaped to have a sufficiently low amplitude such that that the energies of the detection pulse are at least substantially below the lower audible human threshold for hearing, thus at least substantially excluding energies that are above the lower audible human threshold (generally meaning that at least a substantial portion of the energies above the lower audible human threshold are suppressed, and in some cases all the energies above that threshold are excluded).
  • This approach can be used even though the detection pulse may include energies located within the human audible frequency range (e.g., between 20 Hz and 20 kHz) so long as the amplitudes are sufficiently low. This approach may be faster than using a low frequency detection pulse since the frequency of the detection pulse can be higher.
  • the energy of the detection pulse may be more than some particular amount below the lower audible threshold curve (for example, the energy of the detection pulse could be shaped to be at least 10 dB less that the lower audible threshold curve).
  • the low energy (low amplitude) signals can be highly susceptible to noise (e.g., these signals often have a poor signal-to-noise ratio). This may lead to less accurate readings, or at least require multiple detection pulses to validate the result, which can slow the detection process when using this approach.
  • a noise-like signal shaped to be below the lower audible threshold curve may be used for some period of time in order to get a sufficiently good signal-to-noise ratio and still be able to detect impedances in which AC-coupled loads do not present any problems.
  • the detection pulse e.g., detection pulse 264
  • the detection pulse 262 could be a spectrally shaped voltage pulse driven by a pseudorandom noise generator (e.g., the noise generator 262) and then shaped with an inverse filter (e.g., the filter 266 as shown).
  • the detected response signal could then be integrated using the pseudorandom noise generator as reference for a cross-correlation measurement of the impedance value (see for example Figure 10 ).
  • a third approach to avoiding audible artifacts involves using shaped detection pulses with a spectral content that is above the audible frequency range (e.g., a high-frequency detection pulse above 20 kHz) so that the pulse is generally inaudible to a human being (thus at least substantially excluding frequency components within the human audible frequency range).
  • a spectral content that is above the audible frequency range (e.g., a high-frequency detection pulse above 20 kHz) so that the pulse is generally inaudible to a human being (thus at least substantially excluding frequency components within the human audible frequency range).
  • This approach may allow accessory detection to be very fast. Furthermore, higher frequency detection pulses tend to be good at detecting AC-coupled loads since any input coupling capacitors will tend not disturb the impedance measurements at these frequencies. This approach also tends to have a good signal-to-noise ratio and thus be resistant to noise since the energy levels of the detection pulse can be relatively high.
  • the capacitance in series with the load will still conduct current after the initial transient has died out, since the RC-constant for a LINE IN connection is typically below 20 Hz, or significantly below the frequency of operation.
  • the sampling event is instead chosen to be just at the onset of the initial transient, a measurement of the combined resistance and capacitance can be made, after which the capacitance can be estimated by subtracting the resistance.
  • Figure 12 shows how currents from parasitic capacitance may be avoided by careful sampling at specific times. This approach might be used to detect the presence and polarity of an attached cable.
  • a detection pulse with a high amplitude is often used. If such a high amplitude detection pulse is applied directly to a low impedance load, such as a low impedance (e.g., 32 Ohm) sensitive audio accessory (e.g., headphones), large clicks or other audio artifacts might be heard by a user unless precautions are taken to limit the spectral leakage from the detection pulse so it does not appear within the human audible frequency range.
  • a low impedance load such as a low impedance (e.g., 32 Ohm) sensitive audio accessory (e.g., headphones)
  • the active electronics may present an impedance that varies with frequency (e.g., input filters).
  • the measured impedance at a high frequency outside the human audible frequency range may not be the same value as the impedance within the human audible frequency range.
  • spectral leakage when using a slow detection pulse, the spectral leakage from the lower frequencies to higher audible frequencies could be limited to suppress audible artifacts.
  • spectral leakage can be limited by using a detection pulse with a known low leakage (such as a sinc-shaped pulse) that is combined with a known good windowing function (such as a Gaussian, flat-top or Blackman-Harris type windowing function, as shown in Figure 11 for example).
  • a detection pulse with a known low leakage such as a sinc-shaped pulse
  • a known good windowing function such as a Gaussian, flat-top or Blackman-Harris type windowing function, as shown in Figure 11 for example.
  • the desired spectral characteristics of the leaked energy inside the human audible frequency range may determine the dynamic range of the output pulse and the attenuation needed to suppress audible artifacts.
  • the audibility of the detection pulse could be limited by spectral shaping.
  • a noise generator with flat spectral characteristics may be used to generate the detection pulse.
  • the pulse may be attenuated with a transfer function that is generally the opposite of the human hearing ability (e.g., the opposite of the curve 80 as shown in Figure 5 ).
  • short detection pulses can used with an envelope that minimizes the spectral leakage within the human audible frequency range (or at least limits the spectral leakage so that the detection pulses are substantially inaudible).
  • a special spectrally shaped detection pulse can be stored in hardware and generated on command (i.e. as a particular voltage pattern). This sequence can then be sent to the audio accessory as a special inaudible waveform when desired.
  • an inaudible waveform can be stored in a ROM or implemented as an algorithm (e.g., a triangular generator) and controlled by a counter.
  • the output coefficients could be fed to a DAC (digital to analog converter) or they could be directly implemented using variable size resistors or capacitors that (in an analog fashion) implement the waveform shape thereby eliminating the need for a DAC.
  • the pulse waveform When implementing a generally inaudible waveform that exploits the low audibility of low frequencies (e.g., frequencies below the limit of human hearing) the pulse waveform should be implemented correctly.
  • the waveshape chosen can be a window function such as a triangular, Gaussian, Blackman-Harris or some other window shape with known good attenuation of sidelobes and low spectral leakage.
  • a DC voltage source 202 can be used to generate a raw detection pulse 204, which is then subjected to a window function 206.
  • the resulting signals can be multiplied using a multiplier 208 and then outputted as a generally inaudible detection pulse which is sent to the audio accessory.
  • the discretized values since otherwise this may result in audible noise.
  • the use of a triangular window and a low number of discretized steps to represent a waveform may result in audible tones, since this waveform can be represented as the addition of a perfect triangular waveform superimposed with a high frequency square wave with an amplitude equal to the discretization steps. This may be audible due to the high sensitivity to tones of the human ear.
  • dithering or an output filter
  • noise shaping may be used thereby pushing the quantization noise out of the audible range to avoid the effects of quantization.
  • the window function may be implemented using an analog circuit without any steps (e.g., to charge and discharge a capacitor at a controlled rate).
  • An example of a suitable weighting curve for this method is the inverse of the A-weighting curve (e.g., filter 266 shown in Figure 9 ) that may be used in order to send out more energy in the energy bands where the human ear is less sensitive. Accordingly it should be possible to emit a considerable amount of energy within a limited time and make this detection pulse at least substantially inaudible to a human being.
  • the A-weighting curve e.g., filter 266 shown in Figure 9
  • a unipolar pulse as shown in Figure 11 can be compared to a bipolar pulse.
  • a unipolar pulse with a duration of T will have an energy peak at DC and a null at 1/T.
  • This pulse can be represented in the frequency domain as a Tsinc(fT) function and has wide leakage.
  • T 1/32768, although in other embodiments other values for T could be used.
  • the bipolar pulse with duration of T can be represented in the frequency domain as j2Tsinc(fT)sin( ⁇ fT).
  • the spectral leakage will be lower since this pulse has no DC-component. Therefore if this pulse is made sufficiently quick it will be inaudible to a human being.
  • spectral shaping can be performed by a highpass filter 226 located on the electronic device (e.g., the electronic device 12).
  • a high frequency voltage source 222 can gated by a switch controlled by a short pulse generator 224, which is then filtered by the highpass filter 226 to be generally inaudible.
  • the detection pulse be modulated at a frequency that is outside the human audible frequency range, while at the same time be sufficiently long such that the speakers inside the accessory will settle to a stable current when a voltage is applied thereto. This usually happens after a transient period of approximately 12 microseconds (at most), and in some cases much less time is required.
  • a pulse with a half-period of 15 microseconds could be used, which fits particularly well with a frequency of 32.768 kHz (a common oscillation frequency of components in many electronic devices), since this is within the desired frequency range (e.g., above 20 kHz), is very accurate, and may enable low power operation of the electronic device while detection is being accomplished.
  • 32.768 kHz a common oscillation frequency of components in many electronic devices
  • One further limitation is that it is desirable to have short detection times both from a user experience point of view, but also since very long sequences will tend to be more complicated to implement. Moreover, the risk of audio artifacts (e.g., clicks and pops) during insertion of the accessory rises in partial insertion scenarios, where the detection starts after the accessory has been partially inserted and the user subsequently completes the full insertion of the audio plug.
  • audio artifacts e.g., clicks and pops
  • the user may fully connect the audio accessory only after the detection has started, and there will be a chance that am electrical connection is made in the middle of a detection sequence, starting the sequence at high amplitude.
  • the initial transient may be audible as an audio artifact.
  • the method 200 of Figure 6 and the method 220 of Figure 7 may be combined into a method 240 as shown in Figure 8 , which can allow for the selection of either a DC test (e.g., using the DC voltage source 202) or an AC test (e.g., using the high frequency voltage source 222).
  • This embodiment may be particularly useful as it may allow different detection pulses to be sent under different circumstances, for instance for testing of DC and AC coupled loads.
  • the unshaped detection pulse 60 includes a significant portion of energy within the audible frequency range (e.g., between 20 Hz and 20,000 Hz), although a portion of the pulse 60 is above the upper frequency threshold 70 of the audible frequency range (i.e. a portion of the pulse 60 is at a frequency above 20,000 Hz).
  • the shaped detection pulse 62 a significant portion of the energy in the audible frequency range has been suppressed (i.e. the shaped detection pulse 62 at least substantially excludes frequency components within the human audible frequency range).
  • the portion of the shaped detection pulse 62 below the upper threshold 70 is much smaller as compared to the unshaped detection pulse 60.
  • the shaped detection pulse 62 should create far fewer audible artifacts as compared to the unshaped pulse 60.
  • the entire shaped detection pulse 62 should be inaudible to a human user.
  • the sequence ⁇ 2, -6, 12, -18, 21, -18, 12, -6, 2 ⁇ (representing the filter coefficients of a 8-th order finite impulse response (FIR) filter) will give significantly lower audible acoustic output as compared to a single sequence filter (assuming the same energy output).
  • FIR finite impulse response
  • the waveform is implemented using a modulated square wave, this can be done using a current source, with five resistors of values ⁇ 2, 6, 12, 18, 21 and switches to control which resistor value is chosen.
  • resistors with values ⁇ 2, 3, 6, 12 ⁇ can be used in combination to obtain the desired values, but with a smaller total area.
  • coefficients were found by designing a digital 10-th order highpass filter with a cutoff of 0.85 of the Nyquist limit (half the sample frequency) and quantized to finite values using rounding after multiplying with a factor of 100 (the two outermost coefficients became zero).
  • the spectrum of this sequence is shown in Figure 2 and labeled as curve 62 and compared to other sequences.
  • the curves have been A-weighted when the spectrum was plotted and the energy for each pulse was normalized to be the same (A-weighting is an appropriate filtering for this curve, since the amplitude is of low value). This means that the curves represent the disturbance of each pulse as compared to each other, in different frequency bands while taking the sensitivity of the human ear into account.
  • the unshaped pulse 60 shows the spectrum of a monopolar pulse and while curve 66 shows the spectrum of a bipolar pulse.
  • the use of the bipolar pulse 66 results in some attenuation of the audible spectrum below 5 kHz in the range of 10-30 dB, though there is less difference above 5 kHz.
  • a detection pulse using 9 coefficients and labeled as shaped pulse 62 shows an attenuation of more than 30 dB up to about 17 kHz, and considerable attenuation between 17 kHz up to 20 kHz.
  • Curve 64 shows the results of a longer sequence, in this case a triangular window that is modulated by a square wave with 99 taps (resulting in the sequence ⁇ 1, -2, +3, -4, ..., 49, -50, 49, ... -2, 1 ⁇ . It is well known that the triangular window has significant sidelobes. However, this technique may be relatively simple to implement in hardware and the spectral curve 64 shows a very significant attenuation of spectral leakage as compared to the other curves, showing the advantages of using longer sequences.
  • a high frequency periodic sequence outside the audible bandwidth e.g., a sine or a square wave
  • a window function e.g., a Gaussian window
  • DAC D/A converter
  • the shaped pulses were slightly longer in the time domain than the original pulses, but this should be of minor concern, since the extra delays are on the order of fractions of a millisecond.
  • the delay could be up to one millisecond or longer, which could add significant detection delays, particularly if multiple pulses are used to test different impedances and different configurations.
  • multiple pulses may be used in order to check for different impedance values. For instance, a generator with a high output impedance may be used to test high impedance loads, while a lower output impedance may be used to test lower impedance loads. This scheme may be used to reduce the errors made when making electrical measurements of particular audio accessories.
  • One advantage of the longer detection pulses is a further reduction of spectral leakage, thereby making the pulses fully inaudible.
  • quantitative measurements showed more than 30 dB of attenuation using this approach as compared to an unshaped pulse.
  • the systems and methods as described herein can be implemented as part of a custom hardware (e.g., an ASIC or other circuit) that can handle jack detection.
  • a custom circuit may require minimal extra silicon area on the electronic device, thereby making it cost effective and realistic to implement.
  • a detection circuit can be made using an array of charged capacitors of various sizes (or as an array of resistors of various values) that store the desired waveform so that it can be generated when desired. Accordingly, it may not be necessary to implement a digital memory and a digital-analog converter (DAC) in order to carry out some of the spectral shaping methods as generally described herein.
  • DAC digital-analog converter
  • the teachings herein may be implemented using one or more highpass filter (e.g., an analog filter, or digital filter, or both), and in such cases the pulse waveform need not be stored.
  • the suitability of this approach may depend on the spectral characteristics of the available filtering technologies.
  • one approach may be to use matched filtering when receiving the transmitted detection pulse back at the electronic device. In this manner, the influence of noise can be reduced, thereby improving the sensitivity of the detection method.
  • SNR signal-to-noise ratio
  • Figure 10 shows a schematic of a method 280 that includes multiplying the received response signal and the detection pulse (e.g., using a multiplier 282). It is generally beneficial that the signal is sampled after the initial transient has died out to avoid errors due to additional capacitance and inductance in the system and from the accessory.
  • a received detection signal should be sampled at a suitable time using sampler 283. For example, if a resistive measurement is desired, then the sampling should be done at the end of each step value. The resulting signal may then be integrated (e.g., using an integrator 284).
  • the integrated signal may then pass through a threshold detector 286 before the resulting signal is used to determine the impedance of the observed audio accessory.
  • detection could be implemented by multiplying the received pulse with the original waveform, sampling at the correct time, integrating this pulse shape and using the final integrated value for a threshold detector.
  • the shaped detection pulse twice, with the second pulse having an inverted amplitude as compared to the first pulse. By subtracting these two detection pulses, any external influences of noise at lower frequencies should be cancelled, thereby making the detection methods more robust.
  • the received pulses should be gated, and pulses with a certain length should be used in order to make sure the received pulses have fully settled before taking a measurement.
  • a settling time of 15 microseconds or longer may be enough for some audio accessories, such as headsets and headphones.
  • the electronic device 110 may be adapted to detect whether a particular audio accessory 150 is coupled to the electronic device 110 by monitoring an impedance detected by the electronic device 110 through an audio jack 111, with spectral shaping being used to ensure that the detection pulse is generally inaudible and generally does not cause undesired audio artifacts.
  • the audio accessory 150 may include one or more speakers.
  • the input impedance between two pins of the accessory, Z IN 152 represents one or more speakers.
  • Z IN may represent a microphone or the input impedance of an amplifier or some other component.
  • the electronic device 110 is adapted to generate a shaped detection pulse lp on a ground return line 114 of the audio jack 111.
  • the shaped detection pulse I P can be applied to the SLEEVE of a TRS jack, or RING2 and SLEEVE in a TRRS jack, or to the TIP or RING1.
  • the pulse I P could be generated in a raw or unshaped format by a pulse generator 140, and then filtered by a high pass filter 144 to obtain a desired waveform (e.g., with at least a substantial portion of the energy content located outside the audible frequency range).
  • the shaped detection pulse I P is a voltage pulse, which could for example be generated by a voltage source that is coupled to, or part of, the pulse generator 140.
  • the shaped pulse I P may have a magnitude of between about -50 mV to +50 mV.
  • the shaped detection pulse I P is sent to the audio accessory 150 and returns back to the electronic device 110 (in this embodiment) via the audio line 114 as a response signal I R that is indicative of the impedance of the audio accessory 150. As shown, in this embodiment the response signal I R is monitored by a detector 154 on the electronic device 110.
  • the amplifier 110 may be tri-stated in order not to disturb the measurement.
  • a headphone amplifier may be used to directly output the detection pulse.
  • the electronic device 110 can then compare the measured response signal I R to known impedance values for known audio accessories in order to identify the particular audio accessory 150 (i.e. known impedance values for known audio accessories may be stored in a memory).
  • a plurality of shaped detection pulses I P can be sent to the audio accessory to validate the measured impedance.
  • the plurality of shaped detection pulses I P generate a plurality of response signals I R that can be averaged to determine an averaged impedance.
  • the averaging may be done by a processor or other processing device.
  • the electronic device 110 can take one or more actions. For instance, the electronic device 110 may compensate for or take advantage of different functions and capabilities of the particular audio accessory 150. In some examples, the electronic device 110 can adjust audio output volume sent over the audio line 114 of the jack 111 for high output audio accessories, can disable the microphone line when a microphone is not present, can output only mono audio when a mono-only audio accessory is detected, and so on.
  • the lower threshold curve 80 defines the minimum audibility curve for humans. It is evident by visual inspection that there is significant variation in the sensitivity of the average human being to sound intensity depending on the frequency of the sound. For instance, within the audible frequency range, the human ear is especially sensitive to frequencies between 300 Hz and 6 kHz, and even more sensitive to frequencies between 2 kHz and 5 kHz.
  • the spectral shaping of the detection pulse can be configured to particularly reduce the signal amplitude between 300 Hz and 6 kHz, and more particularly between 2 kHz and 5 kHz.
  • the shaped detection pulse 62 has a generally low energy between the two thresholds 74 and 72 corresponding to 2 kHz and 5 kHz ranges, respectively.
  • detection of this shaped detection pulse 62 may be very difficult for a human user.
  • a detection pulse with energies that are either outside of the audible frequency range (e.g., either above or below) or below the threshold curve 80 for human sensitivity, or some combination thereof, it may be possible to detect what particular audio accessories are coupled to the electronic device in a generally inaudible manner.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
  • Headphones And Earphones (AREA)

Claims (11)

  1. Verfahren zum Identifizieren eines Audiozubehörs (14, 150), das an eine elektronische Vorrichtung (12, 110) gekoppelt ist, wobei das Verfahren umfasst:
    Anwenden wenigstens eines Detektionsimpulses (62, 264) auf das Audiozubehör, wobei jeder Detektionsimpuls spektral geformt ist, um für einen menschlichen Anwender unhörbar zu sein, wobei der geformte Detektionsimpuls (62) einen Hochfrequenz-Detektionsimpuls enthält;
    Empfangen wenigstens eines Antwortsignals, das jedem Detektionsimpuls entspricht, und Messen des wenigstens einen Antwortsignals, nachdem es eingeschwungen ist, um eine Impedanz des Audiozubehörs zu bestimmen, wobei jeder Detektionsimpuls ausreichend lang ist, so dass das wenigstens eine Antwortsignal zu einem stabilen Strom einschwingt; und
    basierend auf der bestimmten Impedanz Identifizieren des Audiozubehörs.
  2. Verfahren nach Anspruch 1, wobei der Hochfrequenz-Detektionsimpuls einen Energieinhalt aufweist, der sich über einem hörbaren Frequenzbereich befindet.
  3. Verfahren nach einem vorhergehenden Anspruch, wobei die Form des Detektionsimpulses (62, 264) hardware-technisch gespeichert ist.
  4. Verfahren nach einem vorhergehenden Anspruch, wobei der Detektionsimpuls (62, 264) unter Verwendung eines Hochpassfilters (144, 226), das sich zwischen einem Impulsgenerator (140, 224) in der elektronischen Vorrichtung (12, 110) und dem Audiozubehör (14, 150) befindet, geformt wird und wobei optional das Hochpassfilter eine Grenzfrequenz über 20 kHz aufweist.
  5. Verfahren nach Anspruch 4, wobei wenigstens ein Anteil des geformten Detektionsimpulses (62) durch das Hochpassfilter (144, 226) in den hörbaren Frequenzbereich austritt und wobei optional der ausgetretene Anteil geformt ist, so dass er sich unter der unteren menschlichen Hörbarkeitsschwelle befindet.
  6. Verfahren nach einem vorhergehenden Anspruch, das ferner das Anwenden einer angepassten Filterung auf das Antwortsignal umfasst, um den Einfluss des Rauschens zu verringern.
  7. Verfahren nach einem vorhergehenden Anspruch, wobei der wenigstens eine geformte Detektionsimpuls (62) mehrere geformte Detektionsimpulse enthält und wobei optional die mehreren geformten Detektionsimpulse mehrere Antwortsignale erzeugen und die mehreren Antwortsignale gemittelt werden, um die Impedanz zu bestimmen.
  8. Verfahren nach einem vorhergehenden Anspruch, wobei wenigstens einer der geformten Detektionsimpulse (62) eine invertierte Amplitude aufweist und von einem weiteren geformten Detektionsimpuls abgezogen wird, um den Einfluss des Rauschens zu verringern.
  9. Elektronische Vorrichtung (12, 110), die konfiguriert ist, ein an sie gekoppeltes Audiozubehör (14, 150) zu identifizieren, wobei die Vorrichtung umfasst:
    einen Impulsgenerator (140, 224), der wenigstens einen Hochfrequenz-Detektionsimpuls (264) erzeugt und den Detektionsimpuls über eine Audiobuchse (111) in ein Audiozubehör einspeist, wobei jeder Detektionsimpuls spektral geformt ist, um für einen menschlichen Anwender im Allgemeinen unhörbar zu sein; und
    einen Detektor (154, 286) zum Empfangen wenigstens eines Antwortsignals und Messen des wenigstens einen Antwortsignals, nachdem es eingeschwungen ist, um eine Impedanz des Audiozubehörs zu bestimmen, wobei jedes Antwortsignal einem des wenigstens einen Detektionsimpulses entspricht; wobei
    jeder Detektionsimpuls ausreichend lang ist, so dass das wenigstens eine Antwortsignal zu einem stabilen Strom einschwingt.
  10. Elektronische Vorrichtung (12, 110) nach Anspruch 9, wobei der geformte Detektionsimpuls (62) einen Hochfrequenz-Detektionsimpuls enthält und wobei optional der Hochfrequenz-Detektionsimpuls einen Energieinhalt aufweist, der sich über einem hörbaren Frequenzbereich befindet.
  11. Elektronische Vorrichtung (12, 110) nach Anspruch 9 oder Anspruch 10, wobei der wenigstens eine geformte Detektionsimpuls (62) mehrere geformte Detektionsimpulse enthält, die durch den Impulsgenerator (140, 224) erzeugt werden, und wobei optional die mehreren geformten Detektionsimpulse mehrere Antwortsignale erzeugen, die durch den Detektor (154, 286) detektiert werden, wobei die mehreren Antwortsignale gemittelt werden, um die Impedanz zu bestimmen.
EP13153735.9A 2013-02-01 2013-02-01 Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung Active EP2763433B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP13153735.9A EP2763433B1 (de) 2013-02-01 2013-02-01 Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP13153735.9A EP2763433B1 (de) 2013-02-01 2013-02-01 Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung

Publications (2)

Publication Number Publication Date
EP2763433A1 EP2763433A1 (de) 2014-08-06
EP2763433B1 true EP2763433B1 (de) 2017-04-05

Family

ID=47826843

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13153735.9A Active EP2763433B1 (de) 2013-02-01 2013-02-01 Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung

Country Status (1)

Country Link
EP (1) EP2763433B1 (de)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160373861A1 (en) 2015-06-18 2016-12-22 Blackberry Limited Passive audio headset compatible with binaural recording and off-headset noise cancellation
EP3480950B1 (de) * 2017-11-01 2022-09-07 Nxp B.V. Lastdetektor und lastdetektionsmethode
GB2579677B (en) * 2018-12-11 2021-06-23 Cirrus Logic Int Semiconductor Ltd Load detection

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100907451B1 (ko) * 2004-10-25 2009-07-13 노키아 코포레이션 비디오／오디오 플러그를 통하여 전자 장치에 연결된 주변장치의 감지, 식별 및 동작
KR100619055B1 (ko) * 2004-11-16 2006-08-31 삼성전자주식회사 오디오/비디오 시스템의 스피커 모드 자동 설정 방법 및장치
ATE531208T1 (de) * 2009-02-27 2011-11-15 Research In Motion Ltd Verfahren und system zum steuern einer maximalen signalstufenausgabe and kopfhörer, die mit einer funkvorrichtung gekoppelt sind
US8995672B2 (en) * 2010-11-30 2015-03-31 Blackberry Limited System and method for microphone polarity detection

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP2763433A1 (de) 2014-08-06

Similar Documents

Publication Publication Date Title
US9143863B2 (en) Apparatus, systems and methods for inaudibly identifying an audio accessory using spectral shaping
US9084035B2 (en) System and method of detecting a plug-in type based on impedance comparison
US9124980B2 (en) System and method for optimized playback of audio signals through headphones
EP2986034A1 (de) Tonsystementzerrung für tragbare medienspieler
KR20050026967A (ko) 전기 임피던스에 기반한 오디오 장치내의 오디오 보상 및그 방법
US20140341386A1 (en) Noise reduction
EP3054707B1 (de) Vorrichtung, verfahren und programm zur messung von schallfeldern
US10045124B2 (en) Crosstalk mitigation
EP2763433B1 (de) Vorrichtung, Systeme und Verfahren zur unhörbaren Identifizierung eines Audiozubehörs unter Verwendung von Spektralformung
US8385563B2 (en) Sound level control in responding to the estimated impedances indicating that the medium being an auditory canal and other than the auditory canal
CN104168532A (zh) 扬声器异音检测方法及装置
CN116158092A (zh) 使用外部刺激评估耳封的系统和方法
KR20130112545A (ko) 무선 통신 기기의 간섭 분석 장치와 이를 이용한 간섭 분석 시스템 및 그 방법
TWI790707B (zh) 一種音頻功率放大器晶片電源雜訊抑制比的高效測量方法
US10965265B2 (en) Method and device for adjusting audio signal, and audio system
CN112995882B (zh) 一种智能设备音频开环测试方法
CN113115197B (zh) 噪声检测方法和系统
US6574310B1 (en) Apparatus and method for a wide band spectral balance measurement
Dwisetyo et al. Acoustical periodic test of sound level meter based on smartphone application using free-field method
WO2016077193A1 (en) Portless and membrane-free microphone
EP2958343B1 (de) Vorrichtung zum Testen der Direktionalität in Hörgeräten
CN211296932U (zh) 一种可功率平衡的扬声器
TWI475870B (zh) 電話裝置的測試裝置以及測試方法
JP6282999B2 (ja) 補聴器の指向性を試験するための装置
Jones et al. Testing challenges in personal computer audio devices

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130201

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17Q First examination report despatched

Effective date: 20150318

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20161021

RIN1 Information on inventor provided before grant (corrected)

Inventor name: POULSEN, JENS KRISTIAN

Inventor name: ZHANG, YONG

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 882817

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170415

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013019323

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20170405

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 882817

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170706

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170805

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013019323

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

26N No opposition filed

Effective date: 20180108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180201

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130201

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230223

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240228

Year of fee payment: 12

Ref country code: GB

Payment date: 20240220

Year of fee payment: 12

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602013019323

Country of ref document: DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013019323

Country of ref document: DE

Owner name: MALIKIE INNOVATIONS LTD., IE

Free format text: FORMER OWNER: BLACKBERRY LIMITED, WATERLOO, ONTARIO, CA