EP3163903B1 - Akustischer prozessor für eine mobile vorrichtung - Google Patents

Akustischer prozessor für eine mobile vorrichtung Download PDF

Info

Publication number
EP3163903B1
EP3163903B1 EP15191381.1A EP15191381A EP3163903B1 EP 3163903 B1 EP3163903 B1 EP 3163903B1 EP 15191381 A EP15191381 A EP 15191381A EP 3163903 B1 EP3163903 B1 EP 3163903B1
Authority
EP
European Patent Office
Prior art keywords
signal
speaker
mobile device
acoustic
speaker membrane
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP15191381.1A
Other languages
English (en)
French (fr)
Other versions
EP3163903A1 (de
Inventor
Christophe Marc Macours
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NXP BV
Original Assignee
NXP BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NXP BV filed Critical NXP BV
Priority to EP15191381.1A priority Critical patent/EP3163903B1/de
Priority to CN201610916146.XA priority patent/CN106919225B/zh
Priority to US15/333,897 priority patent/US9866958B2/en
Publication of EP3163903A1 publication Critical patent/EP3163903A1/de
Application granted granted Critical
Publication of EP3163903B1 publication Critical patent/EP3163903B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1688—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being integrated loudspeakers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1686—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being an integrated camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/162—Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H04M1/03—Constructional features of telephone transmitters or receivers, e.g. telephone hand-sets
    • H04M1/035—Improving the acoustic characteristics by means of constructional features of the housing, e.g. ribs, walls, resonating chambers or cavities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/326—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only for microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2400/00—Loudspeakers
    • H04R2400/01—Transducers used as a loudspeaker to generate sound aswell as a microphone to detect sound
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
    • H04R2430/21—Direction finding using differential microphone array [DMA]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/11—Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's

Definitions

  • This disclosure relates to an acoustic processor for a mobile device and a method of acoustic processing for a mobile device.
  • Directional sound capture is used in many applications, ranging from studio music recording to audio-visual recording in both professional and consumer cameras.
  • Directional sound capture uses directional microphones or microphone arrays combined with sophisticated array processing algorithms to obtain the desired spatial responses.
  • Small form factor consumer devices such as mobile phones or wearables are usually equipped with multiple microphones (typically 2 or more) primarily used for beamforming and non-stationary noise suppression in handset and hands-free voice call applications.
  • microphones typically 2 or more
  • Recent applications such as camcording also make use of multiple microphones for stereo or multichannel sound recording.
  • microphones are usually mounted on the edges of the device.
  • the microphone In single-microphone devices, the microphone is placed at the bottom, usually either front-facing or bottom-facing. In dual-microphone devices, one microphone is located at the top, usually either back-facing or top-facing.
  • US 20140270312 A1 describes systems and methods for using a speaker as a microphone in a mobile device.
  • a mobile device comprising: an acoustic processor ; a speaker including a speaker membrane, the speaker membrane having a first speaker membrane side and second speaker membrane side opposite the first speaker membrane side; a mobile device housing for providing a first acoustic path between the first speaker membrane side and the exterior of the mobile device and a second acoustic path between the second speaker membrane side and the exterior of the mobile device; wherein the mobile device is operable to configure the speaker as a microphone, and the acoustic processor is configured and arranged to sense a signal on an acoustic processor input coupled to at least one terminal of the speaker; process the induced signal to discriminate between acoustic waves from different directions; and output the processed signal on an acoustic processor output.
  • the acoustic processor may receive a signal from a speaker used as a microphone and process the signal to allow front to back discrimination i.e. discriminate between and acoustic source located in front of the mobile device and a further acoustic source located behind the mobile device. This may be used for example in camcorder modes of mobile phones to capture a subject's speech and supress speech from a user of the mobile phone.
  • the acoustic processor may be further configured to mix the induced signal with a further signal sensed via a microphone in response to the at least one acoustic source.
  • the acoustic processor may include a mixer to combine the induced signal from the speaker as microphone with a microphone signal.
  • the resulting processed signal from the acoustic process may comprise the mixed signal and further signal in varying proportions.
  • the mixer may apply filtering and beamforming to the input signals.
  • the response of the induced signal may be equalized to for example align the response of the speaker with an omnidirectional microphone. This aligned response may allow beam forming using the signals from the microphone and the speaker to provide directional selectivity.
  • the acoustic processor may be further configured to determine a signal to noise ratio from at least one of the induced signal and the further signal and to alter the mixing ratio between the induced signal and the further signal dependent on the signal to noise ratio.
  • the acoustic processor may be further configured to process the induced signal to discriminate between an acoustic signal received from an acoustic source in a first direction and a further acoustic signal received from a further acoustic source in a second direction.
  • the acoustic processor may be further configured to process the induced signal to discriminate between an acoustic source located in a first direction with respect to the mobile device and a further acoustic source located in a second direction with respect to the mobile device.
  • Each side of the speaker membrane that is to say each of the major surfaces may be acoustically coupled to the exterior of the mobile device.
  • the speaker may have a directional response characteristic.
  • all microphones may be located in the same plane and only allow a spatial discrimination along the sides of the phone, that is to say top-to-bottom, left-to-right. Form factor constraints may make it difficult to mount two microphones in an end-fire configuration along the front-back axis so as to allow front-back directivity control.
  • a third microphone may be placed near the receiver speaker and sharing the same acoustical port. The third microphone may be used for active noise cancellation in handset mode or as additional microphone in camcording mode so as to obtain an improved front-back discrimination due to the acoustical shadowing of the device for sources located at the back of the device.
  • the mobile device may comprise a camera and be configurable in a camcorder mode of operation, and the acoustic processor may be further configured to store the processed signal.
  • the mobile device may be configurable in a video call mode, and further configured to transmit the processed signal to a receiver of a video call.
  • a mobile device including the acoustic processor may be configured as one of a mobile phone, a wearable device, a portable audio player, a personal digital assistant, a laptop, a tablet computer.
  • a method of acoustic processing for a mobile device comprising a speaker including a speaker membrane, the speaker membrane having a first speaker membrane side and second speaker membrane side opposite the first speaker membrane side, a mobile device housing for acoustically coupling each speaker membrane side to the exterior of the mobile device, the method comprising: sensing a signal induced on at least one terminal of the speaker in response to acoustic waves, and processing the induced signal; to discriminate between acoustic waves from different directions; and outputting a processed signal.
  • the method may further comprise sensing a further signal via a microphone, and combining the induced signal and the further signal.
  • the method may further comprise estimating the signal to noise ratio from the further signal.
  • Estimating the signal to noise ratio may be used to determine a measure of the ambient noise.
  • the method may further comprise estimating the signal to noise ratio value from the induced signal.
  • processing the induced signal may comprise at least one of attenuating the induced signal for a signal to noise ratio value above a predetermined threshold and attenuating the further signal for a signal to noise ratio value below the predetermined threshold.
  • the speaker as a directional microphone in a mobile device comprising a speaker including a speaker membrane, the speaker membrane having a first speaker membrane side and second speaker membrane side opposite the first speaker membrane side, a mobile device housing for providing a first acoustic path between the first speaker membrane side and the exterior of the mobile device and a second acoustic path between the second speaker membrane side and the exterior of the mobile device.
  • Embodiments may include an article of manufacture including at least one non-transitory, tangible machine readable storage medium containing executable machine instructions for execution by a processor, wherein the article includes a speaker including a speaker membrane, the speaker membrane having a first speaker membrane side and second speaker membrane side opposite the first speaker membrane side, a housing for acoustically coupling each speaker membrane side to the exterior of the article; and wherein the instructions include sensing a signal induced on at least one terminal of the speaker configured as a microphone in response to at least one acoustic source , and processing the induced signal; wherein at least one characteristic of the processed signal is dependent on the orientation of the mobile device with respect to the at least one acoustic source.
  • the computer program may be a software implementation, and the computer may be considered as any appropriate hardware, including a digital signal processor, a microcontroller, and an implementation in read only memory (ROM), erasable programmable read only memory (EPROM) or electronically erasable programmable read only memory (EEPROM), as non-limiting examples.
  • the software implementation may be an assembly program.
  • the computer program may be provided on a computer readable medium, which may be a physical computer readable medium, such as a disc or a memory device, or may be embodied as a transient signal.
  • a transient signal may be a network download, including an internet download.
  • FIG 1 shows in figure 1A a mobile phone 100 showing the housing and location of speaker aperture.
  • Figure IB shows the mobile phone 100 including a speaker 114 and an acoustic processor 116.
  • mobile phone 100 may have a housing 102 and an aperture 104 in one of the major surfaces of the housing which may be referred to as the front of the mobile phone.
  • the mobile phone 100 may also have operating buttons 106 and 108 located towards one of the edges of the housing 102.
  • the surface of the mobile phone 100 may include a touch sensitive display 110.
  • the mobile phone 100 may also have one or more microphones (not shown).
  • a speaker 114 including a speaker membrane 130 or diaphragm.
  • the speaker membrane 130 may have a first side or surface which may face towards the aperture 104 so may be considered for example front facing.
  • the speaker membrane 130 may have a second side or surface facing away from the aperture 104 which may for example be considered back facing.
  • the speaker 130 may emit the acoustic signal, typically an audible signal which may for example contain speech and/or music via a first acoustic path through the aperture 104.
  • the aperture 104 may therefore acoustically couple the first side of the speaker membrane to the exterior of the mobile phone 100.
  • the speaker 114 may be a receiver speaker for use in a handset voice call mode of operation whereby the receiver speaker is closely coupled to the ear of a user. This is typically referred to as a near field mode of operation.
  • the housing 102 may have a further aperture 128 which may provide a further acoustic path from the exterior of the mobile device 100 to the second side of the speaker membrane via the back of the speaker 114.
  • the skilled person will appreciate that there may be an acoustic path through the back of the speaker to the second side of the speaker membrane 130 via the aperture 128. Consequently the further aperture 128 in the housing 102 may provide an acoustic coupling between the second side of the speaker membrane 130 and the exterior of the mobile phone 100.
  • the speaker 114 may be driven by an amplifier 126 which may be connected to the speaker via a switch 112.
  • the switch 112 may be controlled by a controller (not shown) and connect the amplifier 126 to the speaker 114.
  • the switch 112 may connect an acoustic processor input 120 of the acoustic processor 116 to the speaker 114.
  • the switch 112 may be implemented in hardware using MOS transistors.
  • the switch 112 may be controlled by a controller implemented in hardware or a combination of hardware and software.
  • the switch 112 may be implemented by any hardware or software which allows a signal to be either driven from the amplifier 126 to the speaker 114 or routed from the speaker terminal or terminals 124 to the acoustic processor input 120.
  • the acoustic processor 116 may be implemented in hardware/software or a combination of hardware and software.
  • the acoustic processor 116 may be implemented for example using software running on a digital signal processor.
  • the acoustic processor 116 may apply filters, and/or equalization to an acoustic signal received on the acoustic processor input 120.
  • the acoustic processor 116 may apply threshold value to the acoustic signal that is to say compare the acoustic signal with an expected minimum level. It will be appreciated that the acoustic processor 116, the amplifier 110 which may be a class D audio amplifier and the switch 112 are illustrated as being outside the housing 102 but will be contained within the housing 102 of the mobile device.
  • the amplifier 110 may be connected to the loudspeaker 114 via the switch 112.
  • the speaker 114 is then used in its intended operation mode as a receiver speaker in a so-called near field mode of operation whereby speech received from a third-party may be heard by the user of the mobile phone 100.
  • the speaker 114 is typically closely coupled to the ear of a user 118 in this mode and so is typically at a distance of less than 1 cm from the ear of a user 118.
  • the acoustic processor input 120 may be connected to one or more speaker terminals 124 via the switch 112.
  • the speech of user 118 of the mobile phone 100 may induce a signal on the terminal 124 of the speaker 114.
  • the speech signal may be processed by the acoustic processor 116 and the processed speech signal may be output on the acoustic processor output 122.
  • the response of the speaker used in this way may be direction dependent and so may be processed to, for example, discriminate between acoustic sources dependent on their location with respect to the front or back of the mobile phone 100, since the processed signal characteristics such as amplitude, frequency, gain may vary dependent on the orientation of the mobile phone 100 with respect to a particular acoustic source.
  • the processed signal may improve the quality of the speech signal or other desired source with respect to background acoustic sources which may be considered as noise sources.
  • the speaker 114 may be mounted within the housing in a so-called "open back" configuration whereby the speaker is not enclosed in a separate module but the housing 102 provides the back volume denoted V.
  • the mobile phone housing 102 may have unintentional and uncontrolled acoustic leak apertures, indicated as L 1-4 , which may result in an acoustical shortcut between the front side and the back side of the receiver speaker membrane 130. These acoustic leak apertures may be for example due to gaps in the mobile phone housing 102. Alternatively or in addition these leak apertures may be apertures formed in the housing for other purposes. Alternatively or in addition the mobile phone housing may be at least partially acoustically transparent due to the material and or thickness of areas of the housing.
  • the sound pressure wave may reach the front of the receiver speaker membrane 130 through the front port, denoted as acoustic path F, but also the back of the receiver speaker membrane 130 through the multiple leak paths indicated as R 1 to R 4 .
  • This acoustical shortcut may cause the receiver speaker to behave as a gradient pressure transducer, whereby the amplitude of the sound pressure wave picked-up (or transmitted) by the receiver speaker 114 may be a function of the angle of incidence of the sound wave.
  • the rear leak paths R1 to R4 may be undefined and dependent on the mechanical construction of the device.
  • the surprising result is that the speaker may have a directional response when configured as a microphone.
  • the directional response may make it possible to use the receiver speaker of a mobile phone as a directional microphone across the front-back axis and allow front-back sound discrimination, especially in applications where the mobile phone is held vertically such as during a video calling or while video recording or cam-cording.
  • Figure 3 shows a graph 200 of an example frequency response of a receiver speaker in a mobile phone when configured as a microphone.
  • the x-axis 208 shows the frequency varying between 0 Hz and approximately 10 kHz.
  • the y-axis 206 shows the gain in relative decibels between approximately -100 dB to -30 dB.
  • Frequency response 202 shows the response for the situation where the acoustic sources are located at the front of the mobile phone, that is to say the acoustic waves are travelling from the acoustic sources towards the major surface of the mobile phone containing the speaker aperture 104.
  • the frequency response 204 shows the response for the situation where the acoustic source is behind the mobile phone, that is to say the acoustic waves are travelling from the acoustic sources towards the major surface of the phone which is opposite the surface having the speaker aperture 104.
  • the gain for the front facing response 202 is significantly higher than the gain of the back facing response 204.
  • the gain for the front facing source is approximately -45 dB and the gain for the back-facing source is approximately -57 dB.
  • the gain for the front facing source is approximately -43 dB and the gain for the rear facing source is approximately -55 dB.
  • the gain for the front facing source is approximately -50 dB and the gain for the rear facing source is approximately -64 dB.
  • the gain for a front-facing source may be more than 6 dB higher than for a back facing acoustic source.
  • the desired acoustic source is located in front of the mobile device and a noise source is located at the back of the mobile device, it will be appreciated that this may improve the signal to noise ratio by more than 6 dB.
  • the terms front-facing and back-facing may be considered as relative terms used to refer to opposing major surfaces of the housing of the mobile phone.
  • FIG. 4 shows a mobile device 300.
  • the mobile device 300 may have a housing 308 and an aperture 312 in one of the surfaces of the housing 308 which may be considered as the front of the mobile device 300.
  • the mobile device 300 may have a microphone 314.
  • a speaker 306 may include a speaker membrane or diaphragm 328 and emit the acoustic signal, typically an audible signal which may for example contain speech and/or music through the aperture 310.
  • the speaker membrane 328 may have a first side or surface which may face towards the aperture 312 so may be considered for example front facing.
  • the speaker membrane 328 may have a second side or surface facing away from the aperture 310 so may be considered back facing.
  • the housing 308 may have a further aperture 326 which may provide a further acoustic path from the exterior of the mobile device 300 to the second side of the speaker membrane 328 via the back of the speaker 306.
  • the skilled person will appreciate that there may be an acoustic path through the back of the speaker to the second side of the speaker membrane 328 via the further aperture 326. Consequently the further aperture 326 in the housing 308 may provide an acoustic coupling between the second side of the speaker membrane 328 and the exterior of the mobile device 300.
  • the speaker 306 may be driven by an amplifier 302 which may be a class D audio amplifier connected to the speaker 306 via a switch 304.
  • the switch 304 may be controlled by a controller (not shown) and connect the amplifier 302 to the speaker 306.
  • the switch 304 may connect a first acoustic processor input 318 of the acoustic processor 312 to the speaker 306.
  • a microphone 314 which may be an omnidirectional microphone may be connected to a second acoustic processor input 320.
  • the acoustic processor 312 may be implemented in hardware/software or a combination of hardware and software.
  • the acoustic processor 312 may be implemented for example using software for execution on a digital signal processor.
  • the software may be stored in a memory such as RAM, ROM or EPROM or other non-transitory, tangible machine readable storage medium.
  • the acoustic processor 312 may apply filters, and/or equalization to a signal received on the first acoustic processor input 318 and a further signal received on the second acoustic processor input 320.
  • the acoustic processor 312 may apply threshold value to the acoustic signal that is to say compare the acoustic signal with an expected minimum level.
  • the acoustic processor 312, the amplifier 302 which may be a class D audio amplifier and the switch 304 are illustrated as being outside the housing 308 but may be contained within the housing 308 of the mobile device 300.
  • the amplifier 302 may be connected to the loudspeaker 306 via the switch 304.
  • the speaker 306 may then be used in its intended operation mode as a speaker for emitting an audio signal or other acoustic signal.
  • the acoustic processor input 312 may be connected to one or more speaker terminals 324 via the switch 304. Speech from a user 316 of the mobile device 300 may induce a signal on the terminals 324 of the speaker 306.
  • the speech signal may be processed by the acoustic processor 312 and the processed speech signal may be output on the acoustic processor output 322.
  • the characteristic of acoustic signals received via the speaker 306 used in this way may be direction dependent and so may be processed to discriminate between speech or other acoustic source dependent on the location of the acoustic source with respect to the front surface of the mobile device 300.
  • a further signal sensed via the microphone 314 may not be direction dependent.
  • the acoustic processor may mix the acoustic signal received from the speaker 306 with the further acoustic signal received from the microphone to reduce the background noise present in the processed signal.
  • the processed mixed signal from the output of the acoustic processor may have different polar characteristics than the characteristics of the microphone or speaker in isolation. Filtering or other beam forming techniques may be used to respond selectively to a sound source from a particular direction. Consequently the processed signal may improve the quality of the speech signal or other desired source with respect to background acoustic signals.
  • FIG. 5 shows an example of an acoustic processor 400.
  • An equaliser 402 may receive an input signal from a speaker.
  • the output 408 of the equaliser 402 may be connected to an ambient noise estimator 406.
  • the output 408 of equaliser 402 may also be connected to the mixer 404.
  • a microphone input 410 may be connected to the mixer 404 and the ambient noise estimator 406.
  • Equaliser 402 may align the speaker and microphone responses for a sound source.
  • the mixer output 412 may be the output of the acoustic processor 400.
  • the mixer 404 may delay and sum the responses of particular frequencies for the speaker and the microphone such that they interfere constructively or destructively for a specific location.
  • the mixer 404 may apply other beam forming techniques.
  • the resulting output of the mixer 404 may shape the polar pattern of the signal from the microphone input 410 and the signal from the equalizer output 408 which corresponds to the signal from the speaker used as a microphone.
  • the background noise level may be estimated by the ambient noise estimator 406.
  • the ambient noise estimator may determine an estimate of the background noise level from the signal to noise ratio value of the input signals.
  • the ambient noise estimator 406 may use either or both of the signals received via the equaliser output 408 or the microphone input 410.
  • Dependent on the background noise level the mixing ratio between the equalised speaker signal on equaliser output 408, and the microphone signal received on the microphone input 410, may be modified.
  • a low ambient noise level may be for example a level that corresponds to a signal to noise ratio value of above 20dB measured in decibels for the unprocessed signal.
  • a low ambient noise level may be for example a level that corresponds to a signal to noise ratio value of above 20dB measured in decibels for the unprocessed signal.
  • At a high ambient noise levels only the speaker may be used.
  • a high ambient noise level may for example be a noise level resulting in a signal to noise ratio value of -20dB or less measured in decibels for the unprocessed signal.
  • the processed output signal may include a mix of both signals received from the equaliser output 408 and the microphone input 410 may be used. The skilled person will appreciate that the noise estimation and the mixing may be performed in different frequency bands.
  • FIG. 6 shows a method of acoustic processing for a mobile phone 500.
  • an acoustic signal may be detected by a receiver speaker.
  • the acoustic signal may include a desired signal generated from an acoustic source such as a person speaking, a music source, or some other acoustic source.
  • the acoustic signal may include an undesired noise signal from other acoustic sources.
  • the acoustic signal may be processed to discriminate between a desired acoustic source located at the front of the mobile phone and an undesired acoustic source.
  • the acoustic signal characteristics of the processed signal received via the receiver speaker may vary dependent on whether the acoustic sources are located behind or in front of the mobile phone. These acoustic signal characteristics for example gain may be used to determine whether or not the acoustic source is behind or in front of the phone.
  • the receiver speaker when in a certain operating mode, for example in a mobile phone operating in hands-free mode, the receiver speaker may be configured as a microphone and used to improve the speech capture of a user of the mobile phone.
  • the method steps 500 may be implemented in hardware, software or a combination of hardware and software.
  • the method may be implemented in software running on a digital signal processor.
  • Figure 7 shows a method of acoustic processing for a mobile device 550.
  • a signal may be detected by a receiver speaker.
  • a further signal may be detected via a microphone.
  • the signal may be equalised. Equalisation may align the response of the receiver speaker and the microphone to a source located at the front of a mobile device.
  • the equalization may be an adaptive equalization.
  • the front of the mobile device may be a surface containing the aperture for the receiver speaker.
  • the signal and further signal may be mixed. Mixing the signal and the further signal in different proportions may improve the signal-to-noise ratio of the processed output signal dependent on the ambient noise level. By aligning the response of the microphone to that of the speaker, the response of the microphone may be more sensitive to an acoustic source located at the front of the mobile device.
  • Figure 8 shows a method of acoustic processing for directional sound capture 600 for a mobile device such as a mobile phone.
  • a signal may be detected via a speaker. This signal may be generated from in response to acoustic waves from an acoustic source.
  • a further signal may be detected via a microphone which may be an omnidirectional microphone.
  • the signal may be equalised.
  • a signal to noise ratio may be determined from at least one of the signal and the further signal.
  • a comparison may be made to determine whether the signal to noise ratio is less than a first predetermined threshold.
  • the method may move to step 612 and the further signal may be attenuated, and the signal may be amplified.
  • the attenuation of the further signal may be at a level at which the further signal is completely supressed.
  • the signal and attenuated further signal may be mixed or combined in step 618 to give the processed output signal.
  • the method may move to step 614 where the signal to noise ratio is compared to a second predetermined threshold.
  • step 614 if the signal to noise ratio is greater than the second predetermined threshold then the method moves to step 616, and the signal may be attenuated and the further signal may be amplified. The attenuation of the signal may be at a level at which the signal is completely supressed. Following step 616, the signal and further signal may be mixed or combined in step 618 to give the processed output signal. Returning to step 614 if the signal to noise ratio is less than the second predetermined threshold the signal and further signal may be mixed or combined in step 618 to give the processed output signal.
  • the processed signal may be dominated by the signal received via the speaker.
  • the processed signal may be dominated by the signal received via the microphone.
  • the mobile devices described herein may include a laptop, mobile phone, a wearable device such as a smart watch, a portable digital assistant, a laptop computer, a tablet computer or any other portable audio device having a speaker with an acoustic path which may be a leak path from the exterior of the mobile device to the back volume of the speaker.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Computer Hardware Design (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Telephone Function (AREA)
  • Soundproofing, Sound Blocking, And Sound Damping (AREA)
  • Circuit For Audible Band Transducer (AREA)

Claims (14)

  1. Mobilvorrichtung (100) mit: einem Akustikprozessor (116); einem Lautsprecher (114) einschließlich einer Lautsprechermembran (130), wobei die Lautsprechermembran eine erste Lautsprechermembranseite und eine zweite Lautsprechermembranseite hat, die der ersten Lautsprechermembranseite entgegengesetzt ist; einem Mobilvorrichtungsgehäuse (102) zum Bereitstellen eines ersten Akustikpfades zwischen der ersten Lautsprechermembranseite und der Außenseite der Mobilvorrichtung und eines zweiten Akustikpfades zwischen der zweiten Lautsprechermembranseite und der Außenseite der Mobilvorrichtung;
    wobei die Mobilvorrichtung dazu betreibbar ist, den Lautsprecher als ein Mikrofon zu konfigurieren, und
    der Akustikprozessor dazu konfiguriert und eingerichtet ist:
    ein Signal an einem Akustikprozessoreingang (120) zu erfassen, der zumindest mit einem Anschluss (124) des Lautsprechers gekoppelt ist;
    das induzierte Signal zu verarbeiten, um zwischen Akustikwellen aus verschiedenen Richtungen zu unterscheiden;
    und das verarbeitete Signal an einem Akustikprozessorausgang abzugeben.
  2. Mobilvorrichtung gemäß Anspruch 1, des Weiteren mit einem Mikrofon (314), wobei der Akustikprozessor des Weiteren dazu konfiguriert ist, das induzierte Signal mit einem weiteren Signal zu mischen, das über ein Mikrofon erfasst wird.
  3. Mobilvorrichtung gemäß Anspruch 2, wobei der Akustikprozessor des Weiteren dazu konfiguriert ist, das induzierte Signal anzugleichen und ein Beamforming auf das induzierte Signal und das weitere Signal anzuwenden.
  4. Mobilvorrichtung gemäß Anspruch 2 oder 3, wobei der Akustikprozessor des Weiteren dazu konfiguriert ist, ein Signal-zu-Stör-Verhältnis zumindest entweder aus dem induzierten Signal oder dem weiteren Signal zu bestimmen und das Mischverhältnis zwischen dem induzierten Signal und dem weiteren Signal in Abhängigkeit von dem Signal-zu-Stör-Verhältnis zu ändern.
  5. Mobilvorrichtung gemäß einem der vorherigen Ansprüche, wobei der Akustikprozessor des Weiteren dazu konfiguriert ist, das induzierte Signal zu verarbeiten, um zwischen einem Akustiksignal, das von einer Akustikquelle in einer ersten Richtung empfangen wird, und einem weiteren Akustiksignal zu unterscheiden, das von einer weiteren Akustikquelle in einer zweiten Richtung empfangen wird.
  6. Mobilvorrichtung gemäß einem der vorherigen Ansprüche, mit einer Kamera, wobei die Mobilvorrichtung in einem Camcorder-Betriebsmodus konfigurierbar ist und der Akustikprozessor des Weiteren dazu konfiguriert ist, das verarbeitete Signal zu speichern.
  7. Mobilvorrichtung gemäß einem der Ansprüche 1 bis 5, mit einer Kamera, und wobei die Mobilvorrichtung in einem Videoanrufmodus konfigurierbar ist, und die des Weiteren dazu konfiguriert ist, das verarbeitete Signal zu einem Empfänger eines Videoanrufs zu senden.
  8. Mobilvorrichtung gemäß einem der vorherigen Ansprüche, die entweder als ein Mobiltelefon, eine tragbare Vorrichtung, ein tragbares Audioabspielgerät, ein PDA, ein Laptop oder ein Tabletcomputer konfiguriert ist.
  9. Verfahren einer Akustikverarbeitung (500) für eine Mobilvorrichtung (100), wobei die Mobilvorrichtung einen Lautsprecher, der eine Lautsprechermembran hat, wobei die Lautsprechermembran eine erste Lautsprechermembranseite und eine zweite Lautsprechermembranseite hat, die der ersten Lautsprechermembranseite entgegengesetzt ist, ein Mobilvorrichtungsgehäuse zum akustischen Koppeln der jeweiligen Lautsprechermembranseite zu der Außenseite der Mobilvorrichtung aufweist,
    wobei das Verfahren Folgendes aufweist:
    Erfassen eines Signals (502), das zumindest an einem Anschluss des Lautsprechers als Reaktion auf Akustikwellen induziert wird,
    Verarbeiten des induzierten Signals (504), um zwischen Akustikwellen aus verschiedenen Richtungen zu unterscheiden; und
    Abgeben eines verarbeiteten Signals.
  10. Verfahren gemäß Anspruch 9, des Weiteren mit einem Erfassen eines weiteren Signals über ein Mikrofon und ein Kombinieren des induzierten Signals und des weiteren Signals.
  11. Verfahren gemäß Anspruch 10, des Weiteren mit einem Schätzen des Signal-zu-Stör-Verhältnisses aus dem weiteren Signal.
  12. Verfahren gemäß Anspruch 10 oder 11, des Weiteren mit einem Schätzen des Signal-zu-Stör-Verhältniswerts aus dem induzierten Signal.
  13. Verfahren gemäß einem der Ansprüche 10 bis 12, wobei ein Verarbeiten des induzierten Signals zumindest entweder ein Dämpfen des induzierten Signals für einen Signal-zu-Stör-Verhältniswert oberhalb eines vorbestimmten Schwellwerts oder ein Dämpfen des weiteren Signals für einen Signal-zu-Stör-Verhältniswert unterhalb des vorbestimmten Schwellwerts aufweist.
  14. Bei einer Mobilvorrichtung, die einen Lautsprecher, der eine Lautsprechermembran hat, wobei die Lautsprechermembran eine erste Lautsprechermembranseite und eine zweite Lautsprechermembranseite hat, die der ersten Lautsprechermembranseite entgegengesetzt ist, ein Mobilvorrichtungsgehäuse zum Bereitstellen eines ersten Akustikpfades zwischen der ersten Lautsprechermembranseite und der Außenseite der Mobilvorrichtung und eines zweiten Akustikpfades zwischen der zweiten Lautsprechermembranseite und der Außenseite der Mobilvorrichtung aufweist,
    die Verwendung des Lautsprechers als ein Richtmikrofon.
EP15191381.1A 2015-10-26 2015-10-26 Akustischer prozessor für eine mobile vorrichtung Active EP3163903B1 (de)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP15191381.1A EP3163903B1 (de) 2015-10-26 2015-10-26 Akustischer prozessor für eine mobile vorrichtung
CN201610916146.XA CN106919225B (zh) 2015-10-26 2016-10-20 用于移动装置的声处理器
US15/333,897 US9866958B2 (en) 2015-10-26 2016-10-25 Accoustic processor for a mobile device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP15191381.1A EP3163903B1 (de) 2015-10-26 2015-10-26 Akustischer prozessor für eine mobile vorrichtung

Publications (2)

Publication Number Publication Date
EP3163903A1 EP3163903A1 (de) 2017-05-03
EP3163903B1 true EP3163903B1 (de) 2019-06-19

Family

ID=54360210

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15191381.1A Active EP3163903B1 (de) 2015-10-26 2015-10-26 Akustischer prozessor für eine mobile vorrichtung

Country Status (3)

Country Link
US (1) US9866958B2 (de)
EP (1) EP3163903B1 (de)
CN (1) CN106919225B (de)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10462567B2 (en) * 2016-10-11 2019-10-29 Ford Global Technologies, Llc Responding to HVAC-induced vehicle microphone buffeting
US10112551B2 (en) * 2016-12-19 2018-10-30 Bruce Lee Manes Quick release steering wheel knob with bluetooth audio transducer
US10525921B2 (en) 2017-08-10 2020-01-07 Ford Global Technologies, Llc Monitoring windshield vibrations for vehicle collision detection
US10049654B1 (en) 2017-08-11 2018-08-14 Ford Global Technologies, Llc Accelerometer-based external sound monitoring
US10308225B2 (en) 2017-08-22 2019-06-04 Ford Global Technologies, Llc Accelerometer-based vehicle wiper blade monitoring
US10562449B2 (en) 2017-09-25 2020-02-18 Ford Global Technologies, Llc Accelerometer-based external sound monitoring during low speed maneuvers
US10479300B2 (en) 2017-10-06 2019-11-19 Ford Global Technologies, Llc Monitoring of vehicle window vibrations for voice-command recognition
CN108335701B (zh) * 2018-01-24 2021-04-13 青岛海信移动通信技术股份有限公司 一种进行声音降噪的方法及设备
US11304001B2 (en) 2019-06-13 2022-04-12 Apple Inc. Speaker emulation of a microphone for wind detection
CN112383655B (zh) * 2020-11-02 2022-07-12 Oppo广东移动通信有限公司 电子设备、电子设备的声音增强方法以及存储介质

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6473733B1 (en) * 1999-12-01 2002-10-29 Research In Motion Limited Signal enhancement for voice coding
US20050136848A1 (en) * 2003-12-22 2005-06-23 Matt Murray Multi-mode audio processors and methods of operating the same
US7912230B2 (en) * 2004-06-16 2011-03-22 Panasonic Corporation Howling detection device and method
US8509459B1 (en) * 2005-12-23 2013-08-13 Plantronics, Inc. Noise cancelling microphone with reduced acoustic leakage
US9037468B2 (en) * 2008-10-27 2015-05-19 Sony Computer Entertainment Inc. Sound localization for user in motion
US9838784B2 (en) * 2009-12-02 2017-12-05 Knowles Electronics, Llc Directional audio capture
JP5084945B2 (ja) * 2011-01-07 2012-11-28 シャープ株式会社 表示装置
US8750528B2 (en) * 2011-08-16 2014-06-10 Fortemedia, Inc. Audio apparatus and audio controller thereof
JP5929154B2 (ja) * 2011-12-15 2016-06-01 富士通株式会社 信号処理装置、信号処理方法および信号処理プログラム
US9008344B2 (en) * 2013-03-14 2015-04-14 Cirrus Logic, Inc. Systems and methods for using a speaker as a microphone in a mobile device
KR20150104808A (ko) * 2014-03-06 2015-09-16 삼성전자주식회사 피드백을 출력하는 전자 장치 및 방법
CN204697289U (zh) * 2015-03-23 2015-10-07 钰太芯微电子科技（上海）有限公司 基于麦克风的声源识别系统及智能家电设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
CN106919225A (zh) 2017-07-04
CN106919225B (zh) 2021-07-16
US9866958B2 (en) 2018-01-09
US20170118556A1 (en) 2017-04-27
EP3163903A1 (de) 2017-05-03

Similar Documents

Publication Publication Date Title
US9866958B2 (en) Accoustic processor for a mobile device
KR101566649B1 (ko) 근거리 널 및 빔 형성
US8098844B2 (en) Dual-microphone spatial noise suppression
US8908880B2 (en) Electronic apparatus having microphones with controllable front-side gain and rear-side gain
US10269369B2 (en) System and method of noise reduction for a mobile device
US9525938B2 (en) User voice location estimation for adjusting portable device beamforming settings
US9020163B2 (en) Near-field null and beamforming
US10176823B2 (en) System and method for audio noise processing and noise reduction
EP3864858B1 (de) Direktionale audioaufnahme in kollaborationsendpunkten
WO2018158558A1 (en) Device for capturing and outputting audio
WO2007059255A1 (en) Dual-microphone spatial noise suppression
US10425733B1 (en) Microphone equalization for room acoustics
US10109292B1 (en) Audio systems with active feedback acoustic echo cancellation
CN114762361A (zh) 使用扬声器作为传声器之一的双向传声器系统
CN102970638B (zh) 处理信号
WO2021061055A1 (en) Transducer apparatus: positioning and high signal-to-noise-ratio microphones
US20230097305A1 (en) Audio device with microphone sensitivity compensator
US20230101635A1 (en) Audio device with distractor attenuator
US11700485B2 (en) Differential audio data compensation
EP4156183A1 (de) Audiovorrichtung mit mehreren dämpfungsgliedern
US20240236551A1 (en) Loudspeaker assembly and hand-held device
CN118158590A (zh) 开放式可穿戴声学设备及主动降噪方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20171103

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 1/32 20060101AFI20190108BHEP

Ipc: H04R 3/00 20060101ALI20190108BHEP

INTG Intention to grant announced

Effective date: 20190205

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1147034

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190715

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602015032148

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190920

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1147034

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191021

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191019

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200224

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602015032148

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG2D Information on lapse in contracting state deleted

Ref country code: IS

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191026

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

26N No opposition filed

Effective date: 20200603

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602015032148

Country of ref document: DE

Owner name: GOODIX TECHNOLOGY (HK) COMPANY LIMITED, CN

Free format text: FORMER OWNER: NXP B.V., EINDHOVEN, NL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20191026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191026

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20151026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231026

Year of fee payment: 9

Ref country code: DE

Payment date: 20231020

Year of fee payment: 9