TWI407804B - Phase calibration module, voice processing apparatus, and method for calibrating phase mismatch - Google Patents

Phase calibration module, voice processing apparatus, and method for calibrating phase mismatch Download PDF

Info

Publication number
TWI407804B
TWI407804B TW099100780A TW99100780A TWI407804B TW I407804 B TWI407804 B TW I407804B TW 099100780 A TW099100780 A TW 099100780A TW 99100780 A TW99100780 A TW 99100780A TW I407804 B TWI407804 B TW I407804B
Authority
TW
Taiwan
Prior art keywords
frequency component
signals
low frequency
component signals
microphone
Prior art date
Application number
TW099100780A
Other languages
Chinese (zh)
Other versions
TW201028023A (en
Inventor
Ming Zhang
Xiaoyan Lu
Original Assignee
Fortemedia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fortemedia Inc filed Critical Fortemedia Inc
Publication of TW201028023A publication Critical patent/TW201028023A/en
Application granted granted Critical
Publication of TWI407804B publication Critical patent/TWI407804B/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00Circuits for transducers, loudspeakers or microphones
    • H04R3/005Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166Microphone arrays; Beamforming
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78Detection of presence or absence of voice signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)

Abstract

The invention provides a phase calibration module, calibrating phase mismatch between microphone signals output by a plurality of microphones of an array microphone. In one embodiment, the phase calibration module comprises a subband filter, a delay calculation module, and a delay compensation filter. The subband filter extracts a high frequency component and a low frequency component from each of the microphone signals to obtain a plurality of high-frequency component signals and a plurality of low-frequency component signals. The delay calculation module calculates delays between the low-frequency component signals. The delay compensation filter then compensates the low-frequency component signals for phase mismatches therebetween according to the calculated delays to obtain a plurality of calibrated low-frequency component signals.

Description

相位校正模組、語音處理裝置、及校正相位不匹配的方法Phase correction module, voice processing device, and method for correcting phase mismatch

本發明係有關於陣列麥克風,特別是有關於陣列麥克風之輸出信號的相位不匹配之校正。The present invention relates to array microphones, and in particular to corrections for phase mismatch of the output signals of the array microphones.

陣列麥克風(array microphone)為包含多個麥克風的一裝置。當一聲波傳遞至一陣列麥克風時,陣列麥克風所包含的每一麥克風都會將該聲波轉換為一麥克風信號,因此陣列麥克風可同時產生多個麥克風信號。由於該等麥克風接收聲波的位置有些許差別,該等麥克風所產生的麥克風信號的相位有些許差異。一波束成型(beamforming)模組因此可以依據該等麥克風信號間的相位差決定該聲波的接收方向,並自麥克風信號中律除來自接收方向以外的噪音及干擾。因此,波束成形模組可產生包含較多聲波成分及較少噪音及干擾成份的一目標信號。An array microphone is a device that includes a plurality of microphones. When a sound wave is transmitted to an array microphone, each microphone included in the array microphone converts the sound wave into a microphone signal, so the array microphone can simultaneously generate a plurality of microphone signals. Since the positions of the sound waves received by the microphones are slightly different, the phases of the microphone signals generated by the microphones are slightly different. A beamforming module can therefore determine the direction of reception of the sound wave based on the phase difference between the microphone signals, and remove noise and interference from the receiving direction from the microphone signal. Therefore, the beamforming module can generate a target signal containing more acoustic components and less noise and interference components.

由於波束成形模組係依據麥克風信號間的相位差決定聲波的接收方向,因此麥克風信號間的相位差之精確程度決定了目標信號包含聲波成分的多寡,亦即決定了波束成型模組所產生的目標信號之品質。然而,陣列麥克風所產生的多個麥克風信號間的相位差包含了各麥克風的電路差異所導致的信號延遲時間,而並非完整的反映了各麥克風的接收位置之空間差異。因此,各麥克風的電路差異會使波束成型模組所產生的目標信號之品質下降。因此,需要一相位校正模組以補償陣列麥克風所輸出的麥克風信號間的由於麥克風電路差異而導致的延遲時間差異。Since the beamforming module determines the receiving direction of the sound wave according to the phase difference between the microphone signals, the accuracy of the phase difference between the microphone signals determines the amount of the sound component of the target signal, that is, determines the beamforming module. The quality of the target signal. However, the phase difference between the plurality of microphone signals generated by the array microphone includes the signal delay time caused by the circuit difference of each microphone, and does not completely reflect the spatial difference of the receiving positions of the respective microphones. Therefore, the circuit difference of each microphone will degrade the quality of the target signal generated by the beamforming module. Therefore, a phase correction module is needed to compensate for the difference in delay time between microphone signals output by the array microphone due to microphone circuit differences.

習知的相位校正模組106依據陣列麥克風的多個麥克風的輸出信號決定麥克風的電路差異所導致的延遲時間差異。然而,陣列麥克風的多個麥克風的電路差異於麥克風輸出信號的低頻成分中引起較大的延遲時間差異,而於麥克風輸出信號的高頻成分中引起較小的延遲時間差異。因此,麥克風輸出信號的低頻成分會較麥克風輸出信號的高頻成分包含較多麥克風電路差異所導致的相位差及信號失真。由於習知的相位校正模組於計算電路差異所導致的延遲時間差異時並不區別對待麥克風輸出信號的高頻成分與低頻成分,因此其所計算得到的延遲時間差異並不具有高的精確度,因而使波束成型模組所產生的目標信號之品質下降。因此,需要一種相位校正模組以校正一陣列麥克風所包括的多個麥克風所輸出的多個麥克風信號之間的相位不匹配。The conventional phase correction module 106 determines the delay time difference caused by the circuit difference of the microphone according to the output signals of the plurality of microphones of the array microphone. However, the circuit of the plurality of microphones of the array microphone causes a large delay time difference in the low frequency component of the microphone output signal, and causes a small delay time difference in the high frequency component of the microphone output signal. Therefore, the low frequency component of the microphone output signal may contain a phase difference and signal distortion caused by a difference in the microphone circuit compared to the high frequency component of the microphone output signal. Since the conventional phase correction module does not discriminate between the high frequency component and the low frequency component of the microphone output signal when calculating the difference in delay time caused by the difference in the circuit, the calculated delay time difference does not have high accuracy. Therefore, the quality of the target signal generated by the beamforming module is degraded. Therefore, there is a need for a phase correction module to correct for phase mismatch between multiple microphone signals output by a plurality of microphones included in an array of microphones.

有鑑於此,本發明之目的在於提供一種相位校正模組,以校正一陣列麥克風所包括的多個麥克風所輸出的多個麥克風信號之間的相位不匹配。於一實施例中,該相位校正模組包括一次頻帶濾波器、一延遲時間計算模組、以及一延遲補償濾波器。該次頻帶濾波器自該等麥克風信號分別取出一高頻成分及一低頻成分,以得到多個高頻成分信號以及多個低頻成分信號。該延遲時間計算模組計算該等低頻成分信號間的延遲時間。該延遲補償濾波器依據該等延遲時間補償該等低頻成分信號間之相位不匹配以得到多個校正低頻成分信號。In view of the above, an object of the present invention is to provide a phase correction module for correcting a phase mismatch between a plurality of microphone signals output by a plurality of microphones included in an array microphone. In one embodiment, the phase correction module includes a primary band filter, a delay time calculation module, and a delay compensation filter. The subband filter extracts a high frequency component and a low frequency component from the microphone signals to obtain a plurality of high frequency component signals and a plurality of low frequency component signals. The delay time calculation module calculates a delay time between the low frequency component signals. The delay compensation filter compensates for phase mismatch between the low frequency component signals according to the delay times to obtain a plurality of corrected low frequency component signals.

本發明提供一種校正陣列麥克風之相位不匹配的方法。於一實施例中,該陣列麥克風所包括的多個麥克風將一聲音信號轉換為多個麥克風信號。首先,自該等麥克風信號分別取出一高頻成分及一低頻成分,以得到多個高頻成分信號以及多個低頻成分信號。接著,計算該等低頻成分信號間的延遲時間。最後,依據該等延遲時間補償該等低頻成分信號間之相位不匹配,以得到多個校正低頻成分信號。The present invention provides a method of correcting the phase mismatch of an array microphone. In one embodiment, the plurality of microphones included in the array microphone convert a sound signal into a plurality of microphone signals. First, a high frequency component and a low frequency component are respectively extracted from the microphone signals to obtain a plurality of high frequency component signals and a plurality of low frequency component signals. Next, the delay time between the low frequency component signals is calculated. Finally, the phase mismatch between the low frequency component signals is compensated according to the delay times to obtain a plurality of corrected low frequency component signals.

本發明更提供一種語音處理裝置。於一實施例中,該語音處理裝置包括一陣列麥克風、一相位校正模組、以及一波束成型/信號分離模組。該陣列麥克風包括多個麥克風供產生多個麥克風信號。該相位校正模組自該等麥克風信號分別取出一高頻成分及一低頻成分以得到多個高頻成分信號以及多個低頻成分信號,計算該等低頻成分信號間的延遲時間,並依據該等延遲時間補償該等低頻成分信號間之相位不匹配以得到多個校正低頻成分信號。該波束成型/信號分離模組藉由波束成型(beamforming)技術或信號分離(signal separation)技術依據該等校正信號產生無噪音及干擾成分的一目標信號。The invention further provides a speech processing device. In one embodiment, the voice processing device includes an array microphone, a phase correction module, and a beamforming/signal separation module. The array microphone includes a plurality of microphones for generating a plurality of microphone signals. The phase correction module extracts a high frequency component and a low frequency component from the microphone signals to obtain a plurality of high frequency component signals and a plurality of low frequency component signals, and calculates a delay time between the low frequency component signals, and according to the The delay time compensates for phase mismatch between the low frequency component signals to obtain a plurality of corrected low frequency component signals. The beamforming/signal separation module generates a target signal without noise and interference components according to the beamforming technology or signal separation technique according to the correction signals.

為了讓本發明之上述和其他目的、特徵、和優點能更明顯易懂,下文特舉數較佳實施例,並配合所附圖示,作詳細說明如下:The above and other objects, features, and advantages of the present invention will become more apparent and understood.

第1圖為依據本發明之語音處理裝置100之區塊圖。語音處理裝置100包括麥克風102及103、類比至數位轉換器104及105、相位校正模組106、以及波束成型/信號分離模組108。假設一聲音訊號源距離麥克風102及103為等距。因此,當一聲波產生時,麥克風102及103會同時收到該聲波。麥克風102及103分別轉換聲波為信號s1(t)及s2(t)。類比至數位轉換器104、105接著轉換類比信號s1(t)及s2(t)為數位信號s1(n)及s2(n)。1 is a block diagram of a speech processing apparatus 100 in accordance with the present invention. The speech processing device 100 includes microphones 102 and 103, analog to digital converters 104 and 105, a phase correction module 106, and a beamforming/signal separation module 108. Assume that an audio signal source is equidistant from the microphones 102 and 103. Therefore, when a sound wave is generated, the microphones 102 and 103 receive the sound wave at the same time. The microphones 102 and 103 respectively convert the sound waves into signals s1(t) and s2(t). The analog to digital converters 104, 105 then convert the analog signals s1(t) and s2(t) to digital signals s1(n) and s2(n).

由於聲音訊號源距離麥克風102及103為等距,因此麥克風102及103的接收位置差別不會對信號s1(n)及s2(n)產生相位差或延遲時間差異。當信號s1(n)及s2(n)間存在延遲時間差異時,該延遲時間差必然係由麥克風102及103間的電路差異所造成。相位校正模組106接著可計算信號s1(n)及s2(n)間的延遲時間差。於計算延遲時間差之前,相位校正模組106自信號s1(n)及s2(n)分別抽取高頻成分極低頻成分。接著,相位校正模組106偵測是否高頻成分中包含語音成分。若高頻成分中包含語音成分,則相位校正模組106量測低頻成分之間的延遲時間差,並依據該延遲時間差補償信號s1(n)及s2(n)間的相位不匹配。由於僅有兩麥克風輸出信號s1(n)及s2(n),因此僅有信號s1(n)及s2(n)其中之一需要補償。舉例來說,信號s1(n)的相位被依據延遲時間差調整以得到一校正信號s1c(n)。當陣列麥克風包括多個麥克風時,多個麥克風產生麥克風輸出信號,因此相位校正模組106以同樣方式調整多個麥克風輸出信號的相位。Since the sound signal source is equidistant from the microphones 102 and 103, the difference in the receiving positions of the microphones 102 and 103 does not cause a phase difference or a delay time difference between the signals s1(n) and s2(n). When there is a difference in delay time between the signals s1(n) and s2(n), the delay time difference is necessarily caused by the circuit difference between the microphones 102 and 103. The phase correction module 106 can then calculate the delay time difference between the signals s1(n) and s2(n). Before calculating the delay time difference, the phase correction module 106 extracts the high frequency component extremely low frequency components from the signals s1(n) and s2(n), respectively. Next, the phase correction module 106 detects whether a high frequency component contains a speech component. If the high frequency component includes a speech component, the phase correction module 106 measures the delay time difference between the low frequency components and compensates for the phase mismatch between the signals s1(n) and s2(n) according to the delay time difference. Since only two microphones output signals s1(n) and s2(n), only one of the signals s1(n) and s2(n) needs to be compensated. For example, the phase of the signal s1(n) is adjusted according to the delay time difference to obtain a correction signal s1c(n). When the array microphone includes a plurality of microphones, the plurality of microphones generate a microphone output signal, and thus the phase correction module 106 adjusts the phases of the plurality of microphone output signals in the same manner.

信號s1c(n)及s2(n)接著被送至波束成型/信號分離模組108。波束成型/信號分離模組108接著藉著波束成型技術或信號分離技術依據信號s1c(n)及s2(n)產生具有較多語音成分並經衰減噪音及干擾成分的目標信號d(n)。由於相位校正模組106係量測信號s1(n)及s2(n)的低頻成分的延遲時間差以進行相位校正,因此相位校正模組106所量得的延遲時間差較習知技術中來的精確。因此,信號s1c(n)及s2(n)之間由麥克風102及103間的電路差異所導致的延遲時間差可被完全補償。因此,信號s1c(n)及s2(n)之間的相位差可完整地反映麥克風102及103間的接收位置的空間差異,從而提升波束成型/信號分離模組108所產生的目標信號d(n)的精確度。Signals s1c(n) and s2(n) are then sent to beamforming/signal separation module 108. The beamforming/signal separation module 108 then generates a target signal d(n) having more speech components and attenuating noise and interference components based on the signals s1c(n) and s2(n) by beamforming techniques or signal separation techniques. Since the phase correction module 106 measures the delay time difference of the low frequency components of the signals s1(n) and s2(n) for phase correction, the delay time difference measured by the phase correction module 106 is more accurate than that of the prior art. . Therefore, the delay time difference caused by the circuit difference between the microphones 102 and 103 between the signals s1c(n) and s2(n) can be completely compensated. Therefore, the phase difference between the signals s1c(n) and s2(n) can completely reflect the spatial difference of the receiving positions between the microphones 102 and 103, thereby improving the target signal d generated by the beamforming/signal separation module 108 ( n) accuracy.

第2圖為依據本發明之相位校正模組200的區塊圖。相位校正模組200包括次頻帶濾波器202、語音偵測器204、延遲時間計算模組206、以及延遲濾波器208。由麥克風102及103所產生的信號s1(n)及s2(n)首先被送至次頻帶濾波器202。次頻帶濾波器202將信號s1(n)分為高頻成分信號s1h(n)及低頻成分信號s1l(n),並將信號s2(n)分為高頻成分信號s2h(n)及低頻成分信號s2l(n)。於一實施例中,次頻帶濾波器202包含一高通濾波器及一低通濾波器。高通濾波器有等於界限頻率之一截角頻率,用以過濾信號s1(n)及s2(n)以產生高頻成分信號s1h(n)及s2h(n)。低通濾波器有等於界限頻率之一截角頻率,用以過濾信號s1(n)及s2(n)以產生低頻成分信號s1l(n)及s2l(n)。於一實施例中,該界限頻率之範圍可由500Hz至1000Hz。2 is a block diagram of a phase correction module 200 in accordance with the present invention. The phase correction module 200 includes a subband filter 202, a speech detector 204, a delay time calculation module 206, and a delay filter 208. The signals s1(n) and s2(n) generated by the microphones 102 and 103 are first sent to the subband filter 202. The sub-band filter 202 divides the signal s1(n) into a high-frequency component signal s1h(n) and a low-frequency component signal s1l(n), and divides the signal s2(n) into a high-frequency component signal s2h(n) and a low-frequency component. Signal s2l(n). In one embodiment, the subband filter 202 includes a high pass filter and a low pass filter. The high pass filter has a truncated frequency equal to one of the limit frequencies for filtering the signals s1(n) and s2(n) to generate high frequency component signals s1h(n) and s2h(n). The low pass filter has a truncated frequency equal to one of the limit frequencies for filtering the signals s1(n) and s2(n) to produce low frequency component signals s1l(n) and s2l(n). In an embodiment, the limit frequency can range from 500 Hz to 1000 Hz.

語音偵測器204接著偵測是否高頻成分信號s1h(n)及s2h(n)包含語音成分。若高頻成分信號s1h(n)及s2h(n)包含語音成分,則語音偵測器204產生一語音偵測信號v(n)以致能延遲時間計算模組206計算延遲時間。於一實施例中,語音偵測器204偵測是否高頻成分信號s1h(n)及s2h(n)的功率高於一功率界限值。若高頻成分信號s1h(n)及s2h(n)的功率高於功率界限值,則語音偵測器204決定高頻成分信號s1h(n)及s2h(n)包含語音成分,並致能語音偵測信號v(n)以驅動延遲時間計算模組206。The voice detector 204 then detects whether the high frequency component signals s1h(n) and s2h(n) contain speech components. If the high frequency component signals s1h(n) and s2h(n) contain speech components, the speech detector 204 generates a speech detection signal v(n) to enable the delay time calculation module 206 to calculate the delay time. In one embodiment, the voice detector 204 detects whether the power of the high frequency component signals s1h(n) and s2h(n) is higher than a power limit value. If the power of the high frequency component signals s1h(n) and s2h(n) is higher than the power limit value, the speech detector 204 determines that the high frequency component signals s1h(n) and s2h(n) contain speech components and enables speech. The signal v(n) is detected to drive the delay time calculation module 206.

當語音偵測信號v(n)被致能後,延遲時間計算模組206接著計算低頻成分信號s1l(n)及s2l(n)間的延遲時間差t(n)。一實施例中,延遲時間計算模組206對低頻成分信號s1l(n)及s2l(n)進行相關性運算(correlation),以計算低頻成分信號s1l(n)及s2l(n)間的延遲時間差t(n)。由於本實施例僅包含兩麥克風輸出信號s1(n)及s2(n),僅有麥克風輸出信號s1(n)及s2(n)的其中之一需要校正以消除兩者間的相位差或延遲時間差。延遲時間差t(n)接著被送至延遲濾波器208,而延遲濾波器208接著依據延遲時間差t(n)校正低頻成分信號s1l(n)以得到校正低頻成分信號s1lc(n)。校正低頻成分信號s1lc(n)及高頻成分信號s1h(n)合而為第1圖所示的校正信號s1c(n)。因此,於校正信號s1c(n)與信號s2(n)間不存在麥克風102及103或類比至數位轉換器104及105間的電路差異所倒置的延遲時間差或相位差。接著,波束成型/信號分離模組108可依據校正信號s1c(n)與信號s2(n)產生精確的目標信號d(n)。When the voice detection signal v(n) is enabled, the delay time calculation module 206 then calculates the delay time difference t(n) between the low frequency component signals s1l(n) and s2l(n). In one embodiment, the delay time calculation module 206 performs a correlation operation on the low frequency component signals s1l(n) and s2l(n) to calculate a delay time difference between the low frequency component signals s1l(n) and s2l(n). t(n). Since this embodiment includes only two microphone output signals s1(n) and s2(n), only one of the microphone output signals s1(n) and s2(n) needs to be corrected to eliminate the phase difference or delay between the two. Time difference. The delay time difference t(n) is then sent to the delay filter 208, which in turn corrects the low frequency component signal s1l(n) according to the delay time difference t(n) to obtain the corrected low frequency component signal s1lc(n). The corrected low-frequency component signal s1lc(n) and the high-frequency component signal s1h(n) are combined to be the correction signal s1c(n) shown in FIG. Therefore, there is no delay time difference or phase difference inverted between the microphones 102 and 103 or the analog to digital converters 104 and 105 between the correction signal s1c(n) and the signal s2(n). Next, the beamforming/signal separation module 108 can generate an accurate target signal d(n) according to the correction signal s1c(n) and the signal s2(n).

第3圖為依據本發明之校正陣列麥克風之相位不匹配的方法300的流程圖。首先,接收由一陣列麥克風的多個麥克風轉換一聲音所得到的多個麥克風信號(步驟302)。接著,自該等麥克風信號分別取出一高頻成分及一低頻成分以得到多個高頻成分信號及多個低頻成分信號(步驟304)。接著,決定是否該等高頻成分信號包含語音成分(步驟306)。若該等高頻成分信號包含語音成分,則計算該等低頻成分信號間的多個延遲時間(步驟308)。接著,依據該等延遲時間校正該等麥克風信號間的相位不匹配以得到多個校正信號(步驟310)。最後,以波束成型技術或信號分離技術依據該等校正信號產生不具噪音及干擾成分的一目標信號(步驟312)。Figure 3 is a flow diagram of a method 300 of correcting phase mismatch of array microphones in accordance with the present invention. First, a plurality of microphone signals obtained by converting a sound from a plurality of microphones of an array microphone are received (step 302). Next, a high frequency component and a low frequency component are respectively extracted from the microphone signals to obtain a plurality of high frequency component signals and a plurality of low frequency component signals (step 304). Next, it is determined whether or not the high frequency component signals include speech components (step 306). If the high frequency component signals include speech components, a plurality of delay times between the low frequency component signals are calculated (step 308). Then, the phase mismatch between the microphone signals is corrected according to the delay times to obtain a plurality of correction signals (step 310). Finally, a target signal having no noise and interference components is generated based on the correction signals by beamforming techniques or signal separation techniques (step 312).

雖然本發明已以較佳實施例揭露如上,然其並非用以限定本發明,任何熟習此項技術者,在不脫離本發明之精神和範圍內,當可作些許之更動與潤飾,因此本發明之保護範圍當視後附之申請專利範圍所界定者為準。Although the present invention has been disclosed in the above preferred embodiments, it is not intended to limit the invention, and it is intended that the invention may be modified and modified without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

(第1圖)(Figure 1)

102,103...麥克風102,103. . . microphone

104,105...類比至數位轉換器104,105. . . Analog to digital converter

106...相位校正模組106. . . Phase correction module

108...波束成型/信號分離模組108. . . Beamforming / signal separation module

(第2圖)(Fig. 2)

202...次頻帶濾波器202. . . Secondary band filter

204...語音偵測器204. . . Voice detector

206...延遲時間計算模組206. . . Delay time calculation module

208...延遲濾波器208. . . Delay filter

第1圖為依據本發明之語音處理裝置之區塊圖;Figure 1 is a block diagram of a speech processing apparatus in accordance with the present invention;

第2圖為依據本發明之相位校正模組的區塊圖;以及2 is a block diagram of a phase correction module in accordance with the present invention;

第3圖為依據本發明之校正陣列麥克風之相位不匹配Figure 3 is a phase mismatch of the corrected array microphone in accordance with the present invention.

的方法的流程圖。Flow chart of the method.

100...語音處理裝置100. . . Voice processing device

102、103...麥克風102, 103. . . microphone

104、105...類比至數位轉換器104, 105. . . Analog to digital converter

106...相位校正模組106. . . Phase correction module

108...波束成型/信號分離模組108. . . Beamforming / signal separation module

Claims (22)

一種相位校正模組,用以校正一陣列麥克風所包括的多個麥克風所輸出的多個麥克風信號之間的相位不匹配,包括:一次頻帶濾波器,自該等麥克風信號分別取出一高頻成分及一低頻成分,以得到多個高頻成分信號以及多個低頻成分信號;一延遲時間計算模組,計算該等低頻成分信號間的延遲時間;以及一延遲補償濾波器,依據該等延遲時間補償該等低頻成分信號間之相位不匹配以得到多個校正低頻成分信號。A phase correction module for correcting a phase mismatch between a plurality of microphone signals outputted by a plurality of microphones included in an array microphone, comprising: a primary band filter, respectively extracting a high frequency component from the microphone signals And a low frequency component for obtaining a plurality of high frequency component signals and a plurality of low frequency component signals; a delay time calculation module for calculating a delay time between the low frequency component signals; and a delay compensation filter according to the delay times Compensating for phase mismatch between the low frequency component signals to obtain a plurality of corrected low frequency component signals. 如申請專利範圍第1項所述之相位校正模組,其中該次頻帶濾波器包括一高通濾波器及一低通濾波器,該高通濾波器依據一界限頻率過濾該等麥克風信號以得到該等高頻成分信號,而該低通濾波器依據該界限頻率過濾該等麥克風信號以得到該等低頻成分信號。The phase correction module of claim 1, wherein the subband filter comprises a high pass filter and a low pass filter, the high pass filter filtering the microphone signals according to a limit frequency to obtain the same a high frequency component signal, and the low pass filter filters the microphone signals according to the threshold frequency to obtain the low frequency component signals. 如申請專利範圍第2項所述之相位校正模組,其中該界限頻率的範圍由500Hz至1000Hz。The phase correction module of claim 2, wherein the limit frequency ranges from 500 Hz to 1000 Hz. 如申請專利範圍第1項所述之相位校正模組,其中該相位校正模組包括一語音偵測器,用以偵測是否該等高頻成分信號包含語音成分以產生一語音偵測信號,而該語音偵測信號用以啟動該延遲時間計算模組之延遲時間之計算。The phase correction module of claim 1, wherein the phase correction module comprises a voice detector for detecting whether the high frequency component signals comprise voice components to generate a voice detection signal. The voice detection signal is used to start the calculation of the delay time of the delay time calculation module. 如申請專利範圍第4項所述之相位校正模組,其中該語音偵測器偵測是否該等高頻成分信號之功率超過一功率界限值以決定是否致能該語音偵測信號。The phase correction module of claim 4, wherein the voice detector detects whether the power of the high frequency component signals exceeds a power limit value to determine whether the voice detection signal is enabled. 如申請專利範圍第1項所述之相位校正模組,其中該延遲計算模組將該等低頻成分信號進行相關性計算(correlation)以得到該等延遲時間。The phase correction module of claim 1, wherein the delay calculation module performs correlation correlation on the low frequency component signals to obtain the delay times. 如申請專利範圍第1項所述之相位校正模組,其中該等高頻成分信號及該等校正低頻成分信號經混合後形成分別對應於該等麥克風信號之多個校正信號,而與該相位校正模組串接之一波束成型(beamforming)/信號分離(signal separation)模組接著藉由波束成型技術或信號分離技術依據該等校正信號產生無噪音及干擾成分的一目標信號。The phase correction module of claim 1, wherein the high frequency component signals and the corrected low frequency component signals are mixed to form a plurality of correction signals respectively corresponding to the microphone signals, and the phase A beamforming/signal separation module of the calibration module is then followed by a beamforming technique or a signal separation technique to generate a target signal without noise and interference components based on the correction signals. 一種校正陣列麥克風之相位不匹配的方法,其中該陣列麥克風所包括的多個麥克風將一聲音信號轉換為多個麥克風信號,該方法包括:自該等麥克風信號分別取出一高頻成分及一低頻成分,以得到多個高頻成分信號以及多個低頻成分信號;計算該等低頻成分信號間的延遲時間;以及依據該等延遲時間補償該等低頻成分信號間之相位不匹配,以得到多個校正低頻成分信號。A method for correcting phase mismatch of an array microphone, wherein a plurality of microphones included in the array microphone convert a sound signal into a plurality of microphone signals, the method comprising: respectively extracting a high frequency component and a low frequency from the microphone signals a component for obtaining a plurality of high frequency component signals and a plurality of low frequency component signals; calculating a delay time between the low frequency component signals; and compensating for phase mismatch between the low frequency component signals according to the delay times to obtain a plurality of Correct the low frequency component signal. 如申請專利範圍第8項所述之校正陣列麥克風之相位不匹配的方法,其中該方法更包括:偵測是否該等高頻成分信號包含語音成分,以產生一語音偵測信號;以及依據該語音偵測信號啟動該等延遲時間之計算。The method for correcting the phase mismatch of the array microphone according to claim 8 , wherein the method further comprises: detecting whether the high frequency component signals include a voice component to generate a voice detection signal; The voice detection signal initiates the calculation of the delay times. 如申請專利範圍第9項所述之校正陣列麥克風之相位不匹配的方法,其中該語音偵測信號之產生步驟包括偵測是否該等高頻成分信號之功率超過一功率界限值以決定是否致能該語音偵測信號。The method for correcting the phase mismatch of the array microphone according to claim 9 , wherein the step of generating the voice detection signal comprises detecting whether the power of the high frequency component signal exceeds a power limit value to determine whether The voice detection signal can be used. 如申請專利範圍第8項所述之校正陣列麥克風之相位不匹配的方法,其中該方法更包括:以一高通濾波器依據一界限頻率過濾該等麥克風信號以得到該等高頻成分信號;以及以一低通濾波器依據該界限頻率過濾該等麥克風信號以得到該等低頻成分信號。The method for correcting the phase mismatch of the array microphone according to claim 8 , wherein the method further comprises: filtering the microphone signals according to a limit frequency by a high-pass filter to obtain the high-frequency component signals; The microphone signals are filtered according to the threshold frequency by a low pass filter to obtain the low frequency component signals. 如申請專利範圍第11項所述之校正陣列麥克風之相位不匹配的方法,其中該界限頻率的範圍由500Hz至1000Hz。A method of correcting phase mismatch of an array microphone as described in claim 11 wherein the limit frequency ranges from 500 Hz to 1000 Hz. 如申請專利範圍第8項所述之校正陣列麥克風之相位不匹配的方法,其中該等延遲時間之計算步驟包括將該等低頻成分信號進行相關性計算(correlation)以得到該等延遲時間。A method for correcting phase mismatch of an array microphone as described in claim 8 wherein the step of calculating the delay time comprises correlating the low frequency component signals to obtain the delay times. 如申請專利範圍第8項所述之校正陣列麥克風之相位不匹配的方法,其中該等相位不匹配之計算包括:依據該等延遲時間補償該等低頻成分信號之相位不匹配以得到多個校正低頻成分信號;以及混合該等高頻成分信號及該等校正低頻成分信號以形成分別對應於該等麥克風信號之多個校正信號。The method for correcting phase mismatch of the array microphone according to claim 8 of the patent application, wherein the calculating of the phase mismatch comprises: compensating for phase mismatch of the low frequency component signals according to the delay times to obtain a plurality of corrections a low frequency component signal; and mixing the high frequency component signals and the corrected low frequency component signals to form a plurality of correction signals respectively corresponding to the microphone signals. 如申請專利範圍第14項所述之校正陣列麥克風之相位不匹配的方法,其中該等相位不匹配之計算更包括:藉由波束成型(beamforming)技術或信號分離(signal separation)技術依據該等校正信號產生無噪音及干擾成分的一目標信號。The method for correcting the phase mismatch of the array microphone according to claim 14, wherein the calculation of the phase mismatch further comprises: by beamforming technology or signal separation technology. The correction signal produces a target signal that is free of noise and interference components. 一種語音處理裝置,包括:一陣列麥克風,包括多個麥克風供產生多個麥克風信號;一相位校正模組,自該等麥克風信號分別取出一高頻成分及一低頻成分以得到多個高頻成分信號以及多個低頻成分信號,計算該等低頻成分信號間的延遲時間,並依據該等延遲時間補償該等低頻成分信號間之相位不匹配以得到多個校正低頻成分信號;以及一波束成型/信號分離模組,藉由波束成型(beamforming)技術或信號分離(signal separation)技術依據該等校正信號產生無噪音及干擾成分的一目標信號。A voice processing device includes: an array microphone comprising a plurality of microphones for generating a plurality of microphone signals; and a phase correction module for respectively extracting a high frequency component and a low frequency component from the microphone signals to obtain a plurality of high frequency components a signal and a plurality of low frequency component signals, calculating a delay time between the low frequency component signals, and compensating for phase mismatch between the low frequency component signals according to the delay times to obtain a plurality of corrected low frequency component signals; and a beamforming/ The signal separation module generates a target signal without noise and interference components according to the beamforming technology or signal separation technology according to the correction signals. 如申請專利範圍第16項所述之語音處理裝置,其中該相位校正模組包括:一次頻帶濾波器,自該等麥克風信號分別取出該高頻成分及該低頻成分,以得到該等高頻成分信號以及該等低頻成分信號;一延遲時間計算模組,計算該等低頻成分信號間的該等延遲時間;以及一延遲補償濾波器,依據該等延遲時間補償該等低頻成分信號間之相位不匹配以得到該等校正低頻成分信號;其中該等高頻成分信號及該等校正低頻成分信號經混合後形成分別對應於該等麥克風信號之該等校正信號。The voice processing device of claim 16, wherein the phase correction module comprises: a primary frequency band filter, wherein the high frequency component and the low frequency component are separately extracted from the microphone signals to obtain the high frequency components a signal and the low frequency component signals; a delay time calculation module for calculating the delay times between the low frequency component signals; and a delay compensation filter for compensating the phase between the low frequency component signals according to the delay times Matching to obtain the corrected low frequency component signals; wherein the high frequency component signals and the corrected low frequency component signals are mixed to form the correction signals respectively corresponding to the microphone signals. 如申請專利範圍第17項所述之語音處理裝置,其中該次頻帶濾波器包括一高通濾波器及一低通濾波器,該高通濾波器依據一界限頻率過濾該等麥克風信號以得到該等高頻成分信號,而該低通濾波器依據該界限頻率過濾該等麥克風信號以得到該等低頻成分信號。The speech processing device of claim 17, wherein the subband filter comprises a high pass filter and a low pass filter, the high pass filter filtering the microphone signals according to a limit frequency to obtain the contour Frequency component signals, and the low pass filter filters the microphone signals according to the threshold frequency to obtain the low frequency component signals. 如申請專利範圍第18項所述之語音處理裝置,其中該界限頻率的範圍由500Hz至1000Hz。The speech processing device of claim 18, wherein the limit frequency ranges from 500 Hz to 1000 Hz. 如申請專利範圍第17項所述之語音處理裝置,其中該相位校正模組包括一語音偵測器,用以偵測是否該等高頻成分信號包含語音成分以產生一語音偵測信號,而該語音偵測信號用以啟動該延遲時間計算模組之延遲時間之計算。The voice processing device of claim 17, wherein the phase correction module comprises a voice detector for detecting whether the high frequency component signals comprise voice components to generate a voice detection signal, and The voice detection signal is used to start the calculation of the delay time of the delay time calculation module. 如申請專利範圍第20項所述之語音處理裝置,其中該語音偵測器偵測是否該等高頻成分信號之功率超過一功率界限值以決定是否致能該語音偵測信號。The voice processing device of claim 20, wherein the voice detector detects whether the power of the high frequency component signals exceeds a power limit value to determine whether the voice detection signal is enabled. 如申請專利範圍第20項所述之語音處理裝置,其中該延遲計算模組將該等低頻成分信號進行相關性計算(correlation)以得到該等延遲時間。The speech processing device of claim 20, wherein the delay calculation module performs correlation correlation on the low frequency component signals to obtain the delay times.
TW099100780A 2009-01-13 2010-01-13 Phase calibration module, voice processing apparatus, and method for calibrating phase mismatch TWI407804B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/352,666 US8351617B2 (en) 2009-01-13 2009-01-13 Method for phase mismatch calibration for an array microphone and phase calibration module for the same

Publications (2)

Publication Number Publication Date
TW201028023A TW201028023A (en) 2010-07-16
TWI407804B true TWI407804B (en) 2013-09-01

Family

ID=42319677

Family Applications (1)

Application Number Title Priority Date Filing Date
TW099100780A TWI407804B (en) 2009-01-13 2010-01-13 Phase calibration module, voice processing apparatus, and method for calibrating phase mismatch

Country Status (3)

Country Link
US (1) US8351617B2 (en)
CN (1) CN101794575B (en)
TW (1) TWI407804B (en)

Families Citing this family (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8781142B2 (en) * 2012-02-24 2014-07-15 Sverrir Olafsson Selective acoustic enhancement of ambient sound
US20130315404A1 (en) * 2012-05-25 2013-11-28 Bruce Goldfeder Optimum broadcast audio capturing apparatus, method and system
KR102282366B1 (en) 2013-06-03 2021-07-27 삼성전자주식회사 Method and apparatus of enhancing speech
CN105702261B (en) * 2016-02-04 2019-08-27 厦门大学 Sound focusing microphone array long range sound pick up equipment with phase self-correcting function
US10264030B2 (en) 2016-02-22 2019-04-16 Sonos, Inc. Networked microphone device control
US9947316B2 (en) 2016-02-22 2018-04-17 Sonos, Inc. Voice control of a media playback system
US9772817B2 (en) 2016-02-22 2017-09-26 Sonos, Inc. Room-corrected voice detection
US10509626B2 (en) 2016-02-22 2019-12-17 Sonos, Inc Handling of loss of pairing between networked devices
US10095470B2 (en) 2016-02-22 2018-10-09 Sonos, Inc. Audio response playback
US9965247B2 (en) 2016-02-22 2018-05-08 Sonos, Inc. Voice controlled media playback system based on user profile
US9955260B2 (en) * 2016-05-25 2018-04-24 Harman International Industries, Incorporated Asymmetrical passive group delay beamforming
US9978390B2 (en) 2016-06-09 2018-05-22 Sonos, Inc. Dynamic player selection for audio signal processing
US10134399B2 (en) 2016-07-15 2018-11-20 Sonos, Inc. Contextualization of voice inputs
US10115400B2 (en) 2016-08-05 2018-10-30 Sonos, Inc. Multiple voice services
US9942678B1 (en) 2016-09-27 2018-04-10 Sonos, Inc. Audio playback settings for voice interaction
CN106412763B (en) * 2016-10-11 2019-09-06 国光电器股份有限公司 A kind of method and apparatus of audio processing
US10181323B2 (en) 2016-10-19 2019-01-15 Sonos, Inc. Arbitration-based voice recognition
US10475449B2 (en) 2017-08-07 2019-11-12 Sonos, Inc. Wake-word detection suppression
US10048930B1 (en) 2017-09-08 2018-08-14 Sonos, Inc. Dynamic computation of system response volume
US10446165B2 (en) 2017-09-27 2019-10-15 Sonos, Inc. Robust short-time fourier transform acoustic echo cancellation during audio playback
US10051366B1 (en) 2017-09-28 2018-08-14 Sonos, Inc. Three-dimensional beam forming with a microphone array
US10621981B2 (en) 2017-09-28 2020-04-14 Sonos, Inc. Tone interference cancellation
US10482868B2 (en) 2017-09-28 2019-11-19 Sonos, Inc. Multi-channel acoustic echo cancellation
US10466962B2 (en) 2017-09-29 2019-11-05 Sonos, Inc. Media playback system with voice assistance
US11343614B2 (en) 2018-01-31 2022-05-24 Sonos, Inc. Device designation of playback and network microphone device arrangements
US11175880B2 (en) 2018-05-10 2021-11-16 Sonos, Inc. Systems and methods for voice-assisted media content selection
US10959029B2 (en) 2018-05-25 2021-03-23 Sonos, Inc. Determining and adapting to changes in microphone performance of playback devices
US11076035B2 (en) 2018-08-28 2021-07-27 Sonos, Inc. Do not disturb feature for audio notifications
US10587430B1 (en) 2018-09-14 2020-03-10 Sonos, Inc. Networked devices, systems, and methods for associating playback devices based on sound codes
US11024331B2 (en) 2018-09-21 2021-06-01 Sonos, Inc. Voice detection optimization using sound metadata
US11100923B2 (en) 2018-09-28 2021-08-24 Sonos, Inc. Systems and methods for selective wake word detection using neural network models
US10692518B2 (en) 2018-09-29 2020-06-23 Sonos, Inc. Linear filtering for noise-suppressed speech detection via multiple network microphone devices
US11899519B2 (en) 2018-10-23 2024-02-13 Sonos, Inc. Multiple stage network microphone device with reduced power consumption and processing load
EP3654249A1 (en) 2018-11-15 2020-05-20 Snips Dilated convolutions and gating for efficient keyword spotting
US11183183B2 (en) 2018-12-07 2021-11-23 Sonos, Inc. Systems and methods of operating media playback systems having multiple voice assistant services
US11132989B2 (en) 2018-12-13 2021-09-28 Sonos, Inc. Networked microphone devices, systems, and methods of localized arbitration
US10602268B1 (en) 2018-12-20 2020-03-24 Sonos, Inc. Optimization of network microphone devices using noise classification
US10867604B2 (en) 2019-02-08 2020-12-15 Sonos, Inc. Devices, systems, and methods for distributed voice processing
US11120794B2 (en) 2019-05-03 2021-09-14 Sonos, Inc. Voice assistant persistence across multiple network microphone devices
US10586540B1 (en) 2019-06-12 2020-03-10 Sonos, Inc. Network microphone device with command keyword conditioning
US11200894B2 (en) 2019-06-12 2021-12-14 Sonos, Inc. Network microphone device with command keyword eventing
US10871943B1 (en) 2019-07-31 2020-12-22 Sonos, Inc. Noise classification for event detection
US11189286B2 (en) 2019-10-22 2021-11-30 Sonos, Inc. VAS toggle based on device orientation
US11200900B2 (en) 2019-12-20 2021-12-14 Sonos, Inc. Offline voice control
US11562740B2 (en) 2020-01-07 2023-01-24 Sonos, Inc. Voice verification for media playback
US11556307B2 (en) 2020-01-31 2023-01-17 Sonos, Inc. Local voice data processing
US11308958B2 (en) 2020-02-07 2022-04-19 Sonos, Inc. Localized wakeword verification
US11482224B2 (en) 2020-05-20 2022-10-25 Sonos, Inc. Command keywords with input detection windowing
US11308962B2 (en) 2020-05-20 2022-04-19 Sonos, Inc. Input detection windowing
US11984123B2 (en) 2020-11-12 2024-05-14 Sonos, Inc. Network device interaction by range

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6044162A (en) * 1996-12-20 2000-03-28 Sonic Innovations, Inc. Digital hearing aid using differential signal representations
US6272229B1 (en) * 1999-08-03 2001-08-07 Topholm & Westermann Aps Hearing aid with adaptive matching of microphones

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6741714B2 (en) * 2000-10-04 2004-05-25 Widex A/S Hearing aid with adaptive matching of input transducers
US7171357B2 (en) * 2001-03-21 2007-01-30 Avaya Technology Corp. Voice-activity detection using energy ratios and periodicity
EP1478208B1 (en) * 2003-05-13 2009-01-07 Harman Becker Automotive Systems GmbH A method and system for self-compensating for microphone non-uniformities
KR100935058B1 (en) * 2005-07-25 2009-12-31 후지쯔 가부시끼가이샤 Masturbation device
CN1905763B (en) * 2006-08-07 2011-11-23 北京中星微电子有限公司 System apparatus, device and method for correcting microphone
JP5070993B2 (en) * 2007-08-27 2012-11-14 富士通株式会社 Sound processing apparatus, phase difference correction method, and computer program
CN101911721B (en) * 2007-11-13 2014-04-23 Akg声学有限公司 Method for synthesizing a microphone signal

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6044162A (en) * 1996-12-20 2000-03-28 Sonic Innovations, Inc. Digital hearing aid using differential signal representations
US6272229B1 (en) * 1999-08-03 2001-08-07 Topholm & Westermann Aps Hearing aid with adaptive matching of microphones

Also Published As

Publication number Publication date
US20100179806A1 (en) 2010-07-15
US8351617B2 (en) 2013-01-08
CN101794575A (en) 2010-08-04
TW201028023A (en) 2010-07-16
CN101794575B (en) 2012-04-18

Similar Documents

Publication Publication Date Title
TWI407804B (en) Phase calibration module, voice processing apparatus, and method for calibrating phase mismatch
US7577262B2 (en) Microphone device and audio player
US6655212B2 (en) Sound field measuring apparatus and method
EP2031901B1 (en) Sound processing apparatus, and method and program for correcting phase difference
US8861711B2 (en) Echo cancellation apparatus, conferencing system using the same, and echo cancellation method
US8831235B2 (en) Speaker polarity determination device
US9530429B2 (en) Reverberation suppression apparatus used for auditory device
JP2010140018A5 (en)
WO2013183185A1 (en) Frequency characteristic transformation device
JPWO2007007695A1 (en) Audio system
ATE546746T1 (en) IMAGING PROCESS USING ULTRASOUND AND ERROR CORRECTION PROCESS THEREFOR
EP2707947B1 (en) Time and amplitude alignment in envelope tracking amplification stage
WO2016192277A1 (en) Bone conduction sound transmission device and method
US20150356964A1 (en) Audio signal processing circuit and electronic device using the same
WO2014103346A1 (en) Sound-source separation method, device, and program
TWI393453B (en) Tone detector and method of detecting a tone suitable for a robot
JP4892095B1 (en) Acoustic correction device and acoustic correction method
US9319788B2 (en) Sound processing device, and sound processing method
US20150131806A1 (en) Loudspeaker polarity detector
EP3110004A1 (en) Audio signal amplification apparatus
US8660272B2 (en) Parameter setting method and audio apparatus
US8792654B2 (en) Audio equipment and a signal processing method thereof
US20100208910A1 (en) Acoustic field correction method and an acoustic field correction device
US20140219441A1 (en) Voice echo cancellation method and system
JP2021106359A5 (en)

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees