WO2001086631A2

WO2001086631A2 - Analysis of and noise reduction in a complex waveform

Info

Publication number: WO2001086631A2
Application number: PCT/US2001/040664
Authority: WO
Inventors: Jack W. Smith; Paul Reed Smith; Gary E. Gilbert
Original assignee: Paul Reed Smith Guitars, Limited Partnership
Priority date: 2000-05-05
Filing date: 2001-05-04
Publication date: 2001-11-15
Also published as: AU2001259824A1; WO2001086631A3

Abstract

A method is provided for analysis and noise reduction in a complex waveform using the methods of Fast Find, Harmonic Adjustment and Partial Accentuation. The Fast Find method determines the harmonic content of each tone spectra in the complex waveform. The Harmonic Adjustment method amplifies harmonics in the tone spectra below a crossover frequency. The Partial Accentuation amplifies partials in the complex waveform above the crossover frequency. A Partial Consolidation method consolidates partials by frequency and time into narrow, short term frequency bands for modification by Partial/Resonant Band Accentuation. By combining these methods, noise reduction, audio clarification and other audio enhancements and characterizations are accomplished.

Description

ANALYSIS OF AND NOISE REDUCTION IN A COMPLEX WAVEFORM .

BACKGROUND AND SUMMARY OF THE INVENTION

The present invention is directed to signal analysis and noise reduction.

PRIOR ART

U.S. Patent 4,424,415 to Lin shows a format tracker which determines the fundamental frequency use the fundamental frequency and integer multiples of the fundamental to determine the optimum multiple to identify the format and band width.

U.S. Patent 5,133,013 to Munday shows a noise reduction system using spectral decomposition and nonlinear transformation which progressively attenuates lower intensity spectral components (uncorrelated noise) and passes higher level components (correlated speech) relatively unattenuated and reconstructuring the signal.

U.S. Patent 5,319,736 to Hunt shows a system for separating speech from background noise using two signal channels. Adaptive filter method tracks noise in between spoken sections for removal when speech is present.

U.S. Patent 5,479,560 to Mekata shows a format detecting device and speech processing apparatus which enhances format bands by contrast enhancement that focuses on "natural and comfortable" aspects of speech. If a region is not a formant, then it is reduced in level .

U.S. Patent 5,638,454 to Jones et al . shows noise reduction system including tunable harmonically related bandpass filters which tracks the harmonics using the change of the frequency of a synchronizing signal . 1A

U.S. Patent 5,742,927 to Crozier et al. shows noise reduction apparatus using spectral subtraction

-2 - or scaling followed by signal attenuation between for ant regions .

U.S. Patent 5,768,473 to Eatwell shows adaptive speech filter which includes adaptive spectral estimator for estimating spectral components and a noise power estimator.

WO 99/08380 to Blum et al . shows improved listening enhancement system and method which separate voice band audio from background audio. Band pass and band stop filters are tunable and controllable by user.

The present invention uses the method of the Fast Find Fundamental Method (Fast Find) described in WO 00/26896 published May 11 2000 and U.S. Patent Application No. 09/430,295 filed October 29, 1999, and the Method of Modifying Harmonic Content of a Complex Waveform (Harmonic Modification) described in U.S. Patent Application Serial No. 09/430,293 filed October 29, 1999, and WO 00/26897 published May 11, 2000, which includes Harmonic Adjustment and Partial

Accentuation to accomplish noise reduction of a complex waveform and analysis. These publications and associated patents are incorporated herein by reference . The present methods measure the partials of the input signal and determine its harmonics and harmonic tone spectra with Fast Find. Dynamic/Static thresholds are set to average energy and noise floors, using both recent and long-term historical data. The desired signal content, for example, voice/audio, is amplitude adjusted within the methods of Harmonic Adjustment and Partial Accentuation. Partial Consolidation and Resonant Band Accentuation may be - 3 - carried as well. The results are then combined. A processing control or center coordinates control information.

The methods also include using the Fast Find to differentiate harmonics from partials, placing them in their associated tone spectra, and deducing the fundamental and missing harmonic frequencies.

In a simplified description, the method utilizes Fast Find to deduce harmonics. Harmonic Adjustment amplifies the harmonics below a crossover frequency. Partials above the crossover frequency are consolidated into wide partials and resonant band energy. Partial accentuation then amplifies these partials above a set noise floor. The overall input signal is first amplitude reduced, and the desired harmonics and partials, consolidated partials, and wide partials are amplified before mixing so that the complex wave retains its original amplitude.

Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings .

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a block diagram/flow chart of an analysis and noise reduction algorithm and components according to the principles of the present invention.

Figure 2 is a graph of amplitude versus frequency illustrating the implementation of the noise reduction method according to the principles of the present invention. -4 -

Figure 3 is a graph of a complex wave signal of time versus frequency.

Figure 4 is a graph of time versus frequency illustrating the consolidation method according to the principles of the present invention.

BRIEF DESCRIPTION OF RELATED METHODS The Fast Find method uses the measured amplitudes of a limited set of partial frequencies contained in a compound wave to determine: a) which partials are not members of any tone spectrum within the compound wave; and b) which partials are harmonic members of a tone spectrum or of multiple tone spectra within the compound wave and the ranking number which each such harmonic occupies within each of the tone spectra of which it is a part .

The frequencies of harmonics as yet unmeasured are deduced so that the harmonic frequency content of each tone spectrum is fully defined by the subset of partials which were measured and used.

The Slide Rule Method of the Fast Find method allows calculations in one pass (or cycle) through measured and scaled partial frequencies to determine the harmonic content of all tone spectra contained in the complex compound wave. That method allows the calculations to generate the complete harmonic content of all tone spectra to be carried out very quickly, normally in less than ten milliseconds. Other Fast Find methods provide ways to verify the authenticity of the various tone spectra which have been deduced from the subset of measured partials. The preferable method of Fast Find is the Slide Rule method but other methods described in WO 00/26896 and related patent applications may be used.

HARMONIC MODIFICATION

Methods from Harmonic Modification according to WO 00/26897 and related patent applications may be applied to any incoming source signal waveform or material to alter its perceived quality, to enhance particular aspects or parameters of timbre, or to de- emphasize particular aspects or parameters.

Harmonic Adjustment is a method of modifying the amplitudes of harmonic frequencies 'of a detected tone spectrum in a complex waveform. This is accomplished by applying an amplitude modifying function to each harmonic frequency of the detected tone spectrum selected by harmonic rank. The frequency of each amplitude modifying function is continually set to the frequency corresponding to the harmonic rank as the frequency of the detected tone spectrum containing the selected harmonic frequency changes over time . In essence, this is the manipulation of individual harmonics and/or partials of the spectrum for a given signal. Adjustment of a harmonics or partials is over a finite period of time. This differs from the effect of generic, fixed-band equalization, which is maintained over an indefinite period of time. Processing is accomplished by manipulating the energy level of a harmonic (or group of harmonics) , or by fully removing a harmonic (or group of harmonics) or partials. The manipulations can be tied to the response of any other harmonic or it can be tied to any frequency or ranking number (s) or other parameter - 6 - the user selects. Adjustments can also be generated independently of existing harmonics.

PARTIAL ACCENTUATION Partial Accentuation is a method according to WO

00/26897 and related patent applications of modifying the amplitudes of partials in a complex waveform. The method includes determining a dynamic energy threshold as a function of frequency from the detected energy of partials. It also sets a noise floor threshold as a function of frequency. Then there is a continual determination, with a scaling functions of an amplitude modification for each partial relative to the thresholds. Finally, it applies the determined modification to the partials with amplitude modifying scaling functions. Partial Accentuation provides a method of adjusting partials based upon their amplitude in relation to the amplitude of other partials within associated frequency ranges. Also, the partial' s frequencies are the filter's frequency- adjusting guide, because partials move in frequency as well as amplitude.

Those partials that are lower in amplitude may be boosted relative to the others, and those that are strong may be cut relative to the others, with or without compressing their dynamic range. Partial Accentuation can (1) isolate or highlight relatively quiet sounds or signals; (2) diminish relatively loud or other selected sounds or signals, including among other things background noise or distortion; and (3) effect a more intelligible or otherwise more desirable blend of partials. A program examines a spectral range of a complex waveform and establishes a noise - 7 - floor and a dynamic average energy threshold. An associated program will compare over time partials to amplitude, frequency, and time thresholds and/or parameters, and decide which partial frequencies will be within the thresholds for amplitude modification. The average energy threshold is dynamic and is dependent upon the competing partials surrounding the partial slated for adjustment within some specified frequency range on either side. As the signal's amplitude shifts relative to other signals' amplitudes in the complex waveform, the flexibility of the present invention allows adjustments to be made either (1) on a continuously variable basis, or (2) on a fixed, non-continuously variable basis. The present invention's primary method (or combinations thereof) entails amplitude modifying filters that move in frequency according to what's needed to effect desired adjustments to a particular wave (or a fragment thereof) at a particular point in time. Or, secondarily, the filter processing is "handed off" in a "bucket-brigade" fashion as the partial set for amplitude adjustment moves from one filter's range into the next filter's range. Partial Accentuation can examine frequency, frequency over time, competing partials in frequency bands over time, amplitude, and amplitude over time.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be generally explained with respect to audio input including, for example, voice as by way of example . The present system can be used with any signal which would require analysis or - 8 - noise reduction.

The following is a brief description of the sections of the method shown in Figure 1.

Voice Input - Voice/audio input is converted from analog to digital by an A/D conversion.

Measurement of Partials - Measurement of partials can be accomplished in a variety of ways. FFT's, wavelets, filtering, correlation methods or any new or unknown method is acceptable. The preferred embodiment is via digital signal processing techniques .

Amplitude Adjustment - The entire signal is amplitude reduced equal to the amount of noise reduction desired. The processed voice/audio through the Processing Control is increased and subsequently added back into the original reduced amplitude signal in the Output Mixer such that the original energy level is obtained. For example, the signal may be turned down 12dB and the voice/audio is selectively turned up 12dB. The measurement signal may alternatively be reduced by the Amplitude Adjustment.

Processing Control - This is the higher-level decision portion of the process. It knows when harmonics, tone spectra, resonant band energy, silences, amplitude, frequency changes, and all other aspects or parameters of the audio are present. It controls the appropriate voice/audio processing.

The Voice/Audio input is digitized by Analog to Digital conversion, which is fed into Measurement. Measurement contains a variety of measurement techniques, as appropriate to each of the Dynamic/Static Threshold Processing, Fast Find, Consolidation Center and Amplitude Adjustment. This - 9 - may be from a variety of frequency and amplitude measurement techniques. Processing control utilizes information from the various blocks to optimally apply the appropriate processing based on the characteristics of the signal. The goal is to reduce the noise and maintain a balance or quality of the signal, for example, a voice, relative to the remainder of the input signal . Partial Accentuation and Consolidation are dynamically applied based on information obtained from Dynamic/Static Threshold Processing, Amplitude Adjustment, etc. Fast Find identifies harmonic content and deduces missing or non-measurable harmonics such that Harmonic Adjustment processing may occur. Due to simultaneous processing possibilities, the appropriate mixing between the various processing blocks must be controlled by Processing Control as well.

Dynamic/Static Threshold Processing - This section sets a noise floor whose amplitude level is frequency dependent on a historical basis on sections of silence and a dynamic average energy threshold. Silence, for the terms of this description, are portions of the audio not intended for amplification or generally the noise floor. It is determined by averaging the amplitudes across frequencies over time. Fast Find - This section searches through a limited set of measured partials and determines the harmonic frequencies of one or more tone spectra contained in the complex wave, the frequencies of each harmonic in each tone spectrum, and the harmonic ranking number of each harmonic frequency in each tone spectrum. See referenced Fast Find Fundamental patent application for details. - 10 -

Consolidation Center - The measurement of partials in a voice/audio signal can result in a large number of closely related partials in frequency, or may be closely adjacent in time. An example of measured partials is illustrated in Figure 3. Partial accentuation utilizes a method of frequency-time consolidation to effect the following: (1) to reduce the amount of filtering required, which lessens processing requirements; and (2) to reduce the number of filter transition bands such that sound quality is improved. As partials are measured, surrounding frequencies are compared for partial content. If there exists a partial within a defined range, the filtering requirements are expanded to include that region. The bands are dynamic, but have a maximum adjacency test such that the bands do not expand forever. Likewise, if a particular partial disappears, the filtering is narrowed in frequency as to not allow unwanted signal to be filtered. Partial detection or measurement must meet a minimum time threshold to be qualified as a partial, and a partial must also disappear from some minimum time before filtering is stopped. This is to facilitate a brief interruption of the measurement of a partial, yet continue if it is successfully measured within a threshold time period.

An example of Partial Accentuation consolidation of the measured partial of Figure 3 may be seen in Figure 4. The lower frequencies are consolidated in narrower bands and extend over longer time periods. The higher frequencies consolidate into broader bands and extend over short time periods. Some of the lower frequencies may consolidate into short time periods - 11 - and some higher frequencies may consolidate into longer time periods. Although lower frequencies are shown as consolidated in the example, the consolidated signal is only used in Partial Accentuation and Resonant Band Accentuation for frequencies above these crossover frequencies. Thus, those frequencies below the crossover frequency need not be processed for consolidation.

Harmonic Adjustment - The filters in this part modify the amplitudes of harmonics in the tone spectra. Harmonics may move in frequency over time. Harmonic Adjustment amplitude modifying filters move in frequency aligned with the harmonic's frequency. Either dynamically calculating the frequency response of the filters, or adjoining filters over time in a bucket-brigade fashion may accomplish this. Fast Find and the Processing Control guide these filters in frequency. The algorithm chooses which partials and harmonics are to be modified. Harmonics and partials may either be increased or decreased in amplitude. For many types of noise reduction, Harmonic Adjustment will primarily be employed below a certain maximum or crossover frequency, which is shown as an example in Figure 2. Partial Accentuation - Partial Accentuation amplifies partials above the noise floor using scaling functions. This portion is used for partials and resonant band processing. For many forms of noise reduction, Partial Accentuation is typically applied above some minimum or crossover frequency. This is shown as an example in Figure 2.

Resonant Band Accentuation - When resonant bands are produced by the voice, frequency bands of - 12 - increased energy of varying widths are created. These wide partials are the total accumulated energy in a resonant band over a specific time period above some amplitude. This width can vary greatly. This frequency information is provided to the Processing Control by the Consolidation Center. Partial Accentuation will amplify these consolidated partials (resonant bands) that are above the noise floor.

Output Mixer - The output mixer boosts and mixes the processed components with the original amplitude reduced signal for noise reduction, as determined by the Process Control . The Output Mixer effectively and smoothly combines the results of each processing block to culminate in a useful mix of the applied methods. Figure 2 shows the effect of combining these methods together as it applies to implementation of various aspects or parameters of a voice/audio signal .

The present method measures incoming voice/audio for its partial content. Partials are further qualified using Fast Find as harmonics, which indicate a tone spectra. Portions of the tone spectra include both measured partials that are qualified as harmonics and deduced harmonics that cannot be measured at that moment in time. This deduction includes the fundamental frequency.

Harmonics are tracked to maintain the identification of distinct tone spectra. The measured and deduced harmonic locations guide Harmonic Adjustment of the harmonics. Harmonics are adjusted below a certain maximum or crossover frequency to affect noise reduction as shown in Figure 2. The Fast Find allows identification or deduction of the qualified harmonic frequencies below crossover - 13 - frequency before they occur, not measurable or missing, and aids in tracking.

Partials that are above a certain minimum crossover or frequency, which may include harmonics that fall into that frequency range, are adjusted using scaling function as shown in Figure 2. The scaling function is part of the process of Partial Accentuation. Partial Accentuation utilizes Partial Consolidation shown in Figure 4 to aid in the Partial Accentuation/Resonant Band Accentuation process.

Partial Accentuation is implemented dynamically based on the Static/Dynamic Threshold Processing.

The crossover frequency is a frequency of the desired portion of the input signal at which there is a naturally occurring break between the high and low frequencies of that desired signal. Using a human voice, for example, the crossover frequency may be in the 600-750 hertz range. For the human voice, others may consider the crossover frequency to be in the 600- 1500 hertz range. Each type of desired signal, relative to the background noise, generally will have a crossover frequency. Knowing the type of signal, this crossover frequency can be fixed. If the source of the signal is not known, appropriate programming can be provided in the Processing Control to identify the kind or source of the signal and select an appropriate crossover frequency or continuously select the crossover frequency.

All components are controlled in a coordinated fashion, and further mixed for the desired output. Measurement, Processing Control, and Output Mixer are performed by means of computer instructions by a computing device. This device may be implemented by - 14 -

Digital Signal Processing devices, general computing devices, Application Specific Integrated Circuits

(ASIC) , Hybrid Integrated Circuits, or any combination thereof . Although the present invention has been described and illustrated in detail, it is to be clearly understood that the same is by way of illustration and example only, and is not to be taken by way of limitation. The spirit and scope of the present invention are to be limited only by the terms of the appended claims.

Claims

-15- WHAT IS CLAIMED:

1. A method of noise reduction in a complex wave signal comprising: identifying sets of harmonic frequencies having a common fundamental frequency; adjusting harmonic frequencies below a crossover frequency; accentuating partials above the crossover frequency; and combining the adjusted harmonic frequencies and the accentuated partials to produced a combined signal .

2. The method according to Claiml, wherein identifying includes identifying measured harmonic frequencies and deducing other harmonic frequencies of the set .

3. The method according to Claim 1, wherein identifying is performed using Fast Find Fundamental.

4. The method according to Claiml, wherein accentuating partials includes accentuating harmonic frequencies above the crossover frequency.

5. The method according to Claim 1, including initially reducing the amplitude level of the complex wave signal by a predetermined amount and subsequently increasing the amplitude level of the combined signal by the predetermined amount and adding it to the reduced signal . - 16 -

6. The method according to Claim 1, including consolidating at least partials above the crossover frequency before accentuating.

7. The method according to Claim 6, wherein the partials are consolidated by time and frequency.

8. The method according to Claim 6, wherein the partials are consolidated in resonant bands.

9. The method according to Claim 8, wherein the resonant bands having variable widths.

10. The method according to Claim 1, including selecting the crossover frequence.

11. The method according to Claim 10, wherein the crossover frequency is selected as a function of the source of the complex wave signal .

12. The method according to Claim 10,wherein the crossover frequency is selected by analyzing the complex wave signal .

13. The method according to Claim 10, wherein the crossover frequency is continually selected based on parameters of the complex waveform.