Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
  • G10L25/90—Pitch determination of speech signals

Definitions

• This invention relates generally to digital coding of human speech signals for compact storage and subsequent synthesis and, more particularly, to pitch detection and the simultaneous determination of the voiced and unvoiced characterization of discrete frames of speech.
• Background of the Invention In order to reduce the bandwidth necessary to transmit human speech, it is known to digitize the human speech and then to encode the speech so as to minimize the number of digital bits per second required to store the coded digitized speech for acceptable quality of speech reproduction after the information has been transmitted and decoded for speech reproduction.
• Analog speech samples are customarily partitioned into frames or segments of discrete lengths on the order of 20 milliseconds in duration. Sampling is typically performed at a rate of 8 kilohertz (kHz) and each sample is encoded into a multibit digital number.
• LPC linear predictive coder
• Each filter parameter can be used to estimate present values of each signal sampled efficiently on the basis of the weighted sum of a preselected number of prior sample values.
• the filter parameters model the formant structure of the vocal tract transfer function.
• the speech signal is regarded analytically as being composed of an excitation signal and a formant transfer function.
• the excitation component arises in the larynx or voice box and the formant component results from the operation of the remainder of the vocal tract on the excitation component.
• the excitation component is further classified as voiced or unvoiced, depending upon whether or not there is a fundamental frequency imparted to the air stream by the vocal cords.
• the excitation component is classed as voiced. If the excitation is unvoiced, then the excitation component is simply white noise.
• LPC parameters also referred to as coefficients
• the excitation component it is necessary to determine the excitation component. First, it must be determined whether this component is to be classed as voiced or unvoiced; and if the classification is voiced, then it is necessary to determine the fundamental frequency imparted to the air stream by the vocal cords. A number of methods exist for determining the LPC coefficients.
• pitch detection is more difficult.
• One prior art method of pitch detection is based primarily on an important property of speech which is the long term regularity of the speech waveform.
• voiced speech can be viewed as a periodic signal consisting of a fundamental frequency component and its harmonics. Therefore, the output of a low-pass filter that cuts off at a frequency less than the second harmonic should be appear as a sine wave with frequency equal to the pitch. That frequency then is determined utilizing amplitude detection circuitry.
• This method suffers from the fact that actual speech deviates from this model during the transition regions of speech disturbing the regularity.
• the pitch period itself may vary depending upon whether the speaker is a male or a * female.
• the problems of pitch detection can be enhanced under some conditions by removing the formant structure of the speech which is also referred to as spectrum flattening.
• the spectrum flattening can be done utilizing Fourier transform or linear predictive analysis.
• the use of an LPC filter to flatten the spectrum is also referred to as inverse filtering to subtract the formant structure from the speech signal.
• Such a system is disclosed in U. S. Patent 3,740,476.
• the resultant residual wave that results from the LPC filtering approximates the excitation function of the vocal tract, and pulse amplitude techniques can be utilized to extract the pitch from this information.
• An illustrative pitch detector system and method utilizing a plurality of detectors each responsive to a different portion of the speech signal for estimating a pitch value, another plurality of detectors each responsive to a different portion of a residual signal calculated from the speech signal, and a voter responsive to the estimated pitch values for determining a final pitch value.
• the detectors are identical in design which allows an efficient software implementation since only one type of encoder is necessary to implement all of the encoders.
• the structural embodiment comprises a sample and quantizer circuit that is responsive to human speech to digitize and quantize the speech.
• a digital signal processor is responsive to a first set of program instructions for storing a predetermined number of the digitized samples as a speech frame, responsive to a second set of program instructions• and the digitized speech samples to generate residual samples of the digitized speech samples that remain after the formant effect of the vocal tract has been substantially removed, responsive to a third set of program instructions and individual predetermined portions of the speech samples for estimating pitch values, responsive to a fourth set of program instructions and the residual samples for estimating pitch values, and responsive to a fifth set of program instructions for determining a final pitch value of said speech frames from the estimated pitch values.
• the fifth set of program instructions comprises a first subset of program instructions for calculating a pitch value from the estimated pitch values of the second set of program instruction sets and a second subset of program instructions for constraining the final pitch value so that the calculated pitch value is in agreement with the calculated pitch values from previous frames.
• an unvoiced speech frame is indicated by the calculated pitch value being equal to a predefined value which, advantageously, may be zero; and voiced frames are indicated by the calculated pitch value not being equal to the predefined value.
• the second subset of program instructions further consists of a first group of instructions responsive to a first sequence consisting of voiced, unvoiced, and voiced frames for generating a new calculated pitch value indicating a voiced frame.
• a second group of instructions responsive to a second sequence consisting of unvoiced, voiced, and unvoiced frames for generating a new calculated value indicating an unvoiced frame.
• a third group of instructions responsive to a third sequence consisting of voiced, voiced, and voiced frames for generating a new calculated pitch value having an arithmetic relationship to the calculated pitch values of the frames of said third sequence.
• first group of instructions of the second subset is responsive to the first sequence of frames for setting the calculated pitch value equal to the arithmetic average of the calculated pitch values of the voiced frames of the first sequence
• second group of instructions is responsive to the second sequence of frames for setting the new calculated pitch value to said predefined value
• the second subset of instructions further comprises a fourth group of instructions that are responsive to a fourth sequence consisting of voiced, voiced, and unvoiced frames to calculate a new pitch value equal to the average of the calculated pitch values for the voiced and voiced frames upon the difference between the two voiced frames being less than another predefined value. If the difference between the pitch values for the two voiced frames is greater than the other predefined value, then the new calculated pitch value is set equal to the pitch value of the earlier voiced frame.
• the first subset of program instructions comprises a first group of instructions responsive to all but a subset of the estimated pitch values equaling the predefined value for setting the calculated pitch value equal to the arithmetic average of the subset of values upon the estimated pitch values of the subset of values differing by less than another predefined value from each other.
• the first group of instructions is responsive to all of the estimated pitch values being equal to the predefined value except for a subset of pitch values for setting the calculated pitch value equal to the predefined value upon the difference between each of the pitch values of the subset being greater than the other predefined value.
• the first subset of instructions comprises a second group of instructions responsive to all of the estimated pitch values except one equaling the predefined value for setting the calculated pitch value equal to the estimated pitch value not equal to the predefined value.
• the fourth set of program instructions used to estimate pitch values has a first subset of instructions for locating the sample of maximum amplitude within the predetermined portion of the residual samples within the frame.
• a second subset of instructions locates subsequent maximum samples, also termed candidate samples, in the frame of lesser amplitude than that of the maximum amplitude sample spaced by not less than a minimum distance based on the highest expected fundamental speech frequency from the maximum amplitude sample and each of the other samples within the frame.
• a third subset of instructions measures one by one the distance between adjacent located candidate samples using as a reference the maximum amplitude sample.
• a fourth subset of instructions tests for periodicity by comparing successive distance measurements for substantial equality and rejecting candidate samples that are not periodically related to the maximum amplitude sample.
• a fifth subset of instruction determines the estimated pitch value by calculating the quotient of the distance between extreme valid candidate samples within this speech frame.
• a sixth subset of instruction designates whether the frame is voiced or unvoiced. If the frame is unvoiced, the estimated pitch value is set equal to the predefined value, which advantageously may be zero, to indicate an unvoiced frame.
• the illustrative method functions in a system having a quantizer and a digitizer for converting anaiog speech into frames of digital samples and a digital signal processor that is executing a plurality of program instructions for determining the pitch of a particular frame of digital speech.
• the signal processor determines the pitch by executing the steps of producing residual samples of t ⁇ digitized speech that remain after the formant effect of the vocal tract has been substantially removed, estimating a first pitch value of the present sp ch frame from positive ones of the digitized speech samples, estimating a second pitch value from negative ones of the digitized speech samples, estimating a third value from positive ones of the residual samples, estimating a fourth pitch value from negative ones of the residual samples, and determining a final pitch value for a previous speech frame on the basis of the estimated pitch values determined by the estimating steps for a plurality of previous speech frames.
• the step of determining the final pitch value is performed by the digital signal processor responding to subsets of programmed instructions to performing the steps of calculating the final pitch value from the first, second, third, and fourth pitch values previously estimated and constraining the final pitch value so that the final pitch value is in agreement with the final pitch values from previous frames as previously determined by the digital signal processor.
• FIG. 1 illustrates, in block diagram form, a pitch detector in accordance with this invention
• FIG. 2 illustrates, in block diagram form, pitch detector 108 of FIG. 1;
• FIG. 3 illustrates, in graphic form the candidate samples of a speech frame
• FIG. 4 illustrates, in block diagram form, pitch voter 111 of FIG. 1; and
• FIG. 5 illustrates a digital signal processor implementation of FIG. 1.
• FIG. 1 shows an illustrative pitch detector which is the focus of this invention.
• the pitch detector is responsive to analog speech signals received via conductor 113 to indicate on output bus 114 whether the speech excitation is voiced or unvoiced and, if voiced, to indicate the pitch.
• the latter determinations are performed by pitch voter 111 in response to the outputs of pitch detectors 107 through 110.
• the input speech on conductor 113 is filtered by filter 100 which, advantageously, may be an eighth-order Butterworth analog low-pass filter whose -3 dB frequency is 3.3 kHz.
• the filtered speech is then digitized and quantized by sampler 112 and linear quantizer 101.
• the latter transmits the digitized speech, x(n) , to clippers 103 and 104 and to LPC coder and inverse filter 102.
• the output of coder and filter 102 is the residual signal from the inverse filtering that is transmitted to clippers 105 and 106 via path 116.
• Coder and filter 102 first performs the computations required to determine the filter coefficients that are used by the LPC inverse filter and then uses these filter coefficients to perform the inverse filtering of the digitized voice signal in order to calculate the residual signal, e(n) . This is done in the following manner.
• the digitized speech x(n) is divided into, advantageously, 20 millisecond frames during which it is assumed that the all pole LPC filter is time-invariant.
• the frame of digitized speech is used to compute a set of reflection coefficients which, advantageously, may be 10, using the lattice computation method.
• the resulting tenth order inverse lattice filter generates the forwarded prediction error or residual as well as providing the reflection coefficients.
• the clippers 103 through 106 transform the incoming x and e digitized signals on paths 115 and 116, respectively, into positive-going and negative-going wave forms. The purpose for forming these signals is that whereas the composite waveform might not clearly indicate periodicity the clipped signal might. Hence, the periodicity is easier to detect.
• Clippers 103 and 105 transform the x and e signals, respectively, into positive-going signals and clippers 104 and 106 transform the x and e signals, respectively, into negative-going signals.
• Pitch detectors 107 through 110 are each responsive to their own individual input signals to make a determination of the periodicity of the incoming signal. The output of the pitch detectors occurs two frames after receipt of those signals. Note, that each frame consists of, illustratively, 160 sample points.
• Pitch voter 111 is responsive to the output of the four pitch detectors to make a determination of the final pitch. The output of pitch voter 111 is transmitted via path 114.
• FIG. 2 illustrates in block diagram form, pitch detector 108.
• the other pitch detectors are similar in design.
• the maxima locator 201 is responsive to the digitized signals of each frame for finding the pulses on which the periodicity check is performed.
• the output of maxima locator 201 is two sets of numbers: those representing the maximum amplitudes, M., which are the candidate samples, and those representing the location within the frame of these amplitudes, D.
• Distance detector 202 is responsive to these two sets of numbers to determine a subset of candidate pulses that are periodic. This subset represents distance detector 202' s determination of what the periodicity is for this frame.
• the output of distance detector 202 is transferred to pitch tracker 203.
• the purpose of pitch tracker 203 is to constrain the pitch detector's determination of the pitch between successive frames of digitized signals. In order to perform this function, pitch tracker 203 uses the pitch as determined for the two previous frames.
• Maxima locator 201 first identifies within the samples from the frame, the global maxima amplitude, M Q f and its location, D 0 , in the frame.
• the other points selected for the periodicity check must satisfy all of the following conditions.
• the pulses must be a local maxima, which means that the next pulse picked must be the maximum amplitude in the frame excluding all pulses that have already been picked or eliminated. This condition is applied since it is assumed that pitch pulses usually have higher amplitudes than other samples in a frame.
• the amplitude of the pulse selected must be greater than or equal to a certain percentage of the global maximum.
• the pulse must be advantageously separated by at least 18 samples from all the pulses that have already been located. This condition is based on the assumption that the highest pitch encountered in human speech is approximately 440 Hz which at a sample rate of 8 kHz results in 18 samples.
• Distance detector 202 operates in a recursive-type procedure that begins by considering the distance from the frame global maximum, M-, to the closest adjacent candidate pulse. This distance is called a candidate distance, d . and is given by
• d c I D o - D i I where D ⁇ _ s the in-frame location of the closest adjacent candidate pulse. If such a subset of pulses in the frame are not separated by this distance, plus or minus a breathing space, B, then this candidate distance is discarded, and the process begins again with the next closest adjacent candidate pulse using a new candidate distance.
• B may have a value of 4 to 7. This new candidate distance is the distance to the next adjacent pulse to the global maximum pulse.
• an interpolation amplitude test is applied.
• the interpolation amplitude test performs linear interpolation between M Q an d each of the next adjacent candidate pulses, and requires that the amplitude of the candidate pulse immediately adjacent to M Q is at least q percent of these interpolated values.
• the interpolation amplitude threshold, q percent is 75%.
• Pitch tracker 203 is responsive to the output of distance detector 202 to evaluate the pitch distance estimate which relates to the frequency of the pitch since the pitch distance represents the period of the pitch.
• Pitch tracker 203' s function is to constrain the pitch distance estimates to be consistent from frame to frame by modifying, if necessary, any initial pitch distance estimates received from the pitch detector by performing four tests: voice segment start-up test, maximum breathing and pitch doubling test, limiting test, and abrupt change test. The first of these tests, the voice segment start-up test is performed to assure the pitch distance consistency at the start of a voiced region.
• this test is only concerned with the start of the voiced region, it assumes that the present frame has non-zero pitch period. The assumption is that the preceding frame and the present frame are the first and second voice frames in a voiced region. If the pitch distance estimate is designated by T(i) where i designates the present pitch distance estimate from distance detector 202, the pitch detector 203 outputs T*(i-2) since there is a delay of two frames through each detector. The test is only performed if T(i-3) and T(i-2) are zero or if T(i-3) and T(i-4) are zero while T(i-2) is non-zero, implying that frames i-2 and i-l are the first and second voiced frames, respectively, in a voiced region.
• the voice segment start-up test performs two consistency tests: one for the first voiced frame, T(i-2), and the other for the second voiced frame, T(i- 1) . These two tests are performed during successive frames.
• the purpose of the voice segment test is to reduce the probability of defining the start-up of a voiced region when such a region is not actually begun. This is important since the only other consistency tests for the voice regions are performed in the maximum breathing and pitch doubling tests and there only one consistency condition is required.
• the first consistency test is performed to assure that the distance of the right candidate sample in T(i-2) and the most left candidate sample in T(i-l) and T(i-2) are close to within a pitch threshold B + 2.
• the second consistency test is performed during the next frame to ensure exactly the same result that the first consistency test ensured but now the frame sequence has been shifted by one to the right in the sequence of frames. If the second consistency test is not met, then T(i-l) is set to zero, implying that frame i-l can not be the second voiced frame (if T(i-2) was not set to zero). However, if both of the consistency tests are passed, then frames i-2 and i-l define a start-up of a voiced region.
• T(i-l) is set to zero, while T(i-2) was determined to be non-zero and T(i-3) is zero, which indicates that frame i-2 is voiced between two unvoiced frames, the abrupt change test takes care of this situation and this particular test is described later.
• the maximum breathing and pitch doubling test assures pitch consistency over two adjacent voiced frames in a voiced region. Hence, this test is performed only if T(i-3), T(i-2), and T(i-l) are non- zero.
• the maximum breathing and pitch doubling tests also checks and corrects any pitch doubling errors made by the distance detector 202.
• the pitch doubling portion of the check checks if T(i-2) and T(i-l) are consistent or if T(i-2) is consistent with twice T(i-l), implying a pitch doubling error. This test first checks to see if the maximum breathing portion of the test is met, that is done by
• T(i-2) - T(i-l) I ⁇ A where A may advantageously have the value 10. If the above equation is met, then T(i-l) is a good estimate of the pitch distance and need not be modified. However, if the maximum breathing portion of the test fails., then the test must be performed to determine if the pitch doubling portion of the test is met. The first part of the test checks to see if T(i-2) and twice T(i-l) meet the following condition, given that T(i-3) is non-zero,
• T(i-2) - 2T(i-l) I ⁇ _ Hjl ⁇ If the above condition is met, then T(i-l) is set equal to T(i-2). If the above condition is not met, then T(i-l) is set equal to zero. The second part of this portion of the test is performed if T(i-3) is equal to zero. If the following are met
• T(i-l) T(i-2) . If the above conditions are not met, T(i-l) is set equal to zero. The limiting test which is performed on T(i-l) assures that the.pitch that has been calculated is within the range of human speech which is 50 Hz to 400 Hz. If the calculated pitch does not fall within this range, then T(i-l) is set equal to zero indicating that frame i-l cannot be voiced with the calculated pitch.
• the abrupt change test is performed after the three previous tests have been performed and is intended to determine that the other tests may have allowed a frame to be designated as voiced in the middle of an unvoiced region or unvoiced in the middle of a voiced region. Since humans usually cannot produce such sequences of speech frames, the abrupt change test assures that any voiced or unvoiced segments are at least two frames long by eliminating any sequence that is voiced-unvoiced-voiced or unvoiced-voiced-unvoiced.
• the abrupt change test consists of two separate procedures each designed to detect the two previously mentioned sequences. Once pitch tracker 203 has performed the previously described four tests, it outputs T (i-2) to the pitch voter 111 of FIG. 1. Pitch tracker 203 retains the other pitch distances for calculation on the next received pitch distance from distance detector 202.
• FIG. 4 illustrates in greater detail pitch voter 111 of FIG. 1.
• Pitch value estimator 401 is responsive to the outputs of pitch detectors 107 through 110 to make an initial estimate of what the pitch is for two frames earlier, P(i-2)
• pitch value tracker 402 is responsive to the output of pitch value estimator 401 to constrain the final pitch value for the third previous frame, P(i-3), to be consistent from frame to frame.
• P(i-2) is set equal to the arithmetic average of the two remaining estimates.
• pitch value estimator 401 sets P(i-2) equal to the remaining non-zero estimate. If only two of the estimates are non-zero, pitch value estimator 401 sets P(i-2) equal to the arithmetic average of the two pitch distance estimated values only if the two values are close to within the pitch threshold A. If the two values are not close to within the pitch threshold A, then pitch value estimator 401 sets P(i-2) equal to zero. This determination indicates that frame i-2 is unvoiced, although some individual detectors determined, incorrectly, some periodicity.
• pitch value estimator 401 sets P(i-2) equal to the non-zero value. In this case, it is left to pitch value tracker 402 to check the validity of this pitch distance estimate value so as to make it consistent with the previous pitch estimate. If all of the pitch distance estimate values are equal to zero, then, pitch value estimator 401 sets P(i-2) equal to zero.
• Pitch value tracker 402 is now considered in greater detail.
• Pitch value tracker 402 is responsive to the output of pitch value estimator ' 401 to produce a pitch value estimate for the third previous frame, P (i-3), and makes this estimate based on P(i-2) and i t
• the pitch value P (i-3) is chosen so as to be consistent from frame to frame.
• the first thing checked is a sequence of frames having the form: voiced-unvoiced-voiced, unvoiced-voiced-unvoiced, or voiced-voiced-unvoiced. If the first sequence occurs as is indicated by P(i-4) and P(i-2) being non-zero and P(i-3) is zero, then the final * pitch value, P (i-3) , is set equal to the arithmetic average of P(i-4) and P(i-2) by pitch value tracker 402.
• the final pitch value, P (i-3) is set equal to zero.
• the latter pitch tracker is responsive to P(i-4) and P(i-3) being non-zero and P(_-
• Pitch tracker 402 is responsive to
• P* ( i-3 ) P ( i-4 ) ⁇ P(i-3 ) t if pitch value tracker 402 determines that P(i-3) and P(i-4) do not meet the above condition (that is, they are not close to within, the pitch threshold A), then, pitch value tracker 402 sets P (i-3) equal to the value of P(i-4) .
• pitch value tracker 402 also performs operations designed to smooth the pitch value estimates for certain types of voiced-voiced-voiced frame sequences. Three types of frame sequences occur where these smoothing operations are performed. The first sequence is when the following is true
• pitch value tracker 402 performs a smoothing operation by setting P (i-3) _ P(i-4) + P(i-2)
• the second set of conditions occurs when
• pitch value tracker 402 sets
• pitch value tracker 402 sets
• FIG. 5 illustrates an implementation of the blocks of FIG. 1 utilizing a digital signal processor that may advantageously be a Texas Instruments' TMS320- 20 digital signal processor.
• the latter processor along with PROM memory 502 and RAM memory 503 implements blocks 102 through 111 of FIG. 1.
• the program stored in PROM 502 for implementing the aforementioned elements of FIG. 1 is similar to a C source code program.
• the program is intended for excution on a computer system with suitable digital-to-analog and analog-to-digital converter peripherals or a similar system.
• the pitch detectors 107 through 110 of FIG. 1 are implemented by common code that utilizes separate data storage areas for each pitch detector in RAM 503.
• the details given of FIG. 1 in FIGS. 2 and 4 are implemented by sets of program instructions stored within PROM 502. Each set of program instructions can be further subdivided into subsets and groups of programmed instructions.

Landscapes

• Engineering & Computer Science (AREA)
• Computational Linguistics (AREA)
• Signal Processing (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
• Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=25089225

Family Applications (1)

Country Status (7)

Families Citing this family (37)

Family Cites Families (17)

• 1985
  • 1985-08-28 US US06/770,633 patent/US4879748A/en not_active Expired - Fee Related
• 1986
  • 1986-07-25 KR KR1019870700362A patent/KR950000842B1/ko not_active IP Right Cessation
  • 1986-07-25 JP JP61504126A patent/JPH0820878B2/ja not_active Expired - Lifetime
  • 1986-07-25 EP EP86904722A patent/EP0235181B1/de not_active Expired - Lifetime
  • 1986-07-25 WO PCT/US1986/001552 patent/WO1987001498A1/en active IP Right Grant
  • 1986-07-25 DE DE8686904722T patent/DE3684907D1/de not_active Expired - Fee Related
  • 1986-07-31 CA CA000515088A patent/CA1301339C/en not_active Expired - Fee Related

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events