Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/04—Segmentation; Word boundary detection

Definitions

• the present invention relates generally to speech recognition. More particularly, the present invention relates to a system and method for segmentation of speech signals for purposes of speech recognition.
• Pattern recognition techniques have been widely used in speech recognition.
• the basic idea in the technique is to compare the input speech pattern with a set of templates, each of which represents a pre-recorded speech pattern in a vocabulary.
• the recognition result is the word in the vocabulary associated with the template which has the most similar speech pattern to that of the input speech pattern.
• it is usually not necessary to hear all the detail in an utterance (e.g., a word) in order to recognize the utterance. This fact shows that there are some natural redundancies inherent in speech.
• Many techniques have been developed to recognize speech taking advantage of such redundancies. For example, U.S. Patent No. 5,056,150 to Yu et al.
• a nonlinear time-normalization method is used to normalize a speech pattern to a predetermined length by only keeping spectra with significant time-dyn.arnic attributes. Using this method, the speech pattern is compressed significantly, although it may occasionally keep the same spectrum repeatedly.
• Another technique for speech recognition employs a sequence of acoustic segments, which represent a sequence of spectral frames. The segments are the basic speech units upon which speech recognition is based.
• One procedure for generating the acoustic segments, or performing segmentation is to search for the most probable discontinuity points in the spectral sequence using a dynamic programming method. These selected points are used as the segment boundaries. See J. Cohen, "Segmenting Speech Using Dynamic Programming," J. Acoustic Soc. of America, May 1981, vol. 69(5), pp. 1430-1437.
• This technique like the technique of U.S. Pat. No. 5,056,150 described above, is based on the searching of significant time-dynamic attributes in the speech pattern.
• segment speech is based on the segmental K- means training procedure. See L.R. Rabiner et al., "A Segmental K-means Training Procedure for Connected Word Recognition,” AT&T Technical Journal, May /June 1986 Vol. 65(3), pp. 21-31.
• an iterative training procedure an utterance is segmented into words or subword units. Each of the units is then used as a speech template in a speech recognition system.
• the iterative training procedure requires many steps of computation, so that it cannot be implemented in real time.
• the present invention is directed to a system and method for forming a segmented speech signal from an input speech signal having a plurality of frames.
• the segmented speech signal provides a template upon which speech recognition is based.
• the input speech signal is converted to a frequency domain signal having a plurality of speech frames, wherein each speech frame of the frequency domain signal is represented by at least one but usually multiple spectral values associated with the speech frame.
• the spectral values are generally chosen to encapsulate the acoustic content of the speech frame.
• a spectral difference value is then determined for each pair of adjacent frames of the frequency domain signal.
• the spectral difference value represents a difference between the spectral values for the pair of adjacent frames.
• the spectral difference value is indicative of the time-dynamic attributes between the frames.
• An initial cluster boundary is set between each pair of adjacent frames in the frequency domain signal, and a variance value is assigned to each single-frame cluster in the frequency domain signal, wherein the variance value for each single-frame cluster is equal to the corresponding spectral difference value.
• a cluster merge parameter is calculated for each pair of adjacent clusters. The cluster merge parameter is computed based on the spectral difference values of the adjacent clusters.
• a minimum cluster merge parameter is selected from the plurality of cluster merge parameters. The minimum merge parameter is indicative of the most insignificant time-dynamic attribute.
• a merged cluster is then formed by canceling a cluster boundary between the clusters associated with the minimum merge parameter and assigning a merged variance value to the merged cluster, wherein the merged variance value is representative of the variance values assigned to the clusters associated with the minimum merge parameter.
• the process is repeated in order to form a plurality of merged clusters, and the segmented speech signal is formed in accordance with the plurality of merged clusters.
• Figures 1A and IB are a flow diagram showing the operation of a method for converting a time-domain input speech signal into an output segmented speech signal.
• Figure 2 is a flow diagram showing the operation of a method for saving a plurality of speech templates, wherein each saved speech template is representative of a known speech utterance.
• Figure 3 is a flow diagram showing the operation of a method for recognizing an utterance from an input speech signal.
• Figure 4 is a graph illustrating the frequency domain signals and final cluster boundaries associated with an exemplary utterance that has been processed in accordance with the present invention.
• Figure 5 is a graph illustrating the variance values associated with each final merged cluster shown in Figure 4.
• Figure 6 is a hardware block diagram showing a system for implementing the speech signal segmentation and recognition systems shown in Figures 1-3.
• method 100 includes a "time-clustering" algorithm that non-linearly segments a speech signal in order to reduce memory requirements and facilitate speech recognition.
• a "time-clustering" algorithm that non-linearly segments a speech signal in order to reduce memory requirements and facilitate speech recognition.
• an input time-domain speech signal representative of an "utterance” is converted to a frequency-domain spectral representation using a known transformation algorithm such as discrete Fourier transform (DFT), band pass filter bank, linear predictive coding (LPC) coefficients, line spectrum pairs (LSP), or cepstral coefficients on a frame basis.
• DFT discrete Fourier transform
• LPC linear predictive coding
• LSP line spectrum pairs
• cepstral coefficients on a frame basis.
• a separate frequency-domain spectrum is generated from the input speech waveform every 10 msec using a 20 msec time window.
• the windows have a 10 msec overlap.
• the frequency-domain speech signal output by step 102 will include L speech frames and may thus be represented by expression (1) below:
• the input speech waveform that is converted to the frequency-domain in step 102 is preferably limited to only that portion of a time-domain speech signal that includes an "utterance" (an “utterance” is, for example, a single word or phrase).
• an “utterance” is, for example, a single word or phrase.
• the utterance represented by the graph in Figure 4 corresponds to the word "Catherine.”
• a spectral difference value is calculated for each pair of adjacent frames (n-2, n) in the frequency-domain signal output from step 102.
• the spectral difference value for each pair of adjacent frames (n-1, n) is representative of a difference between the respective spectral values associated with each frame in the pair of adjacent frames.
• the spectral difference value for each pair of adjacent frames (n-1, n) may be represented by expression (2) below:
• the spectral difference value for each pair of adjacent frames would correspond to the Itakura distortion between the pair of spectra
• cepstral coefficients or discrete Fourier transform were used to convert from the time-domain to the frequency-domain in step 102, the spectral difference value for each pair of adjacent frames would correspond to the Euclide.an distance between the pair of spectra.
• an initial cluster boundary (B k ) is assigned between each pair of adjacent frames in the frequency-domain signal output from step 102.
• These initial cluster boundaries are illustrated in Figure 4.
• the frequency-domain signal output from step 102 will be segmented into L clusters, each of which corresponds to one of the frames from the frequency- domain signal output in step 102.
• a counter "c" denoting the current number of clusters in the frequency-domain signal is initialized to L (i.e., "c” is initialized to the number of frames in the frequency-domain signal output from step 102).
• an initial variance value (V n ) is assigned to each cluster in the frequency-domain signal.
• the initial variance value assigned to each cluster will correspond to the spectral difference value (calculated in step 104) associated with the cluster.
• the variance value for each cluster (n) may thus be represented by expression (3) below:
• a cluster merge parameter is calculated for each pair of adjacent clusters in the frequency-domain signal.
• the cluster merge parameter corresponding to each pair of adjacent clusters is representative of the combined variance that would result if the pair of adjacent clusters were merged together.
• the cluster merge parameters " ⁇ CMP, j ⁇ 2 are calculated in accordance with equation (4) below:
• step 114 the set of CMPs calculated in step 112 are evaluated, and the cluster k having the minimum CMP associated therewith is selected in accordance with equation (5) below:
• the cluster having the minimum CMP i.e., the kth. cluster
• the preceding adjacent cluster i.e., the (k-l) ⁇ cluster
• step 120 the value of the counter "c" is decremented by 1, and in step 122, the value of the counter "c" is compared to a desired number of clusters.
• the desired number of clusters is preferably set to achieve a specified level of signal compression.
• the desired number of clusters used in step 122 would be set equal to 10 (1/6 of 58).
• the steps 112, 114, 116, 118 and 120 are repeated until the desired number of clusters is obtained. For instance, referring to Figure 4, the desired number of clusters was set to 10, and the final cluster boundaries (B' k ) for these 10 clusters are shown in Figure 4.
• the variance values are shown in Figure 5.
• the variance values may be computed by summing variance values determined in step 110 and in previous iterations of the process loop, thereby streamlining the computation requirements of the system during execution of the process loop.
• a representative spectrum (S-REP ⁇ for each final cluster is determined by calculating the mean of the spectra (S n ) within each final cluster (as defined by the final cluster boundaries (B' k )) in accordance with equation (8) below:
• N(i) represents the number of frames in cluster i.
• ' may be approximated by a member spectrum, S ⁇ , closest to ' in Euclidean distance.
• S ⁇ member spectrum
• step 210 an input time domain speech signal is processed using known methods to detect the endpoints of a speech utterance.
• step 220 the portion of the input speech signal representing the utterance (i.e., the portion of the speech signal between the endpoints detected in step 210) is converted to a frequency domain representation.
• the method used to convert the input speech signal to the frequency domain representation in step 220 is substantially the same as that used in step 102 described above.
• step 230 the frequency domain signal from step 220 is converted to a segmented speech signal.
• Step 230 is performed substantially in accordance with steps 106-124 described above.
• step 240 the segmented speech signal corresponding to the known utterance is saved in memory.
• One useful application of the present invention is to recall previously stored telephone numbers in a mobile telephone.
• the speech template of a known utterance corresponding to the name of a person can be used by a speech recognition system to recall the desired telephone number.
• the stored speech templates can then be used as part of speech recognition system in order to allow the user of the mobile phone to recall a stored telephone number associated with a particular person simply by reciting the person's name into the microphone.
• step 310 an input time domain speech signal is processed using known methods to detect the endpoints of a speech utterance included within the signal.
• step 320 the portion of the input speech signal representing the utterance (i.e., the portion of the speech signal between the endpoints detected in step 310) is converted to the frequency domain.
• the process used for converting the input speech signal to the frequency domain in step 320 is substantially the same as that used in step 102 described above.
• step 330 the frequency domain signal from step 320 is converted to a segmented speech signal.
• Step 330 is performed substantially in accordance with steps 106- 124 described above.
• the segmented speech signal is compared against speech templates previously saved in memory (as set forth in step 240).
• step 350 the speech template that is closest in Euclidean space to the segmented speech signal is selected, and a signal associated with the selected template is output.
• method 300 may be applied in the context of a mobile telephone to allow a user to automatically recall a telephone number stored in the telephone's memory.
• a user will utter the name of a person that the user wants to call into the telephone's microphone 610 (alternatively, a speech signal representative of the user may be supplied via antenna 612).
• the utterance will then be converted into a segmented speech signal using the time-clustering system and method of the present invention in the microprocessor 630.
• Steps 320 - 350 are preferably implemented in software using microprocessor 630.
• the segmented speech signal will then be compared by microprocessor 630 against speech templates stored in the telephone's memory 640 (wherein each of the stored templates corresponds to the name of a person associated with a telephone number stored in the telephone's memory 640).
• the stored template that is closest to the segmented speech signal will then be selected, and the telephone number (also stored in the phone memory 640) associated with the selected template will then be recalled from memory 640 and provided to the user on the telephone display 650.

Landscapes

• Engineering & Computer Science (AREA)
• Computational Linguistics (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Telephonic Communication Services (AREA)
• Machine Translation (AREA)
• Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)
• Mobile Radio Communication Systems (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=22846699

Family Applications (1)

Country Status (10)

Families Citing this family (23)

Citations (2)

Family Cites Families (2)

• 1999
  • 1999-01-04 US US09/225,891 patent/US6278972B1/en not_active Expired - Lifetime
  • 1999-12-29 CN CNB998153230A patent/CN1173333C/zh not_active Expired - Fee Related
  • 1999-12-29 WO PCT/US1999/031308 patent/WO2000041164A1/en active IP Right Grant
  • 1999-12-29 JP JP2000592818A patent/JP4391701B2/ja not_active Expired - Fee Related
  • 1999-12-29 EP EP99967799A patent/EP1141939B1/en not_active Expired - Lifetime
  • 1999-12-29 AT AT99967799T patent/ATE323932T1/de not_active IP Right Cessation
  • 1999-12-29 DE DE69930961T patent/DE69930961T2/de not_active Expired - Lifetime
  • 1999-12-29 AU AU24015/00A patent/AU2401500A/en not_active Abandoned
  • 1999-12-29 KR KR1020017008529A patent/KR100699622B1/ko not_active IP Right Cessation
• 2002
  • 2002-07-31 HK HK02105630.3A patent/HK1044063B/zh not_active IP Right Cessation

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events