US20050010410A1 - Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium - Google Patents

Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium Download PDF

Info

Publication number
US20050010410A1
US20050010410A1 US10/849,724 US84972404A US2005010410A1 US 20050010410 A1 US20050010410 A1 US 20050010410A1 US 84972404 A US84972404 A US 84972404A US 2005010410 A1 US2005010410 A1 US 2005010410A1
Authority
US
United States
Prior art keywords
speech
model data
echo
speech recognition
acoustic model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/849,724
Other languages
English (en)
Inventor
Tetsuya Takiguchi
Masafumi Nishimura
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NISHIMURA, MASAFUMI, TAKIGUCHI, TETSUYA
Publication of US20050010410A1 publication Critical patent/US20050010410A1/en
Assigned to NUANCE COMMUNICATIONS, INC. reassignment NUANCE COMMUNICATIONS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Priority to US12/476,650 priority Critical patent/US8024184B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/20—Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L2021/02082—Noise filtering the noise being echo, reverberation of the speech

Definitions

  • the present invention relates to speech recognition by a computer device, and in particular to a speech recognition device for sufficiently recognizing an original speech even when the original speech is superimposed with an echo generated by the environment, a speech recognition method, a computer-executable program for causing a computer to execute the control method, and a storage medium.
  • the above-mentioned speech recognition device for recognizing speech as input can be assumed to be utilized for various applications such as dictation of a document, transcription of minutes of a meeting, interaction with a robot, and control of an external machine.
  • the above-mentioned speech recognition device essentially analyzes inputted speech to acquire a feature quantity, selects a word corresponding to the speech based on the acquired feature quantity, and thereby causes a computer device to recognize the speech.
  • Various methods have been proposed to exclude influence from the environment, such as background noises, in performing speech recognition.
  • a typical example is a method in which a user is required to use a hand microphone or a head-set type microphone in order to exclude echoes or noises which may be superimposed with the speech to be recorded and to acquire only the inputted speech.
  • a user is required to use such extra hardware as are not usually used.
  • FIG. 9 shows a typical method in which noises are taken into consideration when performing speech recognition.
  • an inputted signal has a speech signal and output probability distribution in which the speech signal is superimposed with a noise signal. Since, in many cases, a noise occurs suddenly, a method is employed in which a microphone for acquiring an input signal and a microphone for acquiring a noise are used and, with the use of a so-called two-channel signal, a speech signal and a noise signal are separately acquired from the input signal.
  • a traditional speech signal shown in FIG. 9 is acquired from a first channel, and a noise signal is acquired from a second channel, so that, with a use of a two-channel signal, an original speech signal can be recognized from an inputted speech signal even under a noisy environment.
  • the cepstrum mean subtraction (CMS) method has been employed as a method for coping with influence from a speech transfer route.
  • CMS cepstrum mean subtraction
  • a disadvantage has been known that the CMS method is effective when the impulse response of a transfer characteristic is relatively short (several milliseconds to several dozen milliseconds), such as the case of influence of a telephone line, but is not sufficiently effective in performance when the impulse response of a transfer characteristic is longer (several hundred milliseconds), such as the case of an echo in a room.
  • the reason for the disadvantage is that the length of the transfer characteristic of an echo in a room is generally longer than the window width (10 msec-40 msec) for a short-distance analysis used for speech recognition, and therefore the impulse response is not stable in the analysis interval.
  • a method of predicting an intra-frame transfer characteristic H to feed it back for speech recognition has been reported by T. Takiguchi, et al. (“HMM-Separation-Based Speech Recognition for a Distant Moving Speaker”, IEEE Trans. on SAP, Vol. 9, pp. 127-140, No. 2, 2001), for example.
  • a transfer characteristic H in a frame is used to reflect the influence of an echo; a speech input is inputted via a head-set type microphone as a reference signal; an echo signal is separately measured; and then, based on the result of the two-channel measurement, an echo prediction coefficient a for predicting an echo is acquired.
  • the above-mentioned environments include a case where processing is performed based on speech recognition when driving a vehicle or piloting a plane, or during movement within a large space, and a case where speech is inputted into a notebook-type personal computer or a microphone located at a distance for a kiosk device.
  • the term “hands-free” is used to mean a condition in which a speaker can speak at any position without restriction by the position of a microphone.
  • the present invention has been made in consideration of the above-mentioned disadvantages of the conventional speech recognition.
  • a method for coping with influence of an echo in a room by adapting an acoustic model used in speech recognition (hidden Markov model) to a speech signal in an echo environment.
  • the influence of echo components in a short-interval analysis is estimated using a signal observed for input from one microphone (one channel). This method does not require an impulse response to be measured in advance, and enables echo components to be estimated based on the maximum likelihood estimation utilizing an acoustic model by using only a speech signal spoken at any place.
  • the present invention has been made based on the idea that it is possible to perform sufficient speech recognition not by actually measuring a speech signal superimposed with an echo or a noise (hereinafter referred to as a “speech model affected by intra-frame echo influence” in the present invention) with the use of a head-set type microphone or a hand microphone, but by expressing it with an acoustic model used for speech recognition to estimate an echo prediction coefficient based on the maximum likelihood reference.
  • an echo can be defined as an acoustic signal which influences a speech signal for a longer time than an impulse response, the signal which gives the echo being a speaking voice giving the speech signal.
  • an echo can be defined as an acoustic signal which influences a speech signal for a longer time than an impulse response, the signal which gives the echo being a speaking voice giving the speech signal.
  • acoustic model data (an HMM parameter and the like), which is usually used as an acoustic model, can be regarded as a reference signal with a high accuracy, related to a phoneme generated with a speech corpus and the like.
  • a transfer function H in a frame can be predicted with sufficient accuracy based on an existing technique.
  • a “speech model affected by intra-frame echo influence” equivalent to a signal which has been conventionally inputted separately as a reference signal is-generated from an acoustic model with the use of additivity of a cepstrum.
  • an echo prediction coefficient ⁇ can be estimated so that a selected speech signal is given the maximum probability.
  • the echo prediction coefficient is used to generate an adapted acoustic model which has been adapted to an environment to be used by a user, in order to perform speech prediction.
  • speech input as a reference signal is not required, and it is possible to perform speech recognition using only a speech signal from one channel.
  • a speech recognition device configured to include a computer, for recognizing a speech; the speech recognition device comprising: a storage area for storing a feature quantity acquired from a speech signal for each frame; storing portions for storing acoustic model data and language model data, respectively; an echo adaptation model generating portion for generating echo speech model data from a speech signal acquired prior to a speech signal to be processed at the current time point and using the echo speech model data to generate adapted acoustic model data; and recognition processing means for referring to the feature quantity, the adapted acoustic model data and the language model data to provide a speech recognition result of the speech signal.
  • the adapted acoustic model generating means in the present invention can comprise: a model data area transforming portion for transforming cepstrum acoustic model data into linear spectrum acoustic model data; and an echo prediction coefficient calculating portion for adding the echo speech model data to the linear spectrum acoustic model data to generate an echo prediction coefficient giving the maximum likelihood.
  • the present invention comprises an adding portion for generating echo speech model data, and the adding portion can add the cepstrum acoustic model data of the acoustic model and cepstrum acoustic model data of an intra-frame transfer characteristic to generate a “speech model affected by intra-frame echo influence”.
  • the adding portion in the present invention inputs the generated “speech model affected by intra-frame echo influence” into the model data area transforming portion and causes the model data area transforming portion to generate linear spectrum acoustic model data of the “speech model affected by intra-frame echo influence”.
  • the echo prediction coefficient calculating portion in the present invention can use at least one phoneme acquired from an inputted speech signal and the echo speech model data to maximize likelihood of the echo prediction coefficient based on linear spectrum speech model data.
  • the speech recognition device in the present invention preferably performs speech recognition using a hidden Markov model.
  • a speech recognition method for causing a speech recognition device configured to include a computer, for recognizing a speech, to perform speech recognition; the method causing the speech recognition device to execute steps of: storing in a storage area a feature quantity acquired from a speech signal for each frame; reading from the storing portion a speech signal acquired prior to a speech signal to be processed at the current time point to generate echo speech model data and processing speech model data stored in a storing portion to generate adapted acoustic speech model data and store it in a storage area; and reading the feature quantity, the adapted acoustic model data and language model data stored in a storing portion to generate a speech recognition result of the speech signal.
  • the step of generating the adapted acoustic model data can comprise: an adding portion calculating the sum of the read speech signal and an intra-frame transfer characteristic value; and causing a model data area transforming portion to read the sum calculated by the adding portion to transform cepstrum acoustic model data into linear spectrum acoustic model data.
  • the present invention can comprise a step of causing an adding portion to read and add the linear spectrum acoustic model data and the echo speech model data to generate an echo prediction coefficient giving the maximum likelihood.
  • the step of transformation into the linear spectrum acoustic model data can comprise a step of causing the adding portion to add the cepstrum acoustic model data of the acoustic model data and cepstrum acoustic model data of an intra-frame transfer characteristic to generate a “speech model affected by intra-frame echo influence”.
  • the step of generating the echo prediction coefficient in the present invention can comprise a step of determining the echo prediction coefficient so that the maximum likelihood is given to at least one phoneme for which the sum value of the linear spectrum echo model data of the “speech model affected by intra-frame echo influence” and the echo speech model data, which has been generated by the adding portion and stored.
  • a computer-readable program for causing a computer to execute the above-mentioned speech recognition methods and a computer-readable storage medium storing the computer-readable program.
  • FIG. 1 schematically illustrates speech recognition using a hidden Markov model (HMM);
  • FIG. 2 schematically illustrates a process for forming an output probability table based on each state for a speech signal
  • FIG. 3 is a flowchart showing a schematic procedure for a speech recognition method of the present invention.
  • FIG. 4 shows schematic processing in the process described in FIG. 3 ;
  • FIG. 5 is a schematic block diagram of a speech recognition device of the present invention.
  • FIG. 6 shows a detailed configuration of an adapted acoustic model data generating portion used in the present invention
  • FIG. 7 is a schematic flowchart showing a process of a speech recognition method to be performed by a speech recognition device of the present invention.
  • FIG. 8 shows an embodiment in which a speech recognition device of the present invention is configured as a notebook-type personal computer
  • FIG. 9 shows a typical method in which noises are taken into consideration for speech recognition.
  • FIG. 1 schematically illustrates speech recognition using a hidden Markov model (HMM) to be used in the present invention.
  • An acoustic model can be regarded as an automaton in which a word or a sentence is constructed as a sequence of phonemes; three states are typically provided for each phoneme; and a transition probability among these states is specified so that a word or a sentence composed of a sequence of phonemes can be retrieved.
  • FIG. 1 there are illustrated three phonemes S 1 to S 3 .
  • S 0 ) from the state S 1 to S 2 is shown as 0.5, and the transition probability Pr(S 3
  • An output probability to be determined in association with a phoneme given by mixed Gaussian distribution is assigned to each of the states S 1 to S 3 .
  • mixed elements k 1 to k 3 are used for the states S 1 to S 3 .
  • FIG. 1 also shows an output probability distribution of mixed Gaussian distribution for the state S 1 , shown as k 1 to k 3 .
  • the mixed elements are provided with weights w 1 to w 3 , respectively, to be suitably adapted to a particular speaker.
  • the output probability is defined to be given by Pr(O
  • FIG. 2 shows a process for generating an output probability table according to the present invention.
  • the output probability from the state S 1 to the state S 3 can be calculated by composing a trellis as shown in FIG. 2 using a feature quantity series ⁇ ⁇ ⁇ acquired from a speech signal and using an algorithm such as a Viterbi algorithm, a forward algorithm, a beam-search algorithm and the like. More generally, an output probability for a speech signal based on each state can be given as an output probability table, where t is a predetermined frame, Ot is a speech signal at the predetermined frame t, s is a state, ⁇ is a set of HMM parameters.
  • FIG. 3 shows a flowchart showing a schematic procedure of a speech recognition method of the present invention.
  • the process of the speech recognition method of the present invention receives input of a speech signal at step S 10 , and, at S 12 , generates from acoustic model data and an intra-frame transfer characteristic a “speech model affected by intra-frame echo influence”
  • an echo prediction coefficient ⁇ and a speech signal in the past are used to generate echo speech model data ( ⁇ O ⁇ w; tp ⁇ ).
  • the generated echo speech model data is, at step S 16 , added to the “speech model affected by intra-frame echo influence” given at step S 12 as linear spectrum acoustic model data, and then an echo prediction coefficient ⁇ is so determined that the maximum likelihood value can be obtained for a selected word or sentence obtained by processing the speech signal.
  • the determined echo prediction coefficient ⁇ and the speech signal O( ⁇ ; tp) in a frame in the past are used to acquire the absolute value of an echo.
  • a speech model which also includes outer-frame echo components is generated and stored as a set with other parameters.
  • FIG. 4 shows a schematic process for the processing described with reference to FIG. 3 of the present invention.
  • acoustic model data and a cepstrum of an intra-frame transfer characteristic are added to create data of a “speech model affected by intro-frame echo influence”.
  • the generated speech model data is transformed into linear spectrum acoustic model data.
  • an echo prediction coefficient ⁇ is determined so that likelihood is maximized for the feature quantity of an phoneme included in the speech signal selected in the transformed spectrum data.
  • Various methods can be used for the setting, and a predetermined word or a predetermined sentence, for example, may be appropriately used for the determination.
  • the determined echo prediction coefficient ⁇ together with acoustic model data originally stored in the speech recognition device, is used to create adapted acoustic model data.
  • the acoustic model data within the generated linear spectrum area is logarithmically transformed and inverse Fourier transformed to be a cepstrum, and the cepstrum is stored to perform speech recognition.
  • a standard acoustic model generated with a speech corpus and the like can be used in the present invention, and this is referred to as a clean speech signal in the present invention.
  • a prediction value for transfer characteristic in the same frame is used for H.
  • the ⁇ is an echo prediction coefficient showing the rate of an echo to be imposed from a frame in the past to the frame to be evaluated at the current time point.
  • the subscript “cep” indicates a cepstrum.
  • acoustic model data used for speech recognition in the present invention is used instead of a reference signal. Furthermore, the intra-frame transfer characteristic H is acquired as a prediction value, and an echo prediction coefficient is determined using a speech signal selected based on the maximum likelihood reference to generate adapted acoustic model data.
  • an intra-frame transfer characteristic H to acoustic model data can be performed by convolution in a spectrum area, transformation into a cepstrum area enables an addition condition to be satisfied. Therefore, if the intra-frame transfer characteristic H can be estimated by another method, it is possible to easily use additivity with acoustic model data to easily and accurately determine acoustic model data, which takes the intra-frame transfer characteristic H into consideration, through addition to data in the cepstrum area of acoustic model data already registered.
  • a set of parameters for an HMM of a clean speech signal S is indicated by ⁇ (s), cep , a set of HMM parameters for the intra-frame transfer characteristic H is indicated by ⁇ (h′), cep , and a set of HMM parameters for adapted acoustic model data is indicated by ⁇ (O), cep .
  • ⁇ (s) ⁇ j,k , O 2 (s)j,k , W j,k ⁇
  • ⁇ j,k the mean value of the k-th output probability of a state j of a predetermined HMM
  • O 2 (s)j,k distribution
  • W j,k weight.
  • the intra-fame transfer function created can be subject to Discrete Fourier Transformation and indexation processing, then transformed to a cepstrum area, and stored in a storage area.
  • an EM algorithm (An inequality and associated maximization technique in statistical estimation of probabilistic function of a Markov process”, Inequalities, Vol. 3, pp. 1-8, 1972) can be used to calculate a prediction value for the maximum likelihood ⁇ ′.
  • Calculation processing of an echo prediction coefficient ⁇ using the EM algorithm is performed by using the E step and the M step of the EM algorithm.
  • a set of HMM parameters transformed into a linear spectrum area is used to calculate at the E step the Q function shown by the formula (3) below.
  • the index of an HMM parameter (indicating a predetermined phoneme, for example) is indicated by p
  • the n-th observation series is indicated by O p,n related to a phoneme p
  • a state series and a mixed element series for each O p,n are indicated by s p,n and m p,n .
  • the mean value, distribution and weight of the k-th output probability distribution (mixed Gaussian distribution) of a state j of a phoneme p of ⁇ (SH), lin are shown as the expression (4) below.
  • the frame number is indicated by t.
  • the ⁇ p.n.j.k.t is a probability given by the formula (6) below.
  • the Q function is then maximized relative to ⁇ ′ at the M step (maximization) in the EM algorithm.
  • ⁇ ⁇ ⁇ ′ ⁇ p ⁇ ⁇ n ⁇ ⁇ j ⁇ ⁇ k ⁇ ⁇ t ⁇ ⁇ p , n , j , k , t ⁇ O p , n ⁇ ( t ) ⁇ O p , n ⁇ ( t - 1 ) - O p , n ⁇ ( t - 1 ) ⁇ ⁇ ( SH ) , p , j , k ⁇ ( SH ) , p , j , k 2 ⁇ p ⁇ ⁇ n ⁇ ⁇ j ⁇ ⁇ k ⁇ ⁇ t ⁇ p , n , j , k , t ⁇
  • the ⁇ ′ can be estimated for each phoneme p.
  • the ⁇ ′ for each phoneme can be acquired by using a value before calculating the sum for the phoneme p.
  • Which echo prediction coefficient is to be used can be determined according to a particular device and a request such as recognition efficiency and recognition speed. It is also possible to determine ⁇ ′ for each HMM state similar to the formulae (8) and (9). By performing the calculation processing described above, an echo prediction coefficient ⁇ can be acquired only from a speech signal O(t) inputted from one channel away from a speaker using only parameters of the original acoustic model.
  • FIG. 5 shows a schematic block diagram of a speech recognition device of the present invention.
  • the speech recognition device 10 of the present invention is generally configured with a computer including a central processing unit (CPU).
  • the speech recognition device 10 of the present invention comprises a speech signal acquiring portion 12 , a feature quantity extracting portion 14 , a recognition processing portion 16 and an adapted acoustic model data generating portion 18 .
  • the speech signal acquiring portion 12 transforms a speech signal inputted from inputting means such as a microphone (not shown) into a digital signal with an A/D transformer and the like, and stores it in a suitable storage area 20 with its amplitude associated with a time frame.
  • the feature quantity extracting portion 14 is configured to include a model data area transforming portion 22 .
  • the model data area transforming portion 22 comprises Fourier transformation means (not shown), indexation means and inverse Fourier transformation means.
  • the model data area transforming portion 22 reads a speech signal stored in the storage area 20 to generate a cepstrum of the speech signal, and stores it in a suitable area of the storage area 20 .
  • the feature quantity extracting portion 14 acquires a feature quantity series from the generated cepstrum of the speech signal and stores it in association with a frame.
  • the speech recognition device 10 shown in FIG. 5 is configured to further include an acoustic model data storing portion 24 for storing acoustic model data based on an HMM, which has been generated with the use of a speech corpus and the like, a language model data storing portion 26 for storing language model data acquired from a text corpus and the like, and an adapted acoustic model data generating portion 18 for storing adapted acoustic model data generated by the present invention.
  • the recognition processing portion 16 is configured to read adapted acoustic model data from an adapted acoustic model data storing portion 28 , read language model data from the language model data storing portion 26 , and use likelihood maximization to perform speech recognition for each read data based on the cepstrum of the speech signal.
  • Each of the acoustic model data storing portion 24 , the language model data storing portion 26 and the adapted acoustic model data storing portion 28 may be a database constructed in a storage device such as a hard disk.
  • the adapted acoustic model data generating portion 18 shown in FIG. 5 creates adapted acoustic model data through the above-mentioned processing in the present invention, and causes it to be stored in the adapted acoustic model data storing portion 28 .
  • FIG. 6 shows a detailed configuration of an adapted acoustic model data generating portion 18 to be used in the present invention.
  • the adapted acoustic model data generating portion 18 to be used in the present invention is configured to include a buffer memory 30 , model data area transforming portions 32 a and 32 b , an echo prediction coefficient calculating portion 34 , adding portions 36 a and 36 b , and a generating portion 38 .
  • the adapted acoustic model data generating portion 18 reads predetermined observation data older than the frame to be processed at the current time point, and multiplies it by an echo prediction coefficient ⁇ , and stores it in the buffer memory 30 .
  • the adapted acoustic model data generating portion 18 reads acoustic model data from the acoustic model data storing portion 24 , and reads the cepstrum acoustic model data of the intra-frame transfer characteristic H which has been calculated in advance from the storage area 20 and writes it to the buffer memory 30 .
  • both of the acoustic model data stored in the buffer memory 30 and the intra-frame transfer characteristic data are cepstrum acoustic model data, these data are read into the adding portion 36 a and addition is performed to generate a “speech model affected by intra-frame echo influence”.
  • the “speech model affected by intra-frame echo influence” is sent to the model data area transforming portion 32 a to be transformed into linear spectrum acoustic model data, and then it is sent to the adding portion 36 b .
  • the adding portion 36 b reads data obtained by multiplying observation data in the past by an echo prediction coefficient and performs addition to the linear spectrum acoustic model data of the “speech model affected by intra-frame echo influence”.
  • the addition data generated at the adding portion 36 b is sent to the echo prediction coefficient calculating portion 34 storing acoustic model data corresponding to a phoneme and the like selected in advance to determine an echo prediction coefficient ⁇ so that the likelihood is maximal, using an EM algorithm.
  • the determined echo prediction coefficient ⁇ is passed to the generating portion 38 together with acoustic model data stored after being transformed into linear spectrum acoustic model data or still remaining linear spectrum, and created as adapted acoustic model data.
  • the generated adapted acoustic model data is sent to the model data area transforming portion 32 b , and is transformed from linear spectrum acoustic model data into cepstrum acoustic model data. After that, it is stored in the adapted acoustic model data storing portion 28 .
  • FIG. 7 is a schematic flowchart showing a process of a speech recognition method to be performed by a speech recognition device of the present invention.
  • the recognition process to be performed by the speech recognition device of the present invention acquires a speech signal superposed with an echo for each frame and stores in a suitable storage area at least the frame to be processed at the current time point and a preceding frame.
  • the process extracts a feature quantity from the speech signal, acquires data to be used for retrieval of the speech signal based on acoustic model data and language model data, and stores the data as cepstrum acoustic model data in a suitable storage area.
  • step S 34 which can be performed in parallel with step S 32 , a speech signal in a frame in the past and acoustic model data are read from a suitable storage area, transformation into a cepstrum area and transformation into a linear spectrum area are done to create adapted acoustic model data, and the data are stored in a suitable storage area in advance.
  • the adapted acoustic model data and the feature quantity acquired from the speech signal are used to determine a phoneme to which the maximum likelihood is to be given.
  • language model data are used based on the determined phoneme to generate a recognition result, and the result is stored in a suitable storage area. At the same time, the sum of likelihoods at the current time point are stored.
  • step S 40 it is determined whether there remains a frame to be processed. If there is no frame to be processed (no), then a word or a sentence for which the sum of likelihoods is maximal is outputted as a recognition result at step S 42 . If there is any frame yet to be processed, a “yes” determination at step S 40 , then at step S 44 , observation data for the remaining frame is read, and a feature quantity is extracted. The process is then returned to step S 36 , and recognition of the word or sentence is completed by repetition of the process.
  • FIG. 8 shows an embodiment in which a speech recognition device of the present invention is configured as a notebook-type personal computer 40 .
  • An internal microphone 42 is arranged at the upper side of the display part of the notebook-type personal computer 40 to receive speech input from a user.
  • the user moves a cursor displayed on the display part with pointer means 44 such as a mouse and a touch pad installed in office or at home to perform various processings.
  • a user desires to perform dictation with word-processor software, for which software by IBM Corporation (ViaVoice: trademark registered), for example, is used, for speech recognition.
  • word-processor software for which software by IBM Corporation (ViaVoice: trademark registered), for example, is used, for speech recognition.
  • IBM Corporation ViaVoice: trademark registered
  • the word-processor software is activated at the same time that the ViaVoice is activated.
  • a speech recognition program of the present invention is incorporated in the ViaVoice software as a module.
  • a user uses a head-set type microphone or a hand microphone to avoid the influence of echoes and environmental noises when inputting a speech. Furthermore, the user is required to input a speech by separately inputting environmental noises or echoes, and an input speech.
  • the speech recognition method using the notebook-type personal computer 40 shown in FIG. 8 of the present invention the user can perform dictation through speech recognition only by input into the internal microphone 42 in accordance with the present invention.
  • FIG. 8 shows an embodiment in which the present invention is applied to a notebook-type personal computer
  • the present invention is applicable to speech-interaction type processing in a relatively small space where influence of echoes is larger than that of continuous superposition of environmental noises, such as a kiosk device for performing speech-interaction type processing in a relatively small partitioned room, dictation in a car or a plane, and command recognition and the like, in addition the processing shown in FIG. 8 .
  • the speech recognition device of the present invention is capable of communicating with another server computer performing non-speech processing or a server computer suitable for speech processing via a network.
  • the network described above includes the Internet using a communication infrastructure such as a local area network (LAN), a wide area network (WAN), optical communication, ISDN, and ADSL.
  • LAN local area network
  • WAN wide area network
  • optical communication ISDN
  • ADSL ADSL
  • speech recognition method of the present invention only speech signals continuously inputted in chronological order are used, and extra processing steps for separately storing and processing a reference signal using multiple microphones and hardware resources for the extra steps are not required. Furthermore, availability of speech recognition can be expanded without use of a head-set type microphone or a hand microphone for acquiring a reference signal as a “speech model affected by intra-frame echo influence”
  • Each functional portion or functional means is implemented by causing a computer to execute a program, and is not necessarily required to be incorporated as a component for each functional block shown in the drawings.
  • a computer-readable programming language for configuration of a speech recognition device of the present invention the assembler language, the FORTRAN, the C language, the C++ language, Java® and the like are included.
  • a computer-executable program for causing a speech recognition method of the present invention to be executed can be stored in a ROM, EEPROM, flash memory, CD-ROM, DVD, flexible disk, hard disk and the like for distribution.
  • An impulse response actually measured in a room was used to create a speech under echoes.
  • a frame value corresponding to 300 msec was used as an echo time for the embodiment example, an reference example and a comparison example.
  • the distance between a sound source and a microphone was set to be 2 m, and a speaking voice was inputted into the microphone from its front side.
  • the sampling frequency of 12 kHz, the window width of 32 msec, and the analysis period of 8 msec were used as signal analysis conditions.
  • a sixteen dimensional MFCC (Mel Frequency Cepstral Coefficient) was used as an acoustic feature quantity.
  • the recognition success rate was 86%, which is lower than the success rate of the embodiment example of the present invention. That is, it has been proved that, according to the present invention, a recognition success rate better than that of conventional methods can be provided though one-channel data is used therein.
US10/849,724 2003-05-21 2004-05-20 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium Abandoned US20050010410A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/476,650 US8024184B2 (en) 2003-05-21 2009-06-02 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2003143224A JP3836815B2 (ja) 2003-05-21 2003-05-21 音声認識装置、音声認識方法、該音声認識方法をコンピュータに対して実行させるためのコンピュータ実行可能なプログラムおよび記憶媒体
JP20003-143224 2003-05-21

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/476,650 Continuation US8024184B2 (en) 2003-05-21 2009-06-02 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium

Publications (1)

Publication Number Publication Date
US20050010410A1 true US20050010410A1 (en) 2005-01-13

Family

ID=33531072

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/849,724 Abandoned US20050010410A1 (en) 2003-05-21 2004-05-20 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium
US12/476,650 Expired - Fee Related US8024184B2 (en) 2003-05-21 2009-06-02 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/476,650 Expired - Fee Related US8024184B2 (en) 2003-05-21 2009-06-02 Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium

Country Status (2)

Country Link
US (2) US20050010410A1 (ja)
JP (1) JP3836815B2 (ja)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070198255A1 (en) * 2004-04-08 2007-08-23 Tim Fingscheidt Method For Noise Reduction In A Speech Input Signal
WO2007130026A1 (en) * 2006-05-01 2007-11-15 Nippon Telegraph And Telephone Corporation Method and apparatus for speech dereverberation based on probabilistic models of source and room acoustics
US20080175423A1 (en) * 2006-11-27 2008-07-24 Volkmar Hamacher Adjusting a hearing apparatus to a speech signal
US20090144059A1 (en) * 2007-12-03 2009-06-04 Microsoft Corporation High performance hmm adaptation with joint compensation of additive and convolutive distortions
US20090248403A1 (en) * 2006-03-03 2009-10-01 Nippon Telegraph And Telephone Corporation Dereverberation apparatus, dereverberation method, dereverberation program, and recording medium
CN103578465A (zh) * 2013-10-18 2014-02-12 威盛电子股份有限公司 语音辨识方法及电子装置
US20160125880A1 (en) * 2013-05-28 2016-05-05 Zhigang Zhang Method and system for identifying location associated with voice command to control home appliance
US9373338B1 (en) * 2012-06-25 2016-06-21 Amazon Technologies, Inc. Acoustic echo cancellation processing based on feedback from speech recognizer
US9672821B2 (en) 2015-06-05 2017-06-06 Apple Inc. Robust speech recognition in the presence of echo and noise using multiple signals for discrimination
US20170263853A1 (en) * 2014-09-03 2017-09-14 Intel Corporation Spin transfer torque memory and logic devices having an interface for inducing a strain on a magnetic layer therein
CN110503970A (zh) * 2018-11-23 2019-11-26 腾讯科技（深圳）有限公司 一种音频数据处理方法、装置及存储介质
EP3573049A1 (en) * 2018-05-24 2019-11-27 Dolby Laboratories Licensing Corp. Training of acoustic models for far-field vocalization processing systems
CN113327584A (zh) * 2021-05-28 2021-08-31 平安科技（深圳）有限公司 语种识别方法、装置、设备及存储介质
US11257503B1 (en) * 2021-03-10 2022-02-22 Vikram Ramesh Lakkavalli Speaker recognition using domain independent embedding

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4512848B2 (ja) * 2005-01-18 2010-07-28 株式会社国際電気通信基礎技術研究所 雑音抑圧装置及び音声認識システム
JP4586577B2 (ja) * 2005-03-02 2010-11-24 株式会社国際電気通信基礎技術研究所 外乱成分抑圧装置、コンピュータプログラム、及び音声認識システム
JP4107613B2 (ja) 2006-09-04 2008-06-25 インターナショナル・ビジネス・マシーンズ・コーポレーション 残響除去における低コストのフィルタ係数決定法
US7856353B2 (en) 2007-08-07 2010-12-21 Nuance Communications, Inc. Method for processing speech signal data with reverberation filtering
JP4977062B2 (ja) * 2008-02-29 2012-07-18 日本電信電話株式会社 残響除去装置とその方法と、そのプログラムと記録媒体
JP5642339B2 (ja) * 2008-03-11 2014-12-17 トヨタ自動車株式会社 信号分離装置及び信号分離方法
US9634855B2 (en) 2010-05-13 2017-04-25 Alexander Poltorak Electronic personal interactive device that determines topics of interest using a conversational agent
JP6077957B2 (ja) * 2013-07-08 2017-02-08 本田技研工業株式会社 音声処理装置、音声処理方法、及び音声処理プログラム
JP6261043B2 (ja) * 2013-08-30 2018-01-17 本田技研工業株式会社 音声処理装置、音声処理方法、及び音声処理プログラム
CN103632667B (zh) * 2013-11-25 2017-08-04 华为技术有限公司 声学模型优化方法、装置及语音唤醒方法、装置和终端
US9953646B2 (en) 2014-09-02 2018-04-24 Belleau Technologies Method and system for dynamic speech recognition and tracking of prewritten script
CN107564527A (zh) * 2017-09-01 2018-01-09 平顶山学院 嵌入式系统的汉英双语语音识别方法
US10810996B2 (en) * 2018-07-31 2020-10-20 Nuance Communications, Inc. System and method for performing automatic speech recognition system parameter adjustment via machine learning
CN111063342B (zh) * 2020-01-02 2022-09-30 腾讯科技（深圳）有限公司 语音识别方法、装置、计算机设备及存储介质
CN111508498B (zh) * 2020-04-09 2024-01-30 携程计算机技术（上海）有限公司 对话式语音识别方法、系统、电子设备和存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5956679A (en) * 1996-12-03 1999-09-21 Canon Kabushiki Kaisha Speech processing apparatus and method using a noise-adaptive PMC model
US5970446A (en) * 1997-11-25 1999-10-19 At&T Corp Selective noise/channel/coding models and recognizers for automatic speech recognition
US6026359A (en) * 1996-09-20 2000-02-15 Nippon Telegraph And Telephone Corporation Scheme for model adaptation in pattern recognition based on Taylor expansion

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098040A (en) * 1997-11-07 2000-08-01 Nortel Networks Corporation Method and apparatus for providing an improved feature set in speech recognition by performing noise cancellation and background masking

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026359A (en) * 1996-09-20 2000-02-15 Nippon Telegraph And Telephone Corporation Scheme for model adaptation in pattern recognition based on Taylor expansion
US5956679A (en) * 1996-12-03 1999-09-21 Canon Kabushiki Kaisha Speech processing apparatus and method using a noise-adaptive PMC model
US5970446A (en) * 1997-11-25 1999-10-19 At&T Corp Selective noise/channel/coding models and recognizers for automatic speech recognition

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070198255A1 (en) * 2004-04-08 2007-08-23 Tim Fingscheidt Method For Noise Reduction In A Speech Input Signal
US8271277B2 (en) 2006-03-03 2012-09-18 Nippon Telegraph And Telephone Corporation Dereverberation apparatus, dereverberation method, dereverberation program, and recording medium
US20090248403A1 (en) * 2006-03-03 2009-10-01 Nippon Telegraph And Telephone Corporation Dereverberation apparatus, dereverberation method, dereverberation program, and recording medium
WO2007130026A1 (en) * 2006-05-01 2007-11-15 Nippon Telegraph And Telephone Corporation Method and apparatus for speech dereverberation based on probabilistic models of source and room acoustics
US20090110207A1 (en) * 2006-05-01 2009-04-30 Nippon Telegraph And Telephone Company Method and Apparatus for Speech Dereverberation Based On Probabilistic Models Of Source And Room Acoustics
US8290170B2 (en) 2006-05-01 2012-10-16 Nippon Telegraph And Telephone Corporation Method and apparatus for speech dereverberation based on probabilistic models of source and room acoustics
US20080175423A1 (en) * 2006-11-27 2008-07-24 Volkmar Hamacher Adjusting a hearing apparatus to a speech signal
US20090144059A1 (en) * 2007-12-03 2009-06-04 Microsoft Corporation High performance hmm adaptation with joint compensation of additive and convolutive distortions
US8180637B2 (en) 2007-12-03 2012-05-15 Microsoft Corporation High performance HMM adaptation with joint compensation of additive and convolutive distortions
US9373338B1 (en) * 2012-06-25 2016-06-21 Amazon Technologies, Inc. Acoustic echo cancellation processing based on feedback from speech recognizer
US20160125880A1 (en) * 2013-05-28 2016-05-05 Zhigang Zhang Method and system for identifying location associated with voice command to control home appliance
CN103578465A (zh) * 2013-10-18 2014-02-12 威盛电子股份有限公司 语音辨识方法及电子装置
US20170263853A1 (en) * 2014-09-03 2017-09-14 Intel Corporation Spin transfer torque memory and logic devices having an interface for inducing a strain on a magnetic layer therein
US9672821B2 (en) 2015-06-05 2017-06-06 Apple Inc. Robust speech recognition in the presence of echo and noise using multiple signals for discrimination
EP3573049A1 (en) * 2018-05-24 2019-11-27 Dolby Laboratories Licensing Corp. Training of acoustic models for far-field vocalization processing systems
US10872602B2 (en) 2018-05-24 2020-12-22 Dolby Laboratories Licensing Corporation Training of acoustic models for far-field vocalization processing systems
CN110503970A (zh) * 2018-11-23 2019-11-26 腾讯科技（深圳）有限公司 一种音频数据处理方法、装置及存储介质
US11257503B1 (en) * 2021-03-10 2022-02-22 Vikram Ramesh Lakkavalli Speaker recognition using domain independent embedding
CN113327584A (zh) * 2021-05-28 2021-08-31 平安科技（深圳）有限公司 语种识别方法、装置、设备及存储介质

Also Published As

Publication number Publication date
JP2004347761A (ja) 2004-12-09
JP3836815B2 (ja) 2006-10-25
US20090306977A1 (en) 2009-12-10
US8024184B2 (en) 2011-09-20

Similar Documents

Publication Publication Date Title
US8024184B2 (en) Speech recognition device, speech recognition method, computer-executable program for causing computer to execute recognition method, and storage medium
JP6480644B1 (ja) マルチチャネル音声認識のための適応的オーディオ強化
Kumar et al. A Hindi speech recognition system for connected words using HTK
US7016849B2 (en) Method and apparatus for providing speech-driven routing between spoken language applications
EP2609587B1 (en) System and method for recognizing a user voice command in noisy environment
US6424943B1 (en) Non-interactive enrollment in speech recognition
JP5218052B2 (ja) 言語モデル生成システム、言語モデル生成方法および言語モデル生成用プログラム
JP4274962B2 (ja) 音声認識システム
JP4195428B2 (ja) 多数の音声特徴を利用する音声認識
US20170092268A1 (en) Methods for speech enhancement and speech recognition using neural networks
JP5459680B2 (ja) 音声処理システム及び方法
Hirsch et al. A new approach for the adaptation of HMMs to reverberation and background noise
KR101120765B1 (ko) 스위칭 상태 스페이스 모델과의 멀티모덜 변동 추정을이용한 스피치 인식 방법
JPH07334184A (ja) 音響カテゴリ平均値計算装置及び適応化装置
KR20040088368A (ko) 스위칭 상태 공간 모델들을 갖는 변분 추론을 사용하는음성 인식 방법
WO2018078885A1 (ja) 対話装置、対話方法及び対話用コンピュータプログラム
US20040143435A1 (en) Method of speech recognition using hidden trajectory hidden markov models
JP4836076B2 (ja) 音声認識システム及びコンピュータプログラム
Hansen et al. CU-MOVE: Advanced in-vehicle speech systems for route navigation
JP2009116075A (ja) 音声認識装置
JP4362054B2 (ja) 音声認識装置及び音声認識プログラム
Menne Learning acoustic features from the raw waveform for automatic speech recognition
Sarikaya Robust and efficient techniques for speech recognition in noise
Yogapriya et al. Speech Based Access for Agricultural Commodity Prices in Tamil
Scholar Development of a Robust Speech-to-Text Algorithm for Nigerian English Speakers 1Mohammed M. Sulaiman, 2Yahya S. Hadi, 1Mohammed Katun and 1Shehu Yakubu

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TAKIGUCHI, TETSUYA;NISHIMURA, MASAFUMI;REEL/FRAME:015813/0014

Effective date: 20040524

AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:022689/0317

Effective date: 20090331

Owner name: NUANCE COMMUNICATIONS, INC.,MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:022689/0317

Effective date: 20090331

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE