JP2010072446A - Coarticulation feature extraction device, coarticulation feature extraction method and coarticulation feature extraction program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a coarticulation feature extraction device, a coarticulation feature extraction method and a coarticulation feature extraction program capable of extracting unknown words with high phoneme identification accuracy which meets requirements from voice interaction and voice search. <P>SOLUTION: In the coarticulation feature extraction device, a voice which is inputted from an input section 201 is converted to digital in an A-D converter 202, and Fourier analysis and filtering are performed in a feature analysis section 210, as a result, a voice spectrum data are obtained. Then, a coarticulation feature sequence being time sequence data of a coarticulation feature is extracted in a coarticulation feature extraction section 220. A speed component and an acceleration component are extracted from a deviation component of the coarticulation feature sequence in a coarticulation movement modification section 230. Coarticulation movement is modified by passing a neural network based on each component. Based on the modified coarticulation movement, corresponding words are searched in a word classification section 204, and voice recognition is performed. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an articulation feature extraction device, an articulation feature extraction method, and an articulation feature extraction program. More specifically, the present invention relates to an articulation feature extraction device, an articulation feature extraction method, and an articulation feature extraction program for identifying articulation movements accompanying voice utterance with high accuracy.

  Speech recognition technology is generally known as a user interface using speech. In speech recognition technology, pattern recognition processing using phonemes, syllables, words, and the like as recognition units has been generally performed based on the result of feature analysis processing such as frequency spectrum. This is based on the assumption that the human auditory nervous system has the ability to analyze spectrum and subsequently performs higher-level language processing in the cerebrum. Voice recognition devices developed so far perform word classification directly from acoustic features. On the other hand, from recent brain research, the hypothesis that humans perceive speech as articulatory motion rather than speech as an acoustic signal is promising (see Non-Patent Document 1).

  An outline of standard speech recognition technology will be described with reference to FIG. FIG. 15 is a functional block diagram illustrating an example of a standard speech recognition technology installed in the speech recognition apparatus. As shown in FIG. 15, an input unit 101, an A / D conversion unit 102, a feature analysis unit 103, a word classification unit 104, an output unit 105, and a storage unit 106 are provided as functional blocks necessary for speech recognition. . The storage unit 106 also stores a word pronunciation dictionary 107, a hidden Markov model (HMM) 108, a language model 109, and other data. In this speech recognition apparatus, a recognition target word set is determined in advance, and the language model 109 (representing a chain probability between words in a table. Usually, the probability of three word chains is used. This is represented by 3 (tri) -gram. The word string in the speech signal is searched for with reference to FIG.

  The input unit 101 is provided for receiving audio input from the outside and converting it into an analog electrical signal. The A / D converter 102 is provided for converting the analog signal received by the input unit 101 into a digital signal. The feature analysis unit 103 is provided to extract a predetermined feature amount for speech recognition. The word classification unit 104 is provided for searching for a word included in the speech based on the feature amount extracted by the feature analysis unit 103. The storage unit 106 stores data necessary when the word classification unit 104 searches for a word, and is referred to by the word classification unit 104. The output unit 105 is provided to output a word obtained as a result of searching in the word classification unit 104.

  The flow of word string determination based on the functional blocks in FIG. 15 will be outlined. The unknown speech input from the input unit 101 is discretized through the A / D conversion unit 102 and converted into a digital signal. Next, in the feature analysis unit 103, the converted digital signal is Fourier-analyzed and passed through a band-pass filter (BPF) of about 24 channels to remove noise components. As a result, a speech spectrum is extracted. In recent standard speech recognition, the frequency of the speech spectrum is mel scaled according to the auditory characteristics, and the mel cepstrum (MFCC) obtained by performing discrete cosine transform (DCT) on the logarithm of the spectrum is converted into speech. Often used as a spectral feature.

Next, the word classification unit 104 searches for words included in the input speech based on the spectrum obtained by the feature analysis unit 103. In the word classification unit 104, first, phoneme sequences (these are stored in the word pronunciation dictionary 107) constituting the word are extracted. Next, the acoustic likelihood is calculated with reference to the HMM 108 prepared for each phoneme unit. The acoustic likelihood Lk for the word k (or phoneme K) of the input speech feature X is calculated by Equation (1) and then cumulatively added with the acoustic likelihood Lk along the state transition of the HMM 108.

Here, μ _k is an average vector, and Σ _k ⁻¹ and | Σ _k | are an inverse matrix and a determinant of a covariance matrix, respectively. In practice, since the method of sequentially reading the phoneme sequence from the word pronunciation dictionary 107 is inefficient, the phoneme sequence is preliminarily expressed in a single tree structure graph for all the words to be recognized, and the acoustic likelihood of the phoneme is displayed on the graph. A technique such as advancing a search while accumulating degrees is used.

  In addition, a so-called beam search that cuts a path with a low cumulative likelihood is generally applied during the word search in the word classification unit 104 to increase the speed. The language model 109 is referred to when determining which word to search. At the beginning of the search, all the words at the beginning of the sentence are targeted. When this search is completed, the chain probability of the language model 109 is referred to and the next connectable word is determined.

  The cumulative likelihood used during word search in the word classification unit 104 is obtained by weighted addition of acoustic likelihood and word chain likelihood (these are used as logarithm values of probability values). Desired. The weighting coefficient at the time of weighted addition is two different likelihoods: the acoustic likelihood of the HMM 108 and the word chain likelihood obtained from the language corpus (in terms of value, the word chain likelihood is about one digit smaller). It is necessary to combine the degrees, and is determined in a balanced manner from the simulation. At the end of the input speech, a word sequence giving the maximum cumulative likelihood is taken out as a recognition result. (See Non-Patent Document 2 and Non-Patent Document 3)

  The word searched through the above processing is output from the output unit 105 as a result of recognizing the word included in the speech received from the input unit 101. Thus, in the conventional standard speech recognition apparatus, it is possible to obtain high recognition accuracy by combining the acoustic likelihood of the HMM 108 and the word chain likelihood of the language model 109.

Here, the number of words that can be recognized by speech depends on the scale of the language corpus stored in the word pronunciation dictionary. The larger the size of the language corpus, the larger the number of recognizable words. However, the size of the language corpus is limited due to storage area and processing time constraints. Under such circumstances, it is possible to reduce the processing time by suppressing the capacity of the language model while maintaining the speech recognition accuracy by registering and using a word input repeatedly a predetermined number of times as a language model. A voice recognition device has been proposed (see, for example, Patent Document 1).
JP 2007-248529 A Makio Makino, Journal of the Acoustical Society of Japan, Vol. 62, No. 5, pp. 391-396 (2006) Akio Ando, Real-Time Speech Recognition, IEICE (2003), pp. 4-9 “1.3 Outline of Speech Recognition Technology” Kiyohiro Shikano, Katsunobu Ito, Tatsuya Kawahara, Kazuya Takeda, Mikio Yamamoto, Speech Recognition System, Ohmsha (2001 (Heisei 13)) pp. 93-110 "Chapter 6 Large Vocabulary Continuous Speech Recognition Algorithm"

  However, the above speech recognition apparatus has a problem that it is impossible to recognize unknown words. Further, even when a large-scale language corpus is used, there is a problem that it is impossible to cover all words.

  In addition, in order to realize a speech recognition device that can handle unknown words, a means capable of recognizing phonemes with high accuracy is required. However, current speech recognition devices have no language model, that is, refer to a word dictionary. When it is not possible, the phoneme recognition performance is limited to 60 to 80% (note that a human can listen to phonemes with a high accuracy of 98% or more, and thus can efficiently treat unknown words and the like effectively. ). For the above reasons, there is a problem that introduction of a voice interface is a major factor in applications such as voice conversation and voice search in which recognition of unknown words is indispensable.

  On the other hand, methods for expressing speech with articulatory features have been proposed in the field of phonetics for a long time. A standard notation has also been proposed as an International Phonetic Alphabet (IPA). Also, distinguishing features (voiced / non-voiced / continuity / semi-vowel / bursting / friction / friction / tip) classifying phonemes (phonemes) based on structural features related to articulation / Nasal tone / high tongue / low tongue / (the position where the tongue rises) is forward / backward / ...; Distinctive Feature (DF) has been proposed for a long time. In addition, many methods for directly extracting articulatory features such as discriminative features from speech have been proposed, including a method using a neural network (see Non-Patent Document 4).

  FIG. 16 shows a distinctive phonetic feature (DPF) regarding Japanese phonemes. Here, the discriminative phoneme feature is one method of expressing articulatory features. In the figure, the vertical column shows the distinguishing features, and the horizontal column shows the individual phonemes. Then, it is possible to know the operation of the vocal organs necessary for generating one phoneme from this table. In FIG. 16, nil (high / low) and nil (front / rear) are phonemes that do not belong to either high tongue / low tongue, respectively, and (the position where the tongue rises) is forward / backward. In order to assign a discrimination feature to a phoneme that does not belong to any of the above, it indicates that it is a newly added feature. It is known that the speech recognition performance is improved by balancing the phonemes.

  However, when speech recognition is performed from the extracted discriminative phoneme features, the actual situation is that remarkable performance is not obtained as compared with conventional features characterized by a speech spectrum or speech cepstrum (see Non-Patent Document 5). ).

The present invention has been made to solve the above-mentioned problems, and it is capable of dealing with unknown words, and it has a high phoneme identification accuracy capable of withstanding the demands from voice conversation and voice search, It is an object to provide an articulation feature extraction method and an articulation feature extraction program.
Shuichi Itabashi, Speech Engineering, Morikita Publishing (1973) pp. 6-pp. 10 2.1.1. Speech / phonemes / syllables (Table 2.2 Japanese discrimination features) Takashi Fukuda, Tsuneo Nitta, "Orthogonalized Distinctive Phonetic Feature Extraction for Noise-robust Automatic Speech Recognition", IEICE English Journal, Vol.E87-D, No.5, pp.1110-1118, (2004-5 ).

  In order to solve the above-described problem, in the articulatory feature extraction apparatus according to the first aspect of the present invention, a voice acquisition unit that acquires voice, and an articulation feature that extracts the articulation feature of the voice acquired by the voice acquisition unit. An articulation feature sequence that is time-series data of the articulation feature extracted by the extraction means and the articulation feature extraction unit is converted into a motion trajectory, and based on the motion trajectory, the influence of articulation coupling included in the speech is Articulation motion correcting means for correcting articulation motion, which is an articulatory motion represented by the articulatory feature series, so that phoneme recognition is possible, and correction of the articulatory motion corrected by the articulatory motion correcting means. Storage control means for storing the articulatory movement in the storage means.

  According to a second aspect of the present invention, the articulatory feature extracting apparatus further comprises a component extracting means for extracting a speed component and an acceleration component from the displacement component of the articulatory feature series in addition to the configuration of the first aspect of the invention. And the articulatory motion correcting means corrects the articulatory motion to the corrected articulatory motion based on at least one of the displacement component, the velocity component extracted by the component extractor, and the acceleration component. It is characterized by.

  In addition, in the articulatory feature extraction device according to a third aspect, in addition to the configuration according to the second aspect, the displacement is based on at least one of the displacement component, the velocity component, and the acceleration component. When the component is observed along the time axis, it comprises pattern recognition means for recognizing whether the transition is a concave pattern or a convex pattern, and the articulatory movement correction means is recognized by the pattern recognition means Based on the pattern, the articulatory motion is corrected to the corrected articulatory motion.

  In addition, in the articulatory feature extraction device according to a fourth aspect of the invention, in addition to the configuration of the invention according to the second or third aspect, the articulatory motion correcting means includes the displacement component, the velocity component, and the neural network. The articulatory motion is corrected to the corrected articulatory motion by passing at least one of the acceleration components.

  In the articulation feature extraction method of the invention according to claim 5, a sound acquisition step of acquiring sound, an articulation feature extraction step of extracting the articulation feature of the sound acquired in the sound acquisition step, and the articulation feature extraction The articulatory feature sequence, which is time series data of the articulatory features extracted in the step, is converted into a motion trajectory, and based on the motion trajectory, the effect of articulation coupling included in the speech is eliminated so that phoneme recognition is possible. Further, the articulatory motion correcting step for correcting the articulatory motion represented by the articulatory feature series, and the corrected articulatory motion being the articulatory motion corrected in the articulatory motion correcting step are stored in the storage means. And a storage control step.

  Further, in the articulation feature extraction method of the invention according to claim 6, in addition to the configuration of the invention of claim 5, a component extraction step of extracting a speed component and an acceleration component from the displacement component of the articulation feature series is provided. And the articulatory motion correction step corrects the articulatory motion to the corrected articulatory motion based on at least one of the displacement component, the velocity component extracted in the component extraction step, and the acceleration component. It is characterized by.

  In addition, in the articulatory feature extraction method of the invention according to claim 7, in addition to the configuration of the invention of claim 6, the displacement is based on at least one of the displacement component, the velocity component, and the acceleration component. A pattern recognition step for recognizing whether the transition is a concave pattern or a convex pattern when the component is observed along the time axis, and the articulatory motion correction step is recognized in the pattern recognition step Based on the pattern, the articulatory motion is corrected to the corrected articulatory motion.

  In the articulatory feature extraction method according to an eighth aspect of the present invention, in addition to the configuration of the invention according to the sixth or seventh aspect, the articulatory motion correcting step includes a neural network that includes the displacement component, the velocity component, and the The articulatory motion is corrected to the corrected articulatory motion by passing at least one of the acceleration components.

  In the articulation feature extraction program according to the ninth aspect, the computer is driven as each processing means of the articulation feature extraction device according to any one of the first to fourth aspects.

  The articulatory feature extraction apparatus according to the first aspect of the present invention can extract the characteristics of the articulatory operation accompanying voice utterance with high accuracy. Thereby, compared with the conventional speech recognition apparatus which recognizes a speech using a speech spectrum, it becomes possible to perform highly accurate speech recognition.

  In conventional speech recognition characterized by the spectrum of speech, the spectrum varies greatly depending on the speaker, the context at the time of speech, ambient noise, etc., so it is often used for designing HMMs used for obtaining acoustic likelihood. Needed voice data. Further, the number of HMMs to be mixed is required to be 10 or more, and the cost has been increased in order to obtain a high-performance speech recognition apparatus. On the other hand, the articulatory feature extraction apparatus of the present invention can extract articulatory features in speech with high accuracy, so that only a few HMMs are mixed. In the case of the conventional method using the speech spectrum as a feature, various information other than linguistic information, for example, external noise and articulation combination during speech (effect of phonemes before and after) are mixed, resulting in phonemes and words for classification purposes. The deformation of will increase explosively. In recent speech recognition devices based on HMM, the speech spectrum (in fact, the frequency of the speech spectrum is matched to the auditory characteristics and the frequency is Mel scaled, and the logarithmic value of the spectrum is discrete cosine transformed (DCT). When the Mel cepstrum (Mel Frequency Cepstrum Coefficient; commonly called MFCC) is used directly as an input feature, the variation of individual vector elements is expressed from multiple normal distributions. A plurality of normal distributions are called mixed distributions, and in order to cope with the various variations described above, in recent years, those using distributions of 60 to 70 have appeared. In this way, the reason why a large amount of memory and computation are required is the result of trying to classify phonemes and words without specifying the hidden variables in the speech. According to the present invention, as a result of specifying the hidden variable as the articulatory operation, the scale of the phoneme classifier and the word classifier (here, the number of mixtures) can be reduced to a small scale.

  Also, high-precision extraction of articulatory features can dramatically improve phoneme recognition performance, and can respond to the unknown word problem in the same way as humans do. Therefore, it is possible to smoothly advance the dialogue by synthesizing the confirmation utterance sentence using the phoneme sequence.

  Furthermore, in many cases, articulation features correspond one-to-one with text (reading converted into a kana sequence), so that searches for speech documents and text documents can be performed from both speech and text (keyboard). It becomes possible.

  In addition to the effect of the invention described in claim 1, the articulatory feature extraction device according to claim 2 corrects the articulatory motion based on at least one of a displacement component, a velocity component, and an acceleration component. Since the movement is corrected to the articulatory movement, it is possible to extract the characteristics of the articulatory movement accompanying the voice utterance with high accuracy without depending on the speaker, the context at the time of utterance, ambient noise, and the like.

  In addition to the effect of the invention according to claim 2, the articulatory feature extraction device according to claim 3 is characterized in that the modified articulatory motion has a transition pattern (recessed) when the displacement component is observed along the time axis. Pattern, convex pattern), it is possible to extract the characteristics of articulatory movements associated with speech utterance with high accuracy even if the phoneme is different from the state of a single phone due to articulation coupling. It becomes.

  Further, the articulatory feature extraction device of the invention according to claim 4 can obtain a corrected articulatory motion at high speed by using a neural network in addition to the effect of the invention of claim 2 or 3.

  Further, the articulation feature extraction method of the invention according to claim 5 can extract the feature of the articulation operation accompanying the voice utterance with high accuracy. Thereby, compared with the conventional speech recognition apparatus which recognizes a speech using a speech spectrum, it becomes possible to perform highly accurate speech recognition.

  In the conventional speech recognition characterized by the spectrum of speech, the spectrum greatly fluctuates depending on the speaker, the context at the time of speech, ambient noise, etc., so the hidden Markov model (HMM) used when obtaining the acoustic likelihood ) Required a lot of audio data. Further, the number of HMMs to be mixed is required to be 10 or more, and the cost has been increased in order to obtain a high-performance speech recognition apparatus. On the other hand, the articulatory feature extraction apparatus of the present invention can extract articulatory features in speech with high accuracy, so that only a few HMMs are mixed. In the case of the conventional method using the speech spectrum as a feature, various information other than linguistic information, for example, external noise and articulation combination during speech (effect of phonemes before and after) are mixed, resulting in phonemes and words for classification purposes. The deformation of will increase explosively. In recent speech recognition devices based on HMM, the speech spectrum (in fact, the frequency of the speech spectrum is matched to the auditory characteristics and the frequency is Mel scaled, and the logarithmic value of the spectrum is discrete cosine transformed (DCT). When the Mel cepstrum (Mel Frequency Cepstrum Coefficient; commonly called MFCC) is used directly as an input feature, the variation of individual vector elements is expressed from multiple normal distributions. A plurality of normal distributions are called mixed distributions, and in order to cope with the various variations described above, in recent years, those using distributions of 60 to 70 have appeared. In this way, the reason why a large amount of memory and computation are required is the result of trying to classify phonemes and words without specifying the hidden variables in the speech. According to the present invention, as a result of specifying the hidden variable as the articulatory operation, the scale of the phoneme classifier and the word classifier (here, the number of mixtures) can be reduced to a small scale.

  Also, high-precision extraction of articulatory features can dramatically improve phoneme recognition performance, and can respond to the unknown word problem in the same way as humans do. Therefore, it is possible to smoothly advance the dialogue by synthesizing the confirmation utterance sentence using the phoneme sequence.

  Furthermore, in many cases, articulation features correspond one-to-one with text (reading converted into a kana sequence), so that searches for speech documents and text documents can be performed from both speech and text (keyboard). It becomes possible.

  In addition to the effect of the invention described in claim 5, the articulatory feature extraction method of the invention according to claim 6 is modified based on at least one of a displacement component, a velocity component, and an acceleration component. Since the movement is corrected to the articulatory movement, it is possible to extract the characteristics of the articulatory movement accompanying the voice utterance with high accuracy without depending on the speaker, the context at the time of utterance, ambient noise, and the like.

  In addition to the effect of the invention according to claim 6, the articulatory feature extraction method according to the invention according to claim 7 has a modified articulation motion that is a transition pattern (recessed) when the displacement component is observed along the time axis. Pattern, convex pattern), it is possible to extract the characteristics of articulatory movements associated with speech utterance with high accuracy even if the phoneme is different from the state of a single phone due to articulation coupling. It becomes.

  The articulatory feature extraction method of the invention according to claim 8 can obtain a corrected articulatory motion at high speed by using a neural network in addition to the effect of the invention of claim 6 or 7.

  The articulation feature extraction program of the invention according to claim 9 can drive a computer as each processing means of the articulation feature extraction device according to any one of claims 1 to 4.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the articulation feature extraction device and articulation feature extraction method of the present invention will be described below with reference to the drawings. These drawings are used for explaining the technical features that can be adopted by the present invention, and the configuration of the apparatus described, the flowcharts of various processes, etc., unless otherwise specified. It is not intended to be limited to that, but merely an illustrative example.

  First, the electrical configuration of the articulation feature extraction apparatus 1 will be described with reference to FIG. FIG. 1 is a schematic diagram showing an electrical configuration of the articulation feature extraction device 1. As shown in FIG. 1, the articulation feature extraction device 1 includes a central processing unit 11, an input device 12, an output device 13, a storage device 14, and an external storage device 15.

  The central processing unit 11 is provided for performing processing such as numerical computation and control, and performs computation and processing according to the processing procedure described in the present embodiment. For example, a CPU or the like can be used. The input device 12 is configured by a microphone, a keyboard, or the like, and inputs a voice uttered by a user or a character string input by a key. The output device 13 includes a display, a speaker, and the like, and outputs a feature extraction result or information obtained by processing the feature extraction result. The storage device 14 stores a processing procedure (articulation feature extraction program) executed by the central processing unit 11 and temporary data necessary for the processing. For example, ROM (Read Only Memory) or RAM (Random Access Memory) can be used. The external storage device 15 also includes a feature analysis coefficient set used for articulation feature extraction processing, a neural network weighting coefficient set used for articulation feature extraction processing, a coefficient set used for articulation motion correction processing, and speech recognition processing. This model is provided to store models necessary for data, input voice data, analysis result data, and the like. For example, a hard disk drive (HDD) can be used. These are electrically connected via the bus 22 so that data can be transmitted and received between them.

  The hardware configuration of the articulatory feature extraction apparatus 1 of the present invention is not limited to the configuration shown in FIG. Accordingly, a communication I / F connected to a communication network such as the Internet may be provided.

  In the present embodiment, the articulation feature extraction device 1 and the articulation feature extraction program have a configuration independent of other systems, but the present invention is not limited to this configuration. Therefore, a configuration incorporated as a part of another device or a configuration incorporated as a part of another program is also possible. Further, the input in that case is indirectly performed through the other devices and programs described above.

  Next, storage data stored in the external storage device 15 will be described. As shown in FIG. 1, the external storage device 15 stores a word pronunciation dictionary storage area 16 in which a word pronunciation dictionary is stored, a hidden Markov model storage area 17 in which a hidden Markov model is stored, and a language model. Language model storage area 18, coefficient storage area 19 in which coefficients used in each process are stored, input speech storage area 20 in which input speech is stored, and processing result storage area in which processed data is stored 21 and other areas are provided.

  In the word pronunciation dictionary storage area 16, phoneme strings constituting words are stored. The hidden Markov model storage area 17 stores a hidden Markov model that is referred to when speech recognition is performed in the central processing unit 11. In the language model storage area 18, recognizable word models (language corpus) are stored. The coefficient storage area 19 stores a feature analysis coefficient set used for articulation feature extraction processing, a neural network weighting coefficient set used for articulation feature extraction processing, a coefficient set used for articulation motion correction processing, and the like. The The input voice storage area 20 stores voice data input via the input device 12. The processing result storage area 21 stores data obtained as a result of various processes executed in the central processing unit 11. Details of these data will be described later.

  Next, speech recognition processing executed by the articulatory feature extraction apparatus 1 of the present invention will be described with reference to FIGS. FIG. 2 is a functional block diagram showing the articulation feature extraction processing executed by the articulation feature extraction device 1. FIG. 3 is a block diagram illustrating details of functions of the feature analysis unit 210. FIG. 4 is a block diagram showing details of the function of the articulation feature extraction unit 220. FIG. 5 is an example of local features in the time direction of the feature analysis unit obtained from the local feature extraction unit 221. FIG. 6 is an example of local features in the frequency direction obtained from the local feature extraction unit 221. FIG. 7 is an example of the articulation feature obtained by the discriminative phoneme feature extraction unit 222. FIG. 8 is a block diagram showing details of the function of the articulation motion correction unit 230. FIG. 9 is a flowchart showing processing in the articulation motion correction processing unit 232.

  As shown in FIG. 2, as a functional block necessary for the articulation extraction process executed in the articulation feature extraction apparatus 1 of the present invention, an input unit 201, an A / D conversion unit 202, a feature analysis unit 210, and an articulation feature extraction unit 220. , An articulatory motion correction unit 230, a word classification unit 204, an output unit 205, a storage unit 206, and a storage unit 207 are provided.

  The storage unit 207 stores various coefficient sets 2071. The feature analysis unit 210, the articulation feature extraction unit 220, and the articulation motion correction unit 230 are in a state where the stored coefficient set can be referred to. The storage unit 206 stores a pronunciation word dictionary 2061, a hidden Markov model 2062, a language model 2063, and other data. The data stored by the word classification unit 204 can be referred to.

  Note that the input unit 201, A / D conversion unit 202, word classification unit 204, and output unit 205 in FIG. 2 have the same functions as the corresponding parts in the conventional speech recognition processing apparatus shown in FIG. The description is omitted or simplified.

  The input unit 201 is provided for receiving a sound input from the outside and converting it into an analog electric signal. The A / D conversion unit 202 is provided to convert an analog signal received by the input unit 201 into a digital signal. The feature analysis unit 210 is provided to extract a predetermined feature amount necessary for speech recognition (see FIG. 3, details will be described later). The articulation feature extraction unit 220 is provided to extract time series data of articulation features (hereinafter referred to as “articulation feature series”) from the time series data of the feature amounts extracted by the feature analysis unit 210 (FIG. 4). See also details below). The articulatory motion correcting unit 230 is provided to convert the articulatory feature sequence extracted by the articulatory feature extracting unit 220 into a motion trajectory, and further correct the converted motion trajectory based on a predetermined rule (FIG. 8). See details below.)

  The word classification unit 204 is provided for searching for a word included in the speech based on the corrected articulation motion (hereinafter referred to as “corrected articulation motion”) obtained from the articulation motion correction unit 230. The storage unit 207 is referred to when processing is executed in the feature analysis unit 210, the articulation feature extraction unit 220, and the articulation motion correction unit 230. The storage unit 206 is referred to when the word classification unit 204 searches for a word. The output unit 205 is provided to output a word obtained as a result of searching in the word classification unit 204.

  The flow of the speech recognition process based on the functional block of FIG. 2 will be described. The unknown speech input from the input unit 201 is discretized through the A / D conversion unit 202 and converted into a digital signal. The converted digital signal is output to the feature analysis unit 210.

  Details of the function of the feature analysis unit 210 will be described with reference to FIG. As shown in FIG. 3, the feature analysis unit 210 includes a Fourier transform unit 211 and a filter unit 212. In the feature analysis unit 210, the digital signal converted by the A / D conversion unit 202 is first subjected to Fourier analysis (using a Hamming window having a window width of 24 to 32 msec) in the Fourier transform unit 211. Next, in the filter unit 212, the noise component is removed by passing through a band-pass filter of about 24 channels. Thereby, a voice spectrum series and a voice power series at intervals of 5 to 10 msec are extracted. The obtained speech spectrum sequence and speech power sequence are output to the articulation feature extraction unit 220.

  Details of the function of the articulation feature extraction unit 220 will be described with reference to FIG. The articulation feature extraction unit 220 extracts motion features related to articulation. As shown in FIG. 4, the articulation feature extraction unit 220 includes a local feature extraction unit 221 and a discriminative phoneme feature extraction unit 222.

The speech spectrum series obtained from the feature analysis unit 210 is first input to the local feature extraction unit 221. In the local feature extraction unit 221, differential features in the time axis direction and the frequency axis direction are extracted by the time axis differential feature extraction unit 223 and the frequency axis differential feature extraction unit 224. Separately from this, the time axis differential feature of the audio power sequence is calculated. In extracting these differential features (hereinafter referred to as “local features”), linear regression calculation is used to suppress the influence of noise fluctuations and the like. When these differential features are extracted, linear regression operations given by the equations (2) and (3) are used to suppress the influence of noise fluctuations and the like.

Here, x (i, t) represents a speech spectrum sequence or a speech power sequence. i indicates a frequency channel (in the case of an audio power sequence, the relationship of i = 1 is established). t indicates time. Δ _t x (i, t) and Δ _f x (i, t) indicate a primary differential amount in the time direction and a primary differential amount in the frequency direction of x (i, t), respectively.

  K in a formula shows the position which performs linear regression calculation. δ is the width on one side. Specifically, in the case of local feature extraction, δ = 1 and linear regression calculation has three points, that is, three points of t = -1, 0, +1 centered on the time of interest in the time direction, and attention in the frequency direction. The linear regression coefficients are obtained from the three points i = -1, 0, +1 using the equations (2) and (3), respectively. An example of the local feature in the time direction (see FIG. 5) and the local feature in the frequency direction (see FIG. 6) calculated by the local feature extraction unit 221 is shown in FIGS. FIG. 5 and FIG. 6 show local features required for an utterance “artificial satellite”. The extracted local features are output to the discriminative phoneme feature extraction unit 222.

  In addition to the above-described local features, the input data of the discriminative phoneme feature extraction unit 222 is slightly inferior in performance, but the speech spectrum or a cepstrum obtained by orthogonalizing the speech spectrum (actually, the frequency axis is converted into a mel scale). May be used).

  Next, as shown in FIG. 4, the discriminative phoneme feature extraction unit 222 extracts the articulation feature series based on the local features extracted by the local feature extraction unit 221. The discriminative phoneme feature extraction unit 222 includes a two-stage neural network (a first multilayer neural network 225 and a second multilayer neural network 226).

  The neural network constituting the discriminative phoneme feature extraction unit 222 will be described in detail. As shown in FIG. 4, the neural network constituting the discriminative phoneme feature extraction unit 222 is composed of two stages, a first multilayer neural network 225 in the first stage and a second multilayer neural network 226 in the next stage. . The first multilayer neural network 225 extracts the articulation feature series from the correlation between the local features obtained from the speech spectrum series and the speech power series. In the second multilayer neural network 226, a meaningful subspace is extracted from the interdependence of the articulation feature series, and a highly accurate articulation feature series is obtained. An example of the articulation feature extraction result calculated by the discriminative phoneme feature extraction unit 222 is shown in FIG. FIG. 7 shows the articulation feature extraction result obtained for the utterance “artificial satellite” (jinkose).

The configuration of the neural network for obtaining the articulatory feature series can be realized by a one-stage configuration in addition to the two-stage configuration described in FIG. 4 if performance is sacrificed (see Non-Patent Document 5). Each neural network has a hierarchical structure, and has one to two hidden layers excluding an input layer and an output layer (multilayer neural network). Also, a so-called recurrent neural network having a structure that feeds back from the output layer or hidden layer to the input layer may be used. When compared in terms of performance for articulatory feature extraction, the results calculated in each neural network are not significantly different. These neural networks function as articulatory feature extractors through learning of weighting coefficients shown in Non-Patent Document 6 (see Non-Patent Document 6).
Masatoshi Sakawa, Masahiro Tanaka, Introduction to Neurocomputing, Morikita Publishing (1997, 1997) For information on multilayer neural networks, see pp. Chapter 13-48 Chapter 2 “Hierarchical Networks and Learning Mechanisms” describes how to calculate the weighting coefficient using the error back-propagation method. For recurrent neural networks, see pp. 83-96 Chapter 4 “Recurrent Neural Network” also describes the calculation method of the weighting factor.

  Learning by the neural network of the discriminative phoneme feature extraction unit 222 is performed by adding local feature data of speech to the input layer and providing the articulation feature of speech as a teacher signal to the output layer.

  On the other hand, the articulatory feature sequence itself is a signal commanded from the brain to the articulatory organ, and the articulatory feature sequence obtained from the speech is accompanied by a slack caused by the articulatory action in response to the command, that is, the muscle action of the speech organ. it is conceivable that. Therefore, in the present invention, the articulatory motion correcting unit 230 is introduced as a process for bringing the analog muscle motion result of the speech closer to an ideal articulatory sequence (binary discrete sequence).

  The articulatory motion correcting unit 230 will be described with reference to FIG. As shown in FIG. 8, the articulation motion correction unit 230 includes a speed / acceleration component extraction unit 231 and an articulation motion correction processing unit 232. The speed / acceleration component extraction unit 231 obtains the speed and acceleration from the articulation feature sequence (such as the discriminative phoneme feature sequence). The articulatory motion correction processing unit 232 corrects the articulatory motion (referred to as “articulatory motion”) represented by the articulatory feature series based on the speed and acceleration values obtained by the speed / acceleration component extracting unit 231. . The articulatory motion means articulatory motion displacement (displacement component, amplitude value of articulatory feature), articulatory motion speed (speed component, time differential value of articulatory feature), and articulatory motion acceleration (acceleration component, time differential value of articulatory motion speed, (Second order differential value of articulatory movement displacement).

  First, details of processing in the speed / acceleration component extraction unit 231 will be described with reference to FIG. When the velocity / acceleration component extraction unit 231 obtains the velocity component and the acceleration component from the displacement component of the articulation feature sequence, first, x (i, t) in the equation (2) is expressed as the articulation feature sequence DPF (m, t). replace. As a result, the velocity component series VDPF (m, t) is obtained. In the formula, “m” (= 1, 2,... M) indicates an articulation feature number indicating rupture property, high tongue property, etc., and “t” (= 1, 2,... T ) Indicates the time.

  Next, the velocity component series VDPF (m, t) obtained as described above is substituted into x (i, t) in the same expression (2). Thereby, an acceleration component series ADPF (m, t) is obtained. 8, V / ADPF (1) (15) 233 of the speed / acceleration component extraction unit 231 represents this calculation algorithm.

  Next, details of processing in the articulation motion correction processing unit 232 will be described with reference to FIG. The articulatory motion correction processing unit 232 uses the speed component and the acceleration component (VDPF (m, t), ADPF (m, t)) obtained by the speed / acceleration component extracting unit 231 to perform articulatory motion for each articulatory feature m. To correct. 8, MDPF (1)... (15) 234 of the articulation motion correction processing unit 232 indicates this correction algorithm.

  The specific processing content in the articulation motion correction unit 230 will be described with reference to the flowchart shown in FIG. In this process, “articulation movement is performed to realize articulation movement (lips close / front tongue rises / ...), and as a result, upward convex movement is observed. It is based on the assumption that a downwardly convex motion is observed when it is finished.

  As shown in FIG. 9, in the articulation motion correction unit 230, first, the velocity / acceleration component extraction unit 231 calculates the acceleration component ADPF (m, t) from the articulation feature series DPF (m, t) (S11). Next, it is determined whether the calculated acceleration component ADPF (m, t) is positive, negative, or zero (S13, S15). Then, the articulatory movement is corrected according to the determination result (S17, S19, S21). When the acceleration component ADPF (m, t) is negative, the movement trajectory of the articulatory feature series shows a peak, indicating that it is in the middle of moving away after approaching the maximum value (this time point is called an articulation point). means. If it is positive, the motion trajectory of the articulation feature series is in a descending state, i.e., the articulation operation has been completed, or preparation for the next articulation operation has been completed, and the articulation operation has been completed and has moved away from the articulation point. It means the movement that goes.

As shown in FIG. 9, when the ADPF (m, t) value is positive (S13: YES), the acceleration component ADPF (m, t) is substituted into the equation (4) to suppress the articulation operation. The As a result, the suppression enhancement function f (m, t) is obtained (S17). Expression (4) realizes suppression using a sigmoid function that is often used in neural networks. Then, the process proceeds to S23.

On the other hand, when the value of ADPF (m, t) is zero (S13: NO, S15: YES), the acceleration component ADPF (m, t) is not corrected. As a result, 1 is substituted into the suppression enhancement function f (m, t) (see equation (5)) (S19). Then, the process proceeds to S23.

On the other hand, if the value of ADPF (m, t) is negative (S15: NO), acceleration component ADPF (m, t) is substituted into equation (6) to emphasize the articulation operation. As a result, the suppression enhancement function f (m, t) is obtained (S21). The expression (6), like the expression (4), realizes enhancement using a sigmoid function that is often used in a neural network. Then, the process proceeds to S23.

  Next, in S23, the articulation feature series DPF (m, t) is multiplied by the calculated suppression enhancement function f (m, t) (S23). Thereby, articulation movement is corrected. Then, the process ends.

  Thus, in the flowchart shown in FIG. 9, the suppression enhancement function f (m, t) is calculated using the sigmoid function. Then, by multiplying the calculated articulation feature series DPF (m, t) by the calculated value, the articulation motion is corrected to obtain a corrected articulation motion DPF ′.

  Note that the articulation motion correction processing of the articulation motion correction unit 230 described with reference to FIGS. 8 and 9 is not limited to the present embodiment, and can be realized by other methods. With reference to FIG.10 and FIG.11, the modification of a different articulation movement correction part is demonstrated. FIG. 10 is a block diagram illustrating details of functions of the articulation motion correction unit 330. FIG. 11 is a block diagram illustrating details of functions of the articulation motion correction unit 430.

  First, the configuration of the articulatory motion correction unit 330 using a neural network will be described with reference to FIG. In the articulation motion correction unit 330 shown in FIG. 10, the articulation motion correction processing unit 332 is configured by a neural network NDPF (1) (15) 334 provided for each articulation feature. In the articulatory motion correcting unit 330, first, in the speed / acceleration component extracting unit 331, the speed component series VFPF (m, t) and the acceleration component series ADPF (m, t) are generated from the articulatory feature series DPF (m, t). (V / ADPF (1)... (15) 333 of the speed / acceleration component extraction unit 331 in FIG. 10 represents this calculation algorithm).

  The articulatory feature series DPF (m, t), the calculated velocity component series VFPF (m, t), and the acceleration component series ADPF (m, t) are combined into a neural network NDPF (1) of the articulatory motion correction processing unit 332. ) (15) is input to 334. Then, in NDPF (1) (15) 334, the articulatory motion is corrected, and the corrected articulatory motion is output.

  Next, the configuration of the articulatory motion correction unit 430 using an integrated neural network will be described with reference to FIG. In the articulatory motion correction unit 430 shown in FIG. 11, the articulatory motion correction processing unit 432 is an integrated neural network in which constraints between articulation features are inserted instead of the neural network independent for each articulation feature shown in FIG. 10. The network NDPF 434 is configured. The processing in the speed / acceleration component extraction unit 431 and the data output to the articulation motion correction processing unit 432 are the same as those in FIG.

  As shown in FIG. 4, the corrected articulatory motion corrected in the articulatory motion correcting unit 230 (the same applies to the articulatory motion correcting unit 330 and the articulatory motion correcting unit 330) is stored in the word classification dictionary 204 by the word pronunciation dictionary 2061, the HMM 2062, and The language model 2063 is referred to, and the spoken word is specified. Then, the identified word is output from the output unit 205. The calculation process in word classification is the same as the conventional method described in the background art. That is, the word k (or phoneme k) is obtained by substituting the articulatory feature (DPF (m, t)) into the input speech feature x (i, t) in the equation (1) (speech spectrum or MFCC in the conventional method). The acoustic likelihood of is obtained.

  As described above, in the articulatory feature extraction apparatus of the present invention, the articulatory feature sequence extraction process (articulation feature extraction unit 220) and the resulting articulatory feature sequence are modified to approximate the original articulatory operation. Processing (articulation motion correction unit 230) is provided. As a result, it is possible to extract the characteristics of the articulatory movement associated with the speech utterance with high accuracy, so that the speech recognition with higher accuracy can be achieved compared to the conventional speech recognition device that recognizes the speech using the speech spectrum. Can be done.

  In the conventional speech recognition characterized by the spectrum of speech, the spectrum greatly fluctuates depending on the speaker, the context at the time of speech, ambient noise, etc., so the hidden Markov model (HMM) used when obtaining the acoustic likelihood ) Required a lot of audio data. Further, the number of HMMs to be mixed is required to be 10 or more, and the cost has been increased in order to obtain a high-performance speech recognition apparatus. On the other hand, the articulatory feature extraction apparatus of the present invention can extract articulatory features in speech with high accuracy, so that only a few HMMs are mixed. Also, high-precision extraction of articulatory features can dramatically improve phoneme recognition performance, and can respond to the unknown word problem in the same way as humans do. Therefore, it is possible to smoothly advance the dialogue by synthesizing the confirmation utterance sentence using the phoneme sequence.

  In many cases, the articulation feature corresponds to the text (reading converted into a kana series) one-to-one, so that the search for the voice document and the text document can be mutually searched from both the voice and the text (keyboard). It becomes possible.

  In addition, the articulatory motion is corrected to the correct articulatory motion based on the displacement component, velocity component, and acceleration component. Can be extracted with high accuracy.

  In addition, the corrected articulatory motion is corrected based on the motion trajectory pattern (concave pattern, convex pattern). Therefore, even if the phoneme is different from the single tone state due to the articulatory connection, it is accompanied by the voice utterance. It is possible to extract the characteristics of the articulation operation with high accuracy.

  Further, since the process for correcting the articulatory motion is executed via a neural network, the corrected articulatory motion can be obtained at high speed.

  The input device 12 of FIG. 1 corresponds to the “voice acquisition unit” of the present invention, and the central processing unit 11 that performs the processing of the articulation feature extraction unit 220 of FIG. 2 is the “articulation feature extraction unit” of the present invention. The central processing unit 11 that performs the processing of the articulation motion correction unit 230 corresponds to the “articulation motion correction means” of the present invention, and the external storage device 15 in FIG. 1 serves as the “storage means” of the present invention. Correspondingly, the central processing unit 11 that performs processing for storing corrected articulation motion data in the storage means corresponds to the “storage control means” of the present invention.

Further, the central processing unit 11 that performs the process of extracting the speed component and the acceleration component in the speed / acceleration component extraction unit 231 in FIG. 8 corresponds to the “component extraction unit” of the present invention, and in S13 and S15 in FIG. The central processing unit 11 that performs processing corresponds to the “pattern recognition means” of the present invention.
<Experimental example>

  Hereinafter, an experimental example using the above-described articulatory feature extraction apparatus will be described with reference to the drawings. First, with reference to FIGS. 12 and 13, an example of extracting articulation features of speech speech before and after articulation motion correction will be described. FIG. 12 shows an example of articulation feature extraction before articulation motion correction. FIG. 13 shows an example of articulation feature extraction after articulation motion correction. In this embodiment, the discriminative phoneme feature is used as the articulation feature. However, the effect can be obtained by using another articulation feature display (for example, using the articulation feature in the table of international phonetic symbols (IPA)). It is assumed that it will be obtained.

  With reference to FIG. 12, an example of articulation feature extraction before articulation motion correction will be described. FIG. 12 shows an example of articulation feature extraction for the utterance “artificial satellite”. In this example, the input of the neural network (first multilayer neural network 225, second multilayer neural network 226) in the discriminative phoneme feature extraction unit 222 (see FIG. 4) is t− Data extending over three frames of the local feature of the third frame and the local feature of the t + 3 frame is added. At the same time, the output of the neural network in the discriminative phoneme feature extraction unit 222 (see FIG. 4) is also the articulation feature series (DPF (m, t-3), DPF) corresponding to the time (t-3, t, t + 3). (M, t), DPF (m, t + 3)) ((m: number of articulation feature, m = 1, 2,..., 15)) and a format for obtaining articulation feature series including contexts before and after. Adopted. FIG. 12 shows the transition of the articulation feature for the central articulation feature series DPF (m, t).

  In FIG. 12, the distinguishing features are shown as vertical columns, and individual phonemes are shown as horizontal columns. In addition, the portion indicated as “silB” (silence of beginning part) in the top column indicates that there is a silent interval, and the portion indicated as “jinkose” indicates the utterance interval of each voice. Is shown. Also, the dotted line in FIG. 12 indicates an ideal correct articulation feature, and the articulation feature (the state extracted by the discriminative phoneme feature extraction unit 222 (see FIG. 4)) in which the solid line is actually calculated is shown. Show.

  As shown in FIG. 12, when comparing the solid line (calculated data) and the dotted line (ideal data), there is a large gap between the dotted line and the solid line in the silent section and the utterance section, and the transition of the solid line Large fluctuations are confirmed.

  Next, an example of articulation feature extraction after articulation motion correction will be described with reference to FIG. FIG. 13 shows an example of extracting the discrimination liquid phoneme feature for the utterance “artificial satellite”, as in FIG. 12. In this example, the articulation motion correction unit 330 using the neural network learned for each articulation feature described in FIG. 10 is used as the articulation motion correction unit. Also, 7-frame DPF (m, t), VPPF (m, t), and ADPF (m, t) are used as inputs to the neural network NDPF (1) (15) 334. In general, if the number of frames used is small, the effect is small, and if it is too large, it tends to be too smooth. Therefore, it is desirable to use a value between 3 and 9 depending on the articulation characteristics (such as plosives) Is shorter, but longer for vowels etc.)

  As shown in FIG. 13, when comparing the solid line (calculated data) and the dotted line (ideal data), it was found that the values of both coincide well, and the calculated result is very close to the actual utterance. Moreover, it turned out that the transition of a continuous line is smooth compared with the result of FIG. Furthermore, it was found that noise generated in the silent section was also suppressed. It was also confirmed that DPF (m, t) showed a value in line with the actual utterance. As a result, it was found that the articulatory feature sequence was greatly improved by the articulatory motion correction.

  Next, the result of calculating the articulation feature extraction rate will be described with reference to FIG. FIG. 14 is a grab showing calculation results of the articulation feature accuracy rate. The articulation feature accuracy rate is based on the articulation features obtained when the Japanese newspaper reading corpus (about 100 male voice data) is used and whether or not the articulatory motion correction processing is changed and the conditions of the articulatory motion correction processing are changed. It was obtained by calculating the correct answer rate when speech recognition was performed.

  In the case of performing an evaluation when the condition of articulatory motion correction processing is changed, method 1: simple correction processing, method 2: neural network processing for each articulation feature (see FIG. 8), method 3: integrated neural network processing ( The articulation motion correction process was performed under a total of three types of conditions (see FIG. 11). And it performed by calculating the correct answer rate in case speech recognition was performed based on the obtained articulation feature.

  As shown in FIG. 14, when the articulation motion is not corrected (“No correction” in the figure), the extraction performance is less than 90%. On the other hand, it was found that when the articulation movement correction process was performed (“method 1”, “method 2”, and “method 3” in the figure), the accuracy rate was greatly improved as compared to the case where correction was not performed.

  Further, as shown in FIG. 14, it was found that the accuracy rate improved in the order of method 1 (92%), method 2 (93%), and method 3 (94%). However, since the amount of calculation required for the correction processing of articulation motion increases in the order of method 1, method 2, and method 3, it is desirable to select and use the method according to the purpose.

Next, the relationship between the number of mixed HMMs (phoneme recognizers) and the recognition accuracy required when using the above-mentioned articulatory feature extraction apparatus will be described with reference to Table 1. Table 1 shows the relationship between the number of HMM mixtures and the recognition rate at the time of articulation feature extraction. Table 1 shows a result of comparing the case where the MFCC is directly input and the case where the articulatory feature is input to the phoneme recognizer based on the HMM. In Table 1, the articulation feature (uncorrected) indicates the number of mixtures when the discriminative phoneme feature (DPF) is extracted by the neural network, and the articulation feature (corrected) is the modification of the articulation operation according to the present invention. The number of mixtures when adding is shown.

  From the results in Table 1, it was found that a large number of mixtures was required to improve recognition accuracy when the conventional MFCC was directly input. On the other hand, when the articulation feature is input, it was found that a relatively high performance can be obtained even when the number of mixing is one. Furthermore, it was found that a higher performance can be obtained when an articulation feature with a modified articulation operation is input. As a result, it became clear that the scale of the phoneme classifier and the word classifier (here, the number of mixtures) can be reduced to a small scale.

  In the embodiment, a method for correcting the articulatory motion by using a neural network as a means for correcting the articulatory motion and passing the articulatory feature (displacement) component, the velocity component, and the acceleration component through this is shown. The invention is not limited to this. For example, in this application, HMM is used for phoneme and word recognition, but instead of the neural network as articulation feature correction means, statistical pattern classification means such as HMM can be introduced and used as articulation feature correction means. is there. In this case, HMM or the like is not used as a model for phonemes or words but as a model for articulatory features. In short, for the articulatory motion realized by the “means for extracting articulatory features”, a “means for correcting” this is provided, and the articulatory feature (displacement) component, velocity component, and acceleration for expressing the articulatory motion are provided. The key is to realize the correction by passing the component through the articulatory movement correcting means.

2 is a schematic diagram illustrating an electrical configuration of the articulatory feature extraction apparatus 1. FIG. It is a functional block diagram which shows the articulation feature extraction process performed with the articulation feature extraction apparatus. 4 is a block diagram illustrating details of functions of a feature analysis unit 210. FIG. 3 is a block diagram illustrating details of functions of an articulatory feature extraction unit 220. FIG. The feature analysis unit obtained from the local feature extraction unit 221 is an example of local features in the time direction. It is an example of the local feature of the frequency direction obtained from the local feature extraction part 221. It is an example of the articulation feature obtained by the discriminative phoneme feature extraction unit 222. 4 is a block diagram showing details of functions of an articulatory motion correcting unit 230. FIG. 5 is a flowchart showing processing in an articulation motion correction processing unit 232; It is a block diagram which shows the functional detail of the articulation movement correction part 330, It is a block diagram which shows the functional details of the articulation movement correction part 430. It is an example of articulation feature extraction before articulation movement correction. It is an example of extraction of the articulation feature after articulation movement correction. It is the grab which showed the calculation result of the articulation feature extraction rate. It is a functional block diagram which shows the speech recognition process in the conventional speech recognition apparatus. Shows distinctive phoneme features

Explanation of symbols

1 articulation feature extraction device 11 central processing unit 12 input device 13 output device 14 storage device 15 external storage device 201 input unit 202 conversion unit 204 word search unit 205 output unit 206 storage unit 207 storage unit 210 feature analysis unit 212 filter unit 220 Articulation feature extraction unit 230 Articulation motion correction unit 330 Articulation motion correction unit 430 Articulation motion correction unit

Claims (9)

Audio acquisition means for acquiring audio;
Articulation feature extraction means for extracting the articulation characteristics of the voice acquired by the voice acquisition means;
The articulatory feature sequence, which is time series data of the articulatory feature extracted by the articulatory feature extraction means, is converted into a motion trajectory, and based on the motion trajectory, the effect of articulation coupling included in the speech is eliminated. Articulatory motion correcting means for correcting articulatory motion, which is a motion of articulation represented by the articulatory feature series, so that it can be recognized
An articulatory feature extraction apparatus comprising: a storage control unit that stores a modified articulation motion, which is the articulation motion corrected by the articulation motion correction unit, in a storage unit. Component extraction means for extracting a velocity component and an acceleration component from the displacement component of the articulation feature series,
The articulatory movement correcting means includes:
2. The articulatory motion is corrected to the corrected articulatory motion based on at least one of the displacement component, the velocity component and the acceleration component extracted by the component extraction unit. Articulatory feature extraction device. Based on at least one of the displacement component, the velocity component, and the acceleration component, when the displacement component is observed along the time axis, it is recognized whether the transition is a concave pattern or a convex pattern. With pattern recognition means,
The articulatory movement correcting means includes:
The articulatory feature extraction device according to claim 2, wherein the articulatory motion is corrected to the corrected articulatory motion based on the pattern recognized by the pattern recognition means. The articulatory movement correcting means includes:
The articulation according to claim 2 or 3, wherein the articulatory movement is corrected to the corrected articulatory movement by passing at least one of the displacement component, the velocity component, and the acceleration component through a neural network. Feature extraction device. An audio acquisition step for acquiring audio;
Articulation feature extraction step of extracting the articulation feature of the voice acquired in the voice acquisition step;
The articulation feature sequence, which is time series data of the articulation feature extracted in the articulation feature extraction step, is converted into a motion trajectory, and based on the motion trajectory, the effect of articulation coupling included in the speech is eliminated. An articulatory motion correcting step for correcting articulatory motion, which is an articulatory motion represented by the articulatory feature series, so that it can be recognized;
An articulation feature extraction method comprising: a storage control step of storing in a storage means the corrected articulation motion which is the articulation motion corrected in the articulation motion correction step. A component extraction step of extracting a velocity component and an acceleration component from the displacement component of the articulation feature series;
The articulatory motion correcting step includes:
6. The articulatory motion is corrected to the corrected articulatory motion based on at least one of the displacement component, the velocity component and the acceleration component extracted in the component extraction step. Articulatory feature extraction method. Based on at least one of the displacement component, the velocity component, and the acceleration component, when the displacement component is observed along the time axis, it is recognized whether the transition is a concave pattern or a convex pattern. With a pattern recognition step,
The articulatory motion correcting step includes:
The articulatory feature extraction method according to claim 6, wherein the articulatory motion is corrected to the corrected articulatory motion based on the pattern recognized in the pattern recognition step. The articulatory motion correcting step includes:
The articulation according to claim 6 or 7, wherein the articulatory motion is corrected to the corrected articulatory motion by passing at least one of the displacement component, the velocity component, and the acceleration component through a neural network. Feature extraction method.   An articulation feature extraction program for driving a computer as each processing means of the articulation feature extraction device according to any one of claims 1 to 4.