CN1835076A - Speech evaluating method of integrally operating speech identification, phonetics knowledge and Chinese dialect analysis - Google Patents

Abstract

The invention discloses a voice evaluating method of synthetically applying voice recognition, phonetic knowledge and Chinese dialect analysis in a voice evaluating system, making related voice recognition data collection and model establishment, and making phonetic analysis and phonetic processing policy relative to language species, and finally relative to Chinese characteristics, analyzing characteristics of various Chinese dialects and those of Chinese syllable structure and length and making related processing on these characteristics, thus extremely improving system universality on the basis of universal evaluating algorithm and simultaneously improving system performance.

Description

The speech evaluating method that a kind of integrated use speech recognition, phonetics knowledge and Chinese dialects are analyzed

Technical field

The present invention relates to the speech evaluating field, thereby specifically be that integrated use audio recognition method, phonetics knowledge and Chinese dialects analysis knowledge effectively promote the speech evaluating system performance when speaker pronunciation level evaluated and tested using a computer.

Background technology

PSC is the important method of the work of popularizing Beijing pronunciation, and is to make the work of popularizing Beijing pronunciation progressively go on the important behave of scientific, standardization, institutionalization." People's Republic of China (PRC) country general purpose language literal method " regulation that the 18 meeting of Standing Committee of the National People's Congress on October 31 in 2000 passed through must be participated in PSC as announcer, host and video display actor, teacher, the state personnel of working language and reach the classification standard of national regulation with mandarin.

The mode of artificial scoring is all adopted in the PSC work of carrying out at present, 2-3 name test man need be arranged in general each mandarin test examination hall, by the test man examinee's one's voice in speech is marked, 30 people can only be tested in one day in an examination hall, a thousands of people's PSC usually needs to organize at short notice up to a hundred test mans, not only waste time and energy, the testing cost height, and also efficient is low.Because all adopt artificial mode to mark, test man's subjectivity is very strong, has the fairness problem of test result to a certain extent.

Therefore, in conjunction with development of modern computer technology, research and development how with advanced person's information technology application in PSC, replacing whole or part substitute artificial evaluation and test person, thereby remedy the deficiency of traditional PSC method, all be significant for the fairness, the efficient aspect that cost such as use manpower and material resources sparingly and improve test.

The systematic research of computing machine PSC itself is very high for the requirement of aspects such as basic resource storehouse, cooperation interdisciplinary, has bigger difficulty.Just because of the guidance and the broad co-operation that lack system, the mode of computing machine realizes that PSC never obtains researchist's concern for a long time.

Under the trust that the State Language Work Committee scientific research is done, Zhongkeda Xunfei Information Science ﹠. Technology Co., Ltd., Anhui Prov has born national language letter application " 15 " main project---and " application of intelligent sound technology in the mandarin assisted learning ".Collection, the mark work of main dialect corpus have been finished, the computing machine PSC of utilizing project achievement to carry out in part provinces and cities has reached higher precision, can compare evaluation and test accurately to the spoken looks of mandarin, testing level has reached provincial test man's level, can progressively promote the use of in PSC from now on.

Mostly be based at present the evaluating and learning system of computer-assisted language learning (CALL) in the world at the system of language learning, though the application aspect Chinese study and evaluation and test is also arranged, but this type of evaluating system, owing to Chinese is not handled targetedly, system's universality and performance have much room for improvement.If can be in evaluating system integrated use speech recognition general principle, phonetics knowledge at languages, and at the Chinese dialects analysis and the harmonious sounds architectural characteristic of the characteristics of Chinese, the universality and the performance that might realize the Chinese evaluating system are greatly improved, and satisfy the Chinese demand of applying of evaluation and test automatically.

Summary of the invention

Purpose of the present invention is exactly the speech evaluating method that a kind of integrated use speech recognition, phonetics knowledge and Chinese dialects are analyzed, under the framework of speech recognition system, introduce phonetics knowledge and Chinese dialects analysis characteristic, thereby promote evaluating system performance the evaluation and test of speaker pronunciation level.

Concrete implementation is as follows:

The speech evaluating method that a kind of integrated use speech recognition, phonetics knowledge and Chinese dialects are analyzed is characterized in that setting up the standard database at speech evaluating, comprises the Received Pronunciation of monosyllabic word, and the Received Pronunciation of continuous flow; Then, on this standard database basis, use HTK to build speech recognition device based on Hidden Markov Model (HMM); Simultaneously, carry out the phonetics analysis, obtain phonetics knowledge, form phonetics knowledge and phonetics knowledge processing policy at the languages that will evaluate and test; In addition,, the Chinese speech pronunciation characteristics are analyzed, obtained Chinese phonetic parent structure characteristics and duration characteristic and evaluation and test property relationship, form Chinese speech pronunciation characteristics and duration characteristic knowledge base and Chinese speech pronunciation characteristics and duration characteristic processing policy at the Chinese evaluation and test; At last, integrated voice recognizer and phonetics processing policy and Chinese speech pronunciation characteristics and duration characteristic processing policy, under the support of speech evaluating algorithm, output is by reviewer's evaluation and test score.

Speech recognition system to build step as follows:

(1) collects the trainable recognizer voice: according to the application need of speech evaluating, collect or record the pronunciation language material of standard targetedly in advance, and save as recognizer training utterance file, such as the PSC language material of just recording the standard mandarin speaker at the standard Chinese horizontal checkout;

(2) data mark: carry out pinyin marking at the standard corpus of collecting, make that the language material of collecting is pointed to speech evaluating;

(3) model training: utilize the speech recognition device model of HTK training based on Hidden Markov Model (HMM) according to the standard corpus of collecting;

(4) preserve: model is saved in the evaluating system storehouse;

Just can carry out speech evaluating after speech recognition device is built and finished, the evaluation and test algorithm is shown below:

$\mathrm{Score}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P\left(M_i\right|O)$

Wherein, the sound that the O enunciator sends, M _iBe the speech recognition modeling at the word of needs pronunciation, N is the total number that needs in the evaluation and test to pronounce, $P\left(M_i\right|O)=\frac{P\left(O\right|M_i)}{\underset{M_i\∈K_{M_i}}{\mathrm{\Σ}}P\left(O\right|M_j)}$ The sound that is speaker is to M _iThe posterior probability of model,

The utilization phonetics knowledge is evaluated and tested targetedly, the steps include:

(1) metalanguage characteristics, obtain the phonetics knowledge storehouse: before system building, need analyze, obtain the phonetics knowledge storehouse and effectively evaluate and test in order to instruct evaluating system at the languages pronunciation characteristic that will evaluate and test, these phonetics knowledges also can reference language and the mispronounce that provides of phonetics expert get characteristics, may be misread as C by mistake such as phoneme A, D, E, A → (A so, C, D E) is exactly a knowledge;

(2) integrating speech sound is gained knowledge and is entered evaluating system: the phonetics knowledge that obtains on the superincumbent analysis foundation, thereby can using for reference phonetics knowledge, the system that makes evaluates and tests, and processing policy is that modification system evaluation and test algorithm is as follows: $P\left(M\right|O)=\frac{P\left(O\right|M)}{\underset{M_i\∈K_M}{\mathrm{\Σ}}P\left(O\right|M_i)},$ K wherein _MThe phonetics knowledge that obtains above exactly, wherein P (M|O) is the evaluation and test score of a word M;

(3) preserve: phonetics knowledge storehouse that preservation obtains and phonetics knowledge processing policy are to the phonetics processing module storehouse of evaluating system;

Carry out unique processing at the Chinese speech pronunciation characteristics, the steps include:

(1) Chinese feature analysis: sound parent structure and duration characteristic at Chinese are analyzed, and obtain female duration of Chinese phonetic and evaluating system evaluation and test property relationship;

(2) Chinese feature is used: at female duration of Chinese phonetic and evaluation and test property relationship, introduced at Chinese phonetic parent structure and duration processing policy and handled, its processing policy is:

$\left\{\begin{array}{c}{\mathrm{Score}}_{\mathrm{Sent}}^{\′}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P{\left(M_i\right|O)}^{\′}\\ P{\left(M_i\right|O)}^{\′}=P\left(M_{\mathrm{initial}}^i\right|O)\×(1+\frac{{\mathrm{Dur}}_{\mathrm{final}}^i}{{\mathrm{Dur}}_{\mathrm{initial}}^i}\×\mathrm{COEF})+P\left(M_{\mathrm{final}}^i\right|O)\end{array}\right.$

Score ' wherein _SentBe the evaluation and test score after the improvement in short, P (M _Initial ⁱ| O) be the evaluation and test score of initial consonant, P (M _Final ⁱ| O) be the evaluation and test score of simple or compound vowel of a Chinese syllable, P (M _i| O) ' and be the evaluation and test score after the improvement of a syllable, Durf _Inal ⁱBe the duration of i syllable simple or compound vowel of a Chinese syllable, Dur _Initial ⁱIt is the duration of the initial consonant of i syllable; COEF is an adjustable factor, controls the weight of initial consonant;

(3) evaluation and test score output: handle in the above on the basis, the evaluation and test score that obtains revising is also exported score.

The speech evaluating algorithm is the core place of speech evaluating system, after processing policy on carried out improves, can promote the evaluation and test performance of evaluating system greatly, thereby make evaluating system can accomplish to improve rapidly the adaptation concrete condition simultaneously, promoted the universality of system greatly for different language and localism area.

The effect of invention

The universality of evaluating system strengthens.Owing to introduced the specific aim algorithm in the speech recognition system, simultaneously, to analyze at different language, integrating speech sound is gained knowledge and is entered evaluating system, thereby makes the universality of system strengthen, and can carry out the switching of different language rapidly.

Effective lifting of evaluation and test performance.Because at having been undertaken analyzing targetedly and handling by the harmonious sounds structure of each localism area of Chinese and Chinese and duration characteristic, the system that makes has very strong specific aim at each localism area of Chinese, thereby has greatly promoted the performance of evaluating system in the standard Chinese test.

Terminological interpretation

CALL: i.e. computer-assisted language learning (Computer Assisted Language Learning).

HMM: Hidden Markov Model (HMM) (Hidden Markov Model)

One of providing of HTK:Hidden Markov Toolkit Cambridge University is used for building the instrument based on the open source code of the speech recognition device of HMM.

Embodiment

The speech evaluating method that a kind of integrated use speech recognition, phonetics knowledge and Chinese dialects are analyzed may further comprise the steps:

Speech recognition device to build step as follows:

(1) collects the trainable recognizer voice: according to the application need of speech evaluating, collect or record the pronunciation language material of standard targetedly in advance, and save as recognizer training utterance file, such as the PSC language material of just recording the standard mandarin speaker at the standard Chinese horizontal checkout;

(2) data mark: mark at the standard corpus of collecting, make that the language material of collecting is pointed to speech evaluating;

(3) model training: according to the speech recognition device model of the standard corpus training evaluation and test usefulness of collecting.；

(4) preserve: evaluating system is saved in model in the evaluating system storehouse.

Just can evaluate and test after speech recognition device is built and finished, the evaluation and test algorithm is shown below:

$\mathrm{Score}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P\left(M_i\right|O)$

Wherein $P\left(M_i\right|O)=\frac{P\left(O\right|M_i)}{\underset{M_j\∈K_{M_i}}{\mathrm{\Σ}}P\left(O\right|M_j)}$ Be at word M _iPosterior probability.

The utilization phonetics knowledge is evaluated and tested targetedly, the steps include:

(1) metalanguage characteristics, obtain the phonetics knowledge storehouse: before system building, need analyze, obtain the phonetics knowledge storehouse and effectively evaluate and test in order to instruct evaluating system at the languages pronunciation characteristic that will evaluate and test, these phonetics knowledges also can reference language and the mispronounce that provides of phonetics expert get characteristics, may be misread as C by mistake such as phoneme A, D, E, A → (A so, C, D E) is exactly a knowledge;

(2) integrating speech sound is gained knowledge and is entered evaluating system: the phonetics knowledge that obtains on the superincumbent analysis foundation, under the processing of the phonetics processing module of system, introduce phonetics knowledge, evaluate and test thereby the system that makes can use for reference phonetics knowledge, the processing module strategy is:

$P\left(M\right|O)=\frac{P\left(O\right|M)}{\underset{M_i\∈K_M}{\mathrm{\Σ}}P\left(O\right|M_i)},$ K wherein _MThe phonetics knowledge that obtains above exactly, wherein P (M|O) is the score of a word M;

(3) preserve: phonetics knowledge storehouse that preservation obtains and phonetics knowledge processing policy are to the phonetics processing module storehouse of evaluating system.

Carry out unique processing at the Chinese speech pronunciation characteristics, the steps include:

(1) Chinese feature analysis: sound parent structure and duration characteristic at Chinese are analyzed, and obtain female duration of Chinese phonetic and evaluating system evaluation and test property relationship;

(2) Chinese feature is used: at female duration of Chinese phonetic and evaluation and test property relationship, introduced at Chinese phonetic parent structure and duration processing policy and handled, its processing policy is:

${\mathrm{Score}}_{\mathrm{Sent}}^{\′}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P{\left(M_i\right|O)}^{\′}$

$P{\left(M_i\right|O)}^{\′}=P\left(M_{\mathrm{initial}}^i\right|O)\×(1+\frac{{\mathrm{Dur}}_{\mathrm{final}}^i}{{\mathrm{Dur}}_{\mathrm{initial}}^i}\×\mathrm{COEF})+P\left(M_{\mathrm{final}}^i\right|O)$ Wherein

Score ' _SentBe the score after the improvement in short, P (M _i| O) ' and be the score after the improvement of a syllable, Dur _Final ⁱBe the duration of i syllable simple or compound vowel of a Chinese syllable, Dur _Initial ⁱIt is the duration of the initial consonant of i syllable.COEF is an adjustable factor, controls the weight of initial consonant;

(3) evaluation and test score output: handle in the above on the basis, the evaluation and test score that obtains revising is also exported score.

Claims (2)

1, the speech evaluating method that a kind of integrated use speech recognition, phonetics knowledge and Chinese dialects are analyzed is characterized in that setting up the standard database at speech evaluating, comprises the Received Pronunciation of monosyllabic word, and the Received Pronunciation of continuous flow; Then, on this standard database basis, use HTK to build speech recognition device based on Hidden Markov Model (HMM); Simultaneously, carry out the phonetics analysis, obtain phonetics knowledge, form phonetics knowledge and phonetics knowledge processing policy at the languages that will evaluate and test; In addition,, the Chinese speech pronunciation characteristics are analyzed, obtained Chinese phonetic parent structure characteristics and duration characteristic and evaluation and test property relationship, form Chinese speech pronunciation characteristics and duration characteristic knowledge base and Chinese speech pronunciation characteristics and duration characteristic processing policy at the Chinese evaluation and test; At last, integrated voice recognizer and phonetics processing policy and Chinese speech pronunciation characteristics and duration characteristic processing policy, under the support of speech evaluating algorithm, output is by reviewer's evaluation and test score.

2, method according to claim 1 is characterized in that concrete steps are:

Speech recognition system to build step as follows:

(1) collects the trainable recognizer voice: according to the application need of speech evaluating, collect or record the pronunciation language material of standard targetedly in advance, and save as recognizer training utterance file, such as the PSC language material of just recording the standard mandarin speaker at the standard Chinese horizontal checkout;

(2) data mark: carry out pinyin marking at the standard corpus of collecting, make that the language material of collecting is pointed to speech evaluating;

(3) model training: utilize the speech recognition device model of HTK training based on Hidden Markov Model (HMM) according to the standard corpus of collecting;

(4) preserve: model is saved in the evaluating system storehouse;

Just can carry out speech evaluating after speech recognition device is built and finished, the evaluation and test algorithm is shown below:

$\mathrm{Score}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P\left(M_i\right|O)$

Wherein, the sound that 0 enunciator sends, M _iBe the speech recognition modeling at the word of needs pronunciation, N is the total number that needs in the evaluation and test to pronounce, $P\left(M_i\right|O)=\frac{P\left(O\right|M_i)}{\underset{M_j\∈K_{M_i}}{\mathrm{\Σ}}P\left(O\right|M_j)}$ The sound that is speaker is to M _iThe posterior probability of model,

The utilization phonetics knowledge is evaluated and tested targetedly, the steps include:

(1) metalanguage characteristics, obtain the phonetics knowledge storehouse: before system building, need analyze, obtain the phonetics knowledge storehouse and effectively evaluate and test in order to instruct evaluating system at the languages pronunciation characteristic that will evaluate and test, these phonetics knowledges also can reference language and the mispronounce that provides of phonetics expert get characteristics, may be misread as C by mistake such as phoneme A, D, E, A → (A so, C, D E) is exactly a knowledge;

(2) integrating speech sound is gained knowledge and is entered evaluating system: the phonetics knowledge that obtains on the superincumbent analysis foundation, thereby can using for reference phonetics knowledge, the system that makes evaluates and tests, and processing policy is that modification system evaluation and test algorithm is as follows: $P\left(M\right|O)=\frac{P\left(O\right|M)}{\underset{M_i{\∈K}_M}{\mathrm{\Σ}}P\left(O\right|M_i)},$ K wherein _MThe phonetics knowledge that obtains above exactly, wherein P (M|O) is the evaluation and test score of a word M;

(3) preserve: phonetics knowledge storehouse that preservation obtains and phonetics knowledge processing policy are to the phonetics processing module storehouse of evaluating system;

Carry out unique processing at the Chinese speech pronunciation characteristics, the steps include:

(1) Chinese feature analysis: sound parent structure and duration characteristic at Chinese are analyzed, and obtain female duration of Chinese phonetic and evaluating system evaluation and test property relationship;

(2) Chinese feature is used: at female duration of Chinese phonetic and evaluation and test property relationship, introduced at Chinese phonetic parent structure and duration processing policy and handled, its processing policy is:

$\left\{\begin{array}{c}{\mathrm{Score}}_{\mathrm{Sent}}^{\′}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}P{\left(M_i\right|O)}^{\′}\\ P{\left(M_i\right|O)}^{\′}=P\left(M_{\mathrm{initial}}^i\right|O)\×(1+\frac{{\mathrm{Dur}}_{\mathrm{final}}^i}{{\mathrm{Dur}}_{\mathrm{initial}}^i}\×\mathrm{COEF})+P\left(M_{\mathrm{final}}^i\right|O)\end{array}\right.$

ScoreS wherein _Ent' be the evaluation and test score after the improvement in short, P (M _Initial ⁱ| O) be the evaluation and test score of initial consonant, P (M _Final ⁱ| O) be the evaluation and test score of simple or compound vowel of a Chinese syllable, P (M _i| O) ' and be the evaluation and test score after the improvement of a syllable, Dur _Final ⁱBe the duration of i syllable simple or compound vowel of a Chinese syllable, Dur _Initial ⁱIt is the duration of the initial consonant of i syllable; COEF is an adjustable factor, controls the weight of initial consonant;

(3) evaluation and test score output: handle on the basis evaluation and test score that obtains revising and output in the above.