CN101334704B - Multichannel Chinese input method facing to mobile equipment - Google Patents

Abstract

The invention relates to a multi-passage Chinese character input method facing movable equipment, belonging to the field of human-machine interaction. The invention provides a Chinese character input method on the movable equipment and inputs the Chinese characters by using the speech and a keyboard simultaneously. The keyboard input adopts a simple Pinyin method or a simplified stroke input method; the prior art can be adopted; the keyboard input is converted into Pinyin indicating; during the keyboard input, the words to be input are spoken out by speech; the speech recognition result is also converted into the Pinyin; a plurality of input candidate sorting is generated for the keyboard input; subsequently, the keyboard is used for selecting the final input from the candidate list after the resorting, and the input efficiency is improved by the synchronization and the multi-channel combination of the speech and the keyboard input.

Description

A kind of hyperchannel Chinese character coding input method towards mobile device

Technical field

The invention belongs to field of human-computer interaction, be specifically related to a kind of hyperchannel Chinese character coding input method towards mobile device.

Background technology

Mobile device, as mobile phone etc., input mainly depends on keyboard.On these equipment, the user need import URL and query contents, input command and write mail and message etc., and the text input is a difficult problem to be solved such as.Chinese is as a kind of ideographic language, and its field of human-computer interaction that inputs to has been brought new challenge.Chinese has very big difference with English.The elementary cell of Chinese is a character, character is the mixture (list of references: Lin of sound, structure and implication, M., and Sears, A.Graphics matter:a case study of mobile phone keypad design for Chinese input.Ext.Abstracts2005.ACM Press (2005), 1593-1596).On the whole, Chinese character input method can be divided into based on the pronunciation, based on structure and based on other Methods for Coding.At present, the main input method on the mobile device is based on the method for pronunciation and based on the method for stroke, " T9 ".The user can select the character that will import then with the stroke of keyboard input Pinyin or Chinese character from candidate list.Yet the button on the mobile device is often smaller, and 3 to 4 Latin alphabets of each key correspondence, and therefore importing several Chinese characters all needs a lot of touch potentials.

Phonetic entry and pen input are the methods that may replace keyboard on the mobile device.Although voice and pen be nature and directly perceived more, they are based on the method for identification.Identification error is inevitable often, is subject to the computing power of mobile device, and this problem becomes more serious.Studies show that, and hyperchannel fusion can the minimizing effectively identification error and raising input efficiency (list of references: Oviatt, S.Ten Myths of Multimodal Interaction, Communications of the ACM, 42,9 (1999), 74-81).Phonetic entry and keyboard input carrying out hyperchannel are merged, will improve the input in Chinese efficient on the mobile device and have lower error rate.

The input of a few thing research hyperchannel has been arranged at present.Studies show that, improve robustness thereby the system that merges multiple complementary information can reduce the uncertainty of system identification.Bo has studied on the mobile device voice and keyboard combination has been used for the example (list of references: Hsu that text is imported, B.J., Mahajan, M.and Acero, A.Multimodal Text Entry on MobileDevices.IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 2005).But system with voice as main input channel, with keyboard as confirming or the error correction passage.Speech Dasher voice and gesture input text, gesture is used to confirm or error correction (list of references: Vertanen, K.Efficient Computer InterfacesUsing Continuous Gestures, Language Models, and Speech.M.Phil Thesis, University ofCambridge, 2004).Speech pen is by the content of the hyperchannel identification prediction Japanese handwriting input around utilizing, thereby reduce the burden (list of references: Kurihara of Japanese handwriting input, K., Goto, M., Ogata, J.and Igarashi, T.Speech Pen:Predictive Handwriting based on Ambient Multimodal Recognition.In Proc.CHI2006.ACM Press (2006), 851-860).Kaiser utilizes handwriting input that is associated and the phonetic entry that occurs simultaneously, can discern not speech (list of references: Kaiser at dictionary (out-of-vocabulary), E.C.Using redundant speechand handwriting for learning new vocabulary and understanding abbreviations.In Proc ICMI2006.ACM Press (2006), 347-356).Ao and Wang correct mistake (list of references: Ao in the hand-written Chinese identification with voice, X., Wang, X.G., Tian, F., Dai, G.Z.and Wang, H.A.Crossmodal error dorrection ofcontinuous handwriting recognition by speech.In Proc.IUI 2007.ACM Press (2007), 243-250).

The user of mobile device can be skilled the use mobile device on spelling input method or stroke input method, do not need extra study.Therefore, will combining with voice-based fuzzy input method based on the accurate input method of keyboard, is a promising method that solves input in Chinese problem on the mobile device.

Summary of the invention

The object of the present invention is to provide the Chinese character input method on a kind of mobile device, use voice and keyboard input Chinese simultaneously, improve the efficient of input.

The stroke input method of simplicity method or simplification is adopted in the keyboard input.First letter of the phonetic of each word in the word only need be imported in word of simplicity method input, and the stroke input method of simplification then needs the user to import first stroke of each word in the word that will import.They can adopt existing technology.The keyboard input is converted into pinyin representation.The present invention says the word that will import with voice in the keyboard input.Voice identification result is converted into phonetic equally, a large amount of input candidate orderings that input produces to keyboard.Then, use the final input of selection the candidate list of keyboard after reordering.It is critical support technology of the present invention that the synchronous and hyperchannel of voice and keyboard input merges.

Input method of the present invention is applicable to the mobile device of various support voice inputs and keyboard input, as mobile phone, and PDA etc.The input range of supporting is a normal occurring words in the Chinese.Individual character can be seen as monosyllabic word.

Chinese character input method (as shown in Figure 2) based on voice and keyboard the steps include:

1) mobile device receives phonetic entry and keyboard input;

2) simultaneous voice and keyboard input;

3) recognizing voice input obtains the phonetic entry candidate, obtains keyboard input candidate according to button, and all are imported candidate mappings to pinyin representation;

4) adopt the hyperchannel fusion method that the candidate of voice or keyboard input is merged the candidate list after obtaining reordering;

5) candidate list is shone upon back Chinese character and represent, from candidate list, select final input

Described simultaneous voice input is used to obtain effective input with the keyboard input, and its process is: obtain the time of button and the time of voice; Calculate the effective voice time.

The described efficient voice time is max{T (PreConfirmation), the time between T (FirstKeyPress)-Interval} and the T (Confirmation).T (Confirmation) is the affirmation moment of the Chinese word of current input, and T (PreConfirmation) is the affirmation moment of previous Chinese word.T (FirstKeyPress) is the moment that is used to import first button of current word.Interval is a constant, effective input time of restriction voice.

Described input candidate is that phonetic entry and keyboard input are converted into pinyin representation to the mapping of phonetic.Wherein, keyboard is input to the mapping concrete steps of phonetic and is:

Concerning simplicity input, the numeral of button correspondence is mapped to the Latin alphabet, obtain all corresponding simplicity of keyboard input and represent, and obtain all words of simplicity correspondence; Concerning the stroke input of simplifying, obtain all words of button correspondence; Thereby obtain the phonetic of all word correspondences.

Described hyperchannel fusion method utilizes phonetic entry that keyboard input candidate is sorted, thereby makes the most approaching actual Chinese that will import be positioned at the front of input candidate list.Its concrete steps are:

1) calculates a certain voice candidate s _jWith keyboard candidate k _iSimilarity w (k _i, s _j), similarity and s _jAnd k _iBetween editing distance (Vladimir Levenshtein 1965) relevant;

2) calculate given phonetic entry s _jThe time, select keyboard input k _iProbability p (k for final input _i| s _j);

3) calculate selection keyboard input k _iProbability p (k for final input _i);

4) press p (k _i) order of descending is to keyboard input candidate ordering, the candidate list after obtaining reordering uses keyboard to finish final input results by the user.

Advantage of the present invention and good effect are as follows:

1. the hyperchannel input method based on voice and keyboard can improve the efficient of input in the naturality that keeps input.

2. the combination of voice and keyboard is a kind of input method of novelty, in conjunction with both advantages, can improve the discrimination of voice

3. adopt the stroke input of simplicity or simplification to combine, be easy to study and use with phonetic entry

Description of drawings

The synchronous schematic diagram of phonetic entry and keyboard input among Fig. 1 the present invention;

Among Fig. 2 the present invention based on the process flow diagram of the Chinese character input method of voice and keyboard;

The application example process flow diagram of Fig. 3 simplicity and voice combination;

The stroke input that Fig. 4 simplifies and the application example process flow diagram of voice combination;

The process flow diagram of hyperchannel fusion method among Fig. 5 the present invention;

Fig. 6 calculates the process flow diagram of keyboard input candidate and phonetic entry candidate's similarity;

Fig. 7 calculates given phonetic entry and selects the process flow diagram of a certain keyboard input candidate as the probability of final input;

Fig. 8 calculates the process flow diagram of a certain keyboard input candidate as the probability of final input;

Transformational relation between Fig. 9 phonetic, stroke, Chinese character, the simplicity.

Embodiment

In order to make those skilled in the art person better understand the present invention, the present invention is described in further detail below in conjunction with drawings and embodiments.

Provide an example below, use method of the present invention to import the process of Chinese word " network ".With simple phoneticizing (lu's Simple Phoneticizing) input method and voice process, as shown in Figure 3 in conjunction with the Chinese word of input " network ".Step is as follows:

1) on the mobile device, says " network " with voice in the time of by numerical key 9 and 5;

2) from " network, net for catching fish or birds, sieve Wang ", select " network " as final input.

On mobile device, the Latin alphabet of 9 correspondences is " WXYZ ", and the Latin alphabet of 5 correspondences is " JKL ", and all corresponding simplicity of " 95 " be " WJ, WK, WL; XJ, XK, XL, YJ, YK; YL, ZJ, ZK, ZL ", all phonetics that can represent with simplicity of each simplicity correspondence.Wherein, the phonetic that " WL " is corresponding is " wanliu, wangluo, weilai, weile, weili, wulun, wuli, wuliao " etc., and " WL " is the simplicity of " network "; Preceding two identification candidates of voice are " huangluo " and " wangluo ".Candidate pinyin after the fusion is " wangluo ".Phonetic is mapped to Chinese character and is " network, net for catching fish or birds, sieve Wang ", can carry out input in Chinese.

With the stroke input method of simplifying and voice process, as shown in Figure 4 in conjunction with the Chinese word of input " network ".Step is as follows:

1) on the mobile device, says " network " with voice when pressing numerical key 2 and 5;

2) select " network " as input.

Wherein, first stroke " | " of 2 correspondences " net ", first stroke of 5 correspondences " network ".

The specific embodiment of the present invention is as follows:

1. mobile device reads keyboard input information and speech input information;

2. at first synchronous to phonetic entry and keyboard input, as shown in Figure 1.The effective time of voice is for being in max{T (PreConfirmation), the time between T (FirstKeyPress)-Interval} and the T (Confirmation).

Be T (FirstKeyPress)-Interval efficient voice zero hour of first speech among Fig. 1, and efficient voice finish time is the moment that acknowledgement key is pressed, and the value of Interval is 1s.The efficient voice time of second speech is max{T (PreConfirmation), time between T (FirstKeyPress)-Interval} and the T (Confirmation), T (PreConfirmation) presses constantly for the acknowledgement key of first speech, T (FirstKeyPress) is input the pressing constantly of first key during second speech, and T (Confirmation) is the moment that the acknowledgement key of second speech is pressed.Discern the voice in effective time, obtain the candidate collection of speech recognition; Simultaneously, obtain the candidate collection of keyboard input, the candidate collection of the candidate collection of speech recognition and keyboard input all is kept on the mobile device.The candidate is converted into pinyin representation.The speech recognition candidate collection of pinyin representation is S, and keyboard input candidate collection is K.

3. speech recognition candidate collection and keyboard input candidate collection are mapped to pinyin representation, as shown in Figure 9.

Concerning the simplicity input, the keyboard candidate obtains Chinese-character words according to simplicity (phonetic) word corresponding relation dictionary.According to Chinese character and phonetic corresponding relation dictionary word is converted to pinyin representation afterwards.Concerning the stroke input of simplifying, the keyboard candidate obtains Chinese-character words according to stroke and Chinese character corresponding relation, is converted to pinyin representation afterwards.

Described simplicity and word corresponding relation dictionary adopt the simplicity commonly used in the input method and the bilingual dictionary of word, phonetic and Chinese character.

The voice candidate is mapped to voice candidate's pinyin representation then according to the return type of voice identification result.

4. the hyperchannel fusion method merges speech recognition candidate collection S and carries out the hyperchannel fusion with keyboard candidate collection K.The step that hyperchannel merges as shown in Figure 5, its step is as follows:

Suppose that K contains M input candidate, S contains N input candidate, k _iBe i candidate of keyboard input, s _jBe j candidate of phonetic entry;

1) calculates k _iWith s _jSimilarity w (k _i, s _j), as shown in Figure 6, its detailed step is as follows:

Calculate k _iWith s _jEditing distance edit_distance (k _i, s _j);

Editing distance and constant be a's and be edit_distance (k _i, s _j)+a;

Calculate $\frac{1}{\mathrm{edit}\_\mathrm{dis}\tan \mathrm{ce}(k_i,s_j)+a},$ Obtain k _iWith s _jSimilarity w (k _i, s _j), wherein, the denominator value was not 0 when constant a guaranteed similarity calculating, and provided editing distance shared proportion in similarity is calculated.The value of a is greater than 0, and present embodiment is set to 1.

2) calculate given voice s _jThe time, select keyboard input k _iProbable value p (k as output _i| s _j); The given s of described calculating _jSelect k _iProbability p (k _i| s _j), as shown in Figure 7, its detailed step is as follows:

All w (k when calculating i gets 1 to M _i, s _j) and

Be expressed as normalized factor Z ' (j); Calculate w (k _i, s _j) and Z ' ratio (j)

(phonetic entry is identified as s to read the confidence level of speech recognition from speech recognition device (can adopt the speech SDK5.1 of Microsoft) _jProbability be p (s _j)), calculate p (s _j) with

Product $p\left(s_j\right)\×\frac{w(k_i,s_j)}{Z^{\′}\left(j\right)},$ Obtain given s _jSelect k _iProbability p (k _i| s _j).

3) calculate selection k _iProbability p (k as input _i), as shown in Figure 8, from high to low the input candidate among the K is sorted according to input probability; Its detailed step is as follows:

Calculation constant b and k _iFrequency freq (k _i) product b * freq (k _i), b provides frequency and is calculating p (k _i) time proportion, its qualification is set between 0 to 1;

P (k when calculating j gets 1 to N _i| s _j) and

With b * freq (k _i) and $b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j);$

When calculating i gets 1 to M $b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j)$ And, have $\underset{i=1}{\overset{M}{\mathrm{\Σ}}}(b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j))=Z^{\′\′};$

Calculate $b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j)$ With Z " ratio, obtain selecting k _iProbability p (k as input _i).

5. according to p (k _i) candidate is imported tabulation reorder, the front of tabulation is imported in preferred input the candidate.Candidate list is shone upon back Chinese character, shine upon between phonetic and the Chinese character and be specially: from phonetic and Chinese character corresponding relation dictionary, obtain the corresponding Chinese character of phonetic, thereby obtain the input candidate list that Chinese character is represented, use by the user and select final input content the candidate list of keyboard after reordering.

Although disclose specific embodiments of the invention and accompanying drawing for the purpose of illustration, its purpose is to help to understand content of the present invention and implement according to this, but it will be appreciated by those skilled in the art that: without departing from the spirit and scope of the invention and the appended claims, various replacements, variation and modification all are possible.The present invention should not be limited to this instructions most preferred embodiment and the disclosed content of accompanying drawing, and the scope of protection of present invention is as the criterion with the scope that claims define.

Claims (5)

1. hyperchannel Chinese character input method towards mobile device, its step is as follows:

1) on mobile device, uses keyboard and speech input information simultaneously; Obtain the time of button time and phonetic entry, write down each first key of importing Chinese word and acknowledgement key and press constantly, calculate the efficient voice time;

2) read interior voice recognition information of keyboard input information and efficient voice time, obtain keyboard input candidate collection and speech recognition candidate collection;

3) adopt the hyperchannel fusion method to merge to above-mentioned speech recognition candidate collection and keyboard input candidate collection, according to certain keyboard input candidate's the input probability candidate collection that reorders, and the candidate collection after will reordering shines upon back Chinese character, selects final input by the user;

The described efficient voice time is max{T (PreConfirmation), the time between T (FirstKeyPress)-Interval} and the T (Confirmation); Wherein T (Confirmation) is the affirmation moment of the Chinese word of current input, and T (PreConfirmation) is the affirmation moment of the Chinese word of previous input; T (FirstKeyPress) is moment of first button of the Chinese word of current input; Described Interval is the constant of effective input time of restriction voice.

2. a kind of hyperchannel Chinese character input method towards mobile device as claimed in claim 1 is characterized in that the hyperchannel fusion method in the described step 3) is as follows:

1) computing voice identification candidate s _jWith keyboard input candidate k _iSimilarity w (k _i, s _j); K wherein _iBe i candidate of keyboard input, s _jBe j candidate of speech recognition;

2) calculate given speech recognition candidate s _jThe time, select keyboard input candidate k _iProbability p (k for final input _i| s _j);

3) calculate selection keyboard input candidate k _iProbability p (k for final input _i);

4) press p (k _i) order of descending is to keyboard input candidate ordering, the candidate list after obtaining reordering.

3. as a kind of hyperchannel Chinese character input method as described in the claim 2, it is characterized in that: described similarity towards mobile device $w(k_i,s_j)=\frac{1}{\mathrm{edit}\_\mathrm{dis}\tan \mathrm{ce}(k_i,s_j)+a},$ Edit_distance (k wherein _i, s _j) be k _iWith s _jBetween editing distance, the denominator value was not 0 when constant a guaranteed that similarity is calculated, and provided editing distance shared proportion in similarity is calculated, a is greater than 0.

4. as a kind of hyperchannel Chinese character input method as described in the claim 3, it is characterized in that: described towards mobile device $p\left(k_i\right|s_j)=p\left(s_j\right)\×\frac{w(k_i,s_j)}{Z^{\′}\left(j\right)},$ Wherein, $Z^{\′}\left(j\right)=\underset{i=1}{\overset{M}{\mathrm{\Σ}}}w(k_i,s_j),$ P (s _j) obtain by recognizer, the expression phonetic entry is identified as speech recognition candidate s _jProbability, M represents the input candidate number in the keyboard input candidate collection.

5. as a kind of hyperchannel Chinese character input method as described in the claim 4, it is characterized in that: described towards mobile device $p\left(k_i\right)=\frac{b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j)}{\underset{i=1}{\overset{M}{\mathrm{\Σ}}}(b\×\mathrm{freq}\left(k_i\right)+\underset{j=1}{\overset{N}{\mathrm{\Σ}}}p\left(k_i\right|s_j))},$ Freq (k wherein _i) expression k _iFrequency, the span of b is 0 to 1, N represents the input candidate number in the speech recognition candidate collection.

Images