CN1121015C - Intelligent Chinese computer system for the blind - Google Patents

Abstract

The present invention relates to an intelligent Chinese computer system for a blindman, which belongs to the field of a pattern recognition and artificial intelligence technology. The present invention is mainly composed of a personal computer host machine which is connected with the Internet, hardware and a software module arranged in the host machine and the relevant hardware, wherein the hardware is composed of a microphone, a sound box or an earphone, a scanner, a printer and a Braille display for the blindman, and the elements are connected with interfaces of the host machine. The present invention can make the blindman fully exert abilities of listening, speaking and touching when the computer is used, and the blindman can selectively naturally and conveniently operate the computer so as to make an interaction process humanized and intellectualized. The present invention provides a tool for the blindman to process documents and to be communicated with normal persons, and the present invention provides a teaching tool for a teacher in a school for blindmen.

Description

The intelligent Chinese computer system that the blind person uses

Technical field

The invention belongs to pattern-recognition and field of artificial intelligence.Be particularly related to the intelligent computer systems design that Chinese blind person uses.

Background technology

The blind person uses braille (touching the braille symbol of reading) to carry out attending classes and information interchange.In some developed countries, having worked out preferably, the blind person uses computing machine and operating platform thereof.Britain has developed the computing machine that the blind person uses, and each key of its keyboard is to be differed by size, shape, texture, and every key all has the interaction of multimedia information function of acoustic mechanism.Microsoft (Microsoft) expression, plan is cooperated with the Pause Data International of New Zealand dysopia technology manufacturer, but develops the electronic book reading machine of blind man and visually impaired person use.Ground such as Taiwan, Hong Kong also has corresponding braille computing machine (mainly being to have the blind person to put apparent device) to put goods on the market.Price is all very high, and a point shows device and wants 4000～5000 dollars, and general Chinese blind person can't afford.In China, in recent years, for the blind person can being used a computer and can reading the work that plain text has also been done some parts, under the subsidy of China Disabled Federation and China Blind Person Association is supported, develop braille word link writing system as Chinese braille bookstore; Reading machine for the blind was studied in the National Library of China under Dos operating system, be the common Chinese-character text of block letter is discerned by scanning input computer, converted the Chinese character of discerning to sound again and was exported by computing machine; Make the blind person can hear plain text; Department of Automation of Tsing-Hua University studied the blind person and used inputting method, helped word selection with sound, and the conversion of the Chinese character braille under Dos.

In addition, person of good sense's Chinese Character Recognition, speech recognition, speech synthesis technique have reached practical or approaching practical level.But, the intelligent Chinese computer system that does not also have the blind person to use in the world at present.

Summary of the invention

The objective of the invention is to overcome the deficiency of above-mentioned technology, proposed the intelligent Chinese computer system that a kind of blind person uses, the blind person is given full play to listen, say, touch ability when using a computer, more natural selectively, operational computations machine more easily.Compare with traditional man-machine information interaction means, this system has adopted multimodal interactive mode.The user both can use keyboard, blind person to put traditional interactive modes such as apparent device, display, also can adopt voice and OCR new interaction techniques such as (optical character identification) simultaneously.Make reciprocal process hommization more, intellectuality.Give blind person's document process, exchange with the normal person, school for the blind's teachers ' teaching provides instrument.

The intelligent Chinese computer system that a kind of blind person that the present invention proposes uses mainly comprises the common personal computer that can surf the Net, transforms into the also apparent device output unit of point of output of standard braille ASCII character in order to Chinese character or the Braille that computing machine is shown; In order to receive image that scanner obtains and the optical character recognition device that converts computer version by identification to; The keyboard for blind person input and the editing machine of input Chinese character and Braille; Voice messaging or control command are conveyed into the voice input device of computing machine; Printer output device in order to the content output that computing machine is to be exported; In order to statement, phrase, speech or syllable are become the phonetic synthesis output unit of sound waveform and output; It is characterized in that, also comprise be arranged in the said computing machine in order to realize Chinese braille to Chinese character and Chinese character to the blind converter of the Chinese of the automatic conversion of Chinese braille; The comprehensive knowledge base of required various knowledge bases when being arranged in the said computing machine in order to the blind intertranslation of the management Chinese; Be arranged on the speech recognition device in order to said phonetic entry is circulated be changed to computer version or identify the key words in this phonetic entry stream in the said computing machine; Be arranged on the natural language generation device that has the Chinese sentence of intonation in the said computing machine in order to generation; Said optical character recognition device, keyboard for blind person input and editing machine, three kinds of input channels of voice input device constitute unified input interface; Show three kinds of output channels that device output unit, printer output device, phonetic synthesis output unit constitute by point; Be arranged in the said computing machine in order to controller overhead control between said three kinds of inputs, the output channel; Be arranged in the said computing machine in order to the man-machine interaction content is analyzed, in time make the theme prediction, estimate the next action of user, the inference system device of different interactive strategies is specified and carried out to the analysis user demand; The blind converter of the said Chinese comprises the automatic switch of Chinese braille to the automatic switch of Chinese character and Chinese character to Chinese braille, wherein, Chinese braille is discerned braille to the automatic switch of Chinese character in order to books printed in braille are scanned the back, or with keyboard with after the braille input, the notion of braille by phonetic is converted to Chinese character; Each link of said phonetic and Chinese character conversion, utilize the Chinese braille comprehensive knowledge base, phonetic in band transition probability weight adopts the searching method of state transition path to obtain N optimum in order to Chinese character conversion search graph, realizes by the automatic conversion of braille to Chinese character; Said Chinese character is done braille word link writing according to Chinese braille word link writing rule to Chinese-character text for elder generation to the automatic switch of Chinese braille, then speech is converted to braille; Said participle is one by one speech branch to be come write; Said write the two or more syllables of a word together are the characteristics according to braille, and the principle that is the right length by logicality and custom, the syllable of Chinese grammar, voice links up some speech to be write, and too disperses to avoid syllable structure, is convenient to touch and reads.

The function of each module and implementation method are described as follows among the present invention:

OCR (optical character identification) device: adopt prior art.Both person of good sense's the Braille text that is printed on the Chinese-character text on the paper or is engraved on the paper can be sent into computing machine by optical scanner, automatically handle, comprise automatic identification to printed Chinese character, handwritten Chinese character, Braille, image file is converted into e-text, provides necessary condition for reading processing such as (reading aloud), editor.

Voice input device: facing to the microphone speech, voice messaging (control command etc.) is sent into computing machine, adopt prior art.

The apparent device output unit of point: Chinese character or Braille that computing machine is shown transform into standard braille ASCII character, output to the apparent device of point that the blind person uses, the blind person can be read by touching, understand computing machine, reach and use a computer and the purpose mutual with it just in content displayed.

Table one: the table of comparisons of braille ASCII character and Braille sign indicating number (the braille code be by about 2 each three point of row, from left bank is 1,2,3 points from top to bottom, row is 4,5,6 points from top to bottom from the right side, is referred to as a position, and the binary value of braille code is from left to right to be followed successively by 1-6 point position)

ASCII character is worth 16 systems	ASCII code value 10 systems	Control character	Symbol	The Braille code, 2 systems	Braille code 16 systems
						?20	?32		Short side
?21	?33		Exclamation mark	011101	?1D
						?22	?34		Double quotation marks	000010	?02
?23	?35		Pound sign	001111	?0F
						?24	?36		Dollar number	110101	?35
?25	?37		Percentage sign	100101	?25
						?26	?38		With number	111101	?3D
?27	?39		Single quotation marks	001000	?08
						?28	?40		Open parenthesis	111011	?3B
?29	?41		Close symbol	011111	?1F
						?2A	?42		Asterisk	100001	?21
?2B	?43		Plus sige	001101	?0D
						?2C	?44		Comma	000001	?01
?2D	?45		Minus sign	001001	?09
						?2E	?46		Round dot	000101	?05
?2F	?47		Oblique fraction line	001100	?0C
						?30	?48		0	?001011	?0B
?31	?49		1	?010000	?10

32	50		2	011000	18
						33	51		3	010010	12
34	52		4	010011	13
						35	53		5	010001	11
36	54		6	011010	1A
						37	55		7	011011	1B
38	56		8	011001	19
						39	57		9	001010	0A
3A	58		Colon	100011	23
						3B	59		Branch	000011	03
3C	60		Is less than	110001	31
						3D	61		Equal sign	111111	3F
3E	62		Greater-than sign	001110	0E
						3F	63		Question mark	100111	27
40	64	NUL (sky)	Circle a	000100	04
						41	65	SOH (start of header)	A	100000	20
42	66	STX (start of text)	B	110000	30
						43	67	ETX (end of text)	C	100100	24
44	68	EOT (end of transmission (EOT))	D	100110	26
						45	69	ENQ (inquiry)	E	100010	22
46	70	ACK (admitting)	F	110100	34
						47	71	BEL (bell character (BEL))	G	110110	36
48	72	BS (backspace)	H	110010	32
						49	73	HT (horizontal tabulation)	I	010100	14
4A	74	LF (line feed)	J	010110	16
						4B	75	VT (vertical tab)	K	101000	28
4C	76	FF (skipping)	L	111000	38
						4D	77	CR (carriage return)	M	101100	2C
4E	78	SO (displacement output)	N	101110	2E
						4F	79	SI (displacement input)	O	101010	2A
50	80	DLE (data link escape)	P	111100	3C
						51	81	DC1 (device control 1)	Q	111110	3E
52	82	DC2 (device control 2)	R	111010	3A
						53	83	DC3 (device control 3)	S	011100	1C
54	84	DC4 (device control 4)	T	011110	1E
						55	85	NAK (negating)	U	101001	29
56	86	SYN (synchronously)	V	111001	39
						57	87	ETB (transmission block end)	W	010111	17
58	88	CAN (calcellation)	X	101101	2D
						59	89	EM (medium are with finishing)	Y	101111	2F
5A	90	SUB (displacement)	Z	101011	2B
						5B	91	ESC (escape)	Open bracket	010101	15
5C	92	FS (file separator)	Fall oblique line	110011	33
						5D	93	GS (group separater)	Close bracket	110111	37

5E	94	RS (rs chacter)	Last pinnacle	000110	06
						5F	95	US (unit separator)	Short-term	000111	07
60	96		Single apostrophe	000100	04
						61	97		a	100000	20
62	98		b	110000	30
						63	99		c	100100	24
64	100		d	100110	26
						65	101		e	100010	22
66	102		f	110100	34
						67	103		g	110110	36
68	104		h	110010	32
						69	105		i	010100	14
6A	106		j	010110	16
						6B	107		k	101000	28
6C	108		l	111000	38
						6D	109		m	101100	2C
6E	110		n	101110	2E
						6F	111		o	101010	2A
70	112		p	111100	3C
						71	113		q	111110	3E
72	114		r	111010	3A
						73	115		s	011100	1C
74	116		t	011110	1E
						75	117		u	101001	29
76	118		v	111001	39
						77	119		w	010111	17
78	120		x	101101	2D
						79	121		y	101111	2F
7A	122		z	101011	2B
						7B	123		Open braces	010101	15
7C	124		Two vertical lines	110011	33
						7D	125		Close braces	110111	37
7E	126		Tilde	000110	06
						7F	127		Deletion	000111	07

The printer output device: the content that computing machine is to be exported outputs to blind person's printer or Chinese characters in current use printer, adopts prior art.

Phonetic synthesis output unit: statement, phrase, speech or syllable are become sound waveform, sound by audio amplifier or earphone.Adopt prior art.

The blind converter of the Chinese: comprise the automatic conversion of Chinese braille to the automatic conversion of Chinese character and Chinese character to Chinese braille.Wherein, Chinese braille to the Implementation of automatic transformation method of Chinese character is: with books printed in braille scanning back identification braille, or with keyboard with the braille input after, the notion of braille by phonetic is converted to Chinese character; Each link of said phonetic and Chinese character conversion, utilize the Chinese braille comprehensive knowledge base, phonetic in band transition probability weight adopts Viterbi searching method (a kind of dynamic programming algorithm to Chinese character conversion search graph, here be used for the search of state transition path) obtain N optimum in order, realize by the automatic conversion of braille to Chinese character.

Said Chinese braille comprehensive knowledge base: comprise electronic dictionary, rule base and statistical information storehouse (showing the probability storehouse together in abutting connection with speech) by what the extensive real corpus of statistics obtained.

Above-mentioned Chinese braille comprises following concrete steps to the automatic switching method of Chinese character:

1) reads in the not whole continuous non-Braille symbol of converting text head;

Whether 2) current input point word symbol represents non-Chinese character meaning, if the expression Chinese character changes step 4); If the non-Chinese character of expression is searched for the N-best path and selected best path in the Viterbi search graph, obtain transformation result, and the non-Braille symbol that begins to read in is inserted into correspondence position;

3) transformation result of minute book sentence, the transformation result of the input point word symbol of the non-Chinese character meaning of record expression empties the viterbi search graph, changes step 5) over to;

4) search all Chinese character speech candidates that the braille symbol of current input can mate, and in the Viterbi search graph the corresponding node of structure.

5) judge whether that all conversion finishes? if, output conversion back Chinese character result; If not, change step 1).

Chinese character to the Implementation of automatic transformation method of Chinese braille is:

At first Chinese-character text is done the braille word link writing, convert speech to braille then according to Chinese braille word link writing rule; Said participle is one by one speech branch to be come write; Said write the two or more syllables of a word together are the characteristics according to braille, and the principle that is the right length by logicality and custom, the syllable of Chinese grammar, voice links up some speech to be write, and too disperses to avoid syllable structure, is convenient to touch and reads.

Above-mentioned Chinese character specifically can may further comprise the steps to the automatic switching method of Chinese braille:

1) at first non-Chinese symbol is carried out pre-cutting and handle, read in one section continuous Chinese character string, use MM method (oppositely maximum match method) and RMM method (oppositely maximum match method) respectively, carry out participle according to vocabulary;

2) relatively whether MM is identical, identical with the RMM word segmentation result, and the record word segmentation result changes step 4) over to;

3) when MM and RMM word segmentation result are inequality, the ambiguity tree of structure ambiguity field is searched for optimum word segmentation result, and the record word segmentation result changes step 4) over to;

4) do you judge that the text participle finishes? if, according to braille word link writing rule word segmentation result is made amendment, generate the Braille of word segmentation result correspondence; If not, change step 1).

Comprehensive knowledge base: required various knowledge bases when being the blind intertranslation of the Chinese comprise:

(1) electronic dictionary for Braille: comprise the electronic dictionary of Chinese character (60,000 speech) to the electronic dictionary of braille, braille (60,000 speech) to Chinese character, Chinese word segmenting dictionary etc.

(2) rule base: comprise Chinese braille word link writing rule, morphological rule, phrase rule, syntactic rule etc.

(3) statistical information storehouse: in order to reflect the Chinese context relation, the adjacent speech of Chinese that obtains with several hundred million word real corpus statistics connects dependence statistical knowledge etc. with showing between probability storehouse, part of speech.

Blind person's common keyboard input interface module: the blind person imports Chinese character or Braille with the common computer keyboard by blind person's input method of Chinese character and braille input method.Adopt prior art.

Speech recognition device: dual mode is arranged

(1) unspecified person, continuous speech recognition are prior arts.

(2) keyword voice identification: the key word recognition that will import in the voice flow is come out, and is convenient to speaker's semantic understanding.Be used to differentiate the computer command of the various sayings that the blind person sends.It is prior art.

Natural language generation device: according to the mutual content of control, as when needing voice suggestion or inquiry, produce and have Chinese sentence intonation, that the blind person easily understands.What adopt at present is the voice that record in advance according to content choice, plays.

Controller: be the overhead control between above several input, the output channel.The dialogue management layer is the core of system, it is organized the whole session process according to certain dialog strategy, is responsible for the communication between each module, make the reaction of system according to corresponding decision rule, so that man-machine interaction is normally carried out under Expected Results.

Controller is by state analyzer, and dialog manager and state storehouse are formed.

The state of storing in the state storehouse comprises system state and dialogue state.System state is described the situation (as: program name, ongoing operation, operation requirement etc.) that the current application program module starts and moves with certain data structure, has also comprised the pattern of the input and output of using simultaneously.Dialogue state has reflected the situation of current man-machine interaction process.Because the restriction of the form of system operation order, dialogue state is by (env; Act; Obj; Condition) case-frame (case grammar) expression.

Dialog manager is made corresponding dialogue action by dialogue state is analyzed, and perhaps finishes system acting, perhaps carries out corresponding system prompt.Dialog manager adopts present general slot-filling algorithm (groove fill method) to realize corresponding dialog strategy, manages and dispatches in order to the process to dialogue.Prior art.

State analyzer is responsible for the multi-mode input of the system that accepts, and selects following action to carry out according to current system state: start dialog manager; Start the corresponding application module; Send message to application program module; Transfer the I/O control to application program module.The final state analyzer converts input to canonical form and is put in the system state storehouse.Prior art.

The inference system device

This device is taked based on the semantic description system of case grammar with based on the analytical algorithm of robust mode coupling.Form with case frame (Case Frame) is carried out semantic expressiveness to discourse content, a case frame comprises a notion and relevant attribute (being groove) thereof, with recursive transition network (Recursive Transition Networks, RTN) the possible linguistic form to these grooves is described; Carry out the Viterbi beam search of (Viterbi beta pruning, a kind of optimized Algorithm are used to improve the arithmetic speed of state transition path search here) then, obtain the result of ultimate analysis according to certain evaluation mark.

The present invention can realize following function:

1. the present invention's input has three passages: common keyboard input, OCR input and phonetic entry; Output has three passages: voice output, printout and point show device output.Input and output following several modes capable of being combined:

1) braille computing machine input, Chinese-character text output (point shows device output, display output, printer output).To have under voice suggestion helps that document on the braille paper converts the electronic edition document to by the OCR converter or by keyboard input and editor's braille document, and convert thereof into by blind Chinese translation function and be the Chinese character document.By normal printer or display output.The order of equipment therefor is: OCR converter, input interface, the blind converter of the Chinese (braille is converted to Chinese character), natural-sounding are understood device, speech recognition device, natural language generation device, phonetic synthesis output unit, comprehensive knowledge base, controller, printer output device (universal printer).(blind person and person of good sense use alternately)

2) braille computing machine input, braille text output (printer output).The document that will have under voice suggestion helps on the braille paper is converted to the electronic edition document or is imported and edit the braille document by keyboard by the OCR converter.Directly show device output by braille printer or point.The order of equipment therefor is: OCR converter, input interface, controller, natural language understanding device, speech recognition device, natural language generation device, phonetic synthesis output unit, point show device output unit, printer output device (braille printer).(blind person and blind person use alternately)

3) Chinese-character text input, (point shows device output, printer output in braille output.To have under voice suggestion helps that document on the Chinese character paper converts the electronic edition document to by the OCR converter or by keyboard input and editor's Chinese character document, and convert thereof into by the blind translation function of the Chinese and be the braille document.Show device output by braille printer or point.The order of equipment therefor is: OCR converter, input interface, controller, the blind converter of the Chinese (Chinese character is converted to braille), natural language understanding device, speech recognition device, natural language generating apparatus, phonetic synthesis output unit, point show device output unit, comprehensive knowledge base, printer output device (braille printer).(annotate: blind person's teaching and braille publishing are used)

4) Chinese-character text input, Chinese character output (display, printer output).The document that will have under voice suggestion helps on the Chinese character paper is converted to the electronic edition document or is imported and edit the Chinese character document by keyboard by the OCR converter.Directly by normal printer or display output.The order of used module is: OCR converter, input interface, controller, natural language understanding device, speech recognition device, natural language generating apparatus, phonetic synthesis output unit, printer output device (universal printer).(blind person and person of good sense use alternately)

2. braille Chinese character auto-conversion function: the braille document is converted into the Chinese character document automatically.Equipment therefor is: the blind converter of the Chinese (braille is converted to Chinese character), comprehensive knowledge base.

3. Chinese character braille auto-conversion function: the Chinese character document is automatically converted to the braille document.Equipment therefor is: the blind converter of the Chinese (Chinese character is converted to braille), comprehensive knowledge base.

4. the blind person listens and reads Chinese-character text (novel, magazine, newspaper, Chinese character mail), and the order of equipment therefor is: OCR converter, controller, speech recognition device, natural language generating apparatus, phonetic synthesis output unit, natural language understanding device.

5. the blind person uses email manager: but blind person's sending and receiving Email, and read aloud mail of receiving and the mail of writing.Relate to blind person's Voice Navigation, Braille or Chinese character input and Chinese-character text output, document function of reading aloud.Equipment therefor is: input interface, speech recognition device, natural-sounding understand that device, inference system device, controller, natural language generation device, phonetic synthesis output unit, the blind converter of the Chinese, comprehensive knowledge base, natural-sounding are understood device, point shows device output unit, printer output device.

6. the blind person uses browser: the various information on blind person's browse network.The function of reading aloud of use blind person's Voice Navigation, the input of blind person's computing machine and Chinese-character text output, the blind person listening Chinese-character text.Equipment therefor is: input interface, speech recognition device, natural-sounding understand that device, inference system device, control interface, natural language generation device, phonetic synthesis output unit, the blind conversion of the Chinese, comprehensive knowledge base, natural-sounding are understood device, point shows device output unit, printer output device.

7. braille file manager: the mode with the order bar helps braille managing queries file.Equipment therefor is: input interface, inference system device, controller, natural language generation device, speech synthetic device, the blind converter of the Chinese, comprehensive knowledge base, natural-sounding are understood device.

8. blind person's Voice Navigation: the blind person can use a computer and network freely.The controllable order of every menu and hot key all can exhale order to replace with mouth, simultaneously can mouth exhales to close mouse, close order such as phonetic entry.The order of equipment therefor is: speech recognition device, natural-sounding are understood device, inference system device, controller, natural language generation device, phonetic synthesis output unit.

Characteristics of the present invention are: have multiple interactive mode, can select separately hardware configuration according to economic conditions and needs, that gives full play to the blind person when using a computer listens, says, touches ability.Make the blind person can be more natural selectively, operational computations machine more easily.Compare with traditional man-machine information interaction means, this system has adopted multimodal interactive mode.The user both can use keyboard, mouse, blind person to put traditional interactive modes such as apparent device, display, also can adopt new interaction techniques such as voice and OCR simultaneously.Make reciprocal process hommization more, intellectuality.Give blind person's document process, exchange with the normal person, school for the blind's teachers ' teaching provides instrument.

Description of drawings

Fig. 1 is that blind person's computer system of the present invention constitutes synoptic diagram.

Fig. 2 is embodiments of the invention Braille input synoptic diagram.

Fig. 3 is an embodiments of the invention blind person editing machine synoptic diagram.

Fig. 4 uses the email manager synoptic diagram for the embodiments of the invention blind person.

Fig. 5 uses the browser synoptic diagram for the embodiments of the invention blind person.

Embodiment

The intelligent Chinese computer system that a kind of blind person that the present invention proposes uses is described as follows in conjunction with each drawings and Examples:

A kind of embodiment that the present invention proposes is the minimum system that the blind person uses, its hardware comprises: the common personal computer that can surf the Net, the suitable Intel Pentium of basic hardware configuration requirement: CPU II is more than 400, more than the internal memory 128M, more than the hard disk 4G, sound card, microphone, audio amplifier or earphone and the required basic configuration of general computing machine.Basic software comprises: operating system Microsoft Windows9x or Windows 2000.

The composition and the course of work of present embodiment each several part are described in detail as follows:

1. keyboard input: dual mode can be arranged

(1) Braille input: international standard braille keyboard, use FDS, six keys of JKL are corresponding braille one side respectively, promptly from left to right, six points from top to bottom.In proper order: 3 points in the first left side, from top to bottom, 3 points in the right, back, from top to bottom.In the process of input phonetic entry prompting is arranged, make the blind person know oneself to hit down be which key, what sound.

(2) Chinese phonetic alphabet input method: can select western language, words spelling, words Two bors d's oeuveres etc.In the process of input phonetic entry prompting is arranged, make the blind person know oneself to hit down be which key, what sound.Can select the candidate of polyphone by voice suggestion.

Open or a newly-built braille file, promptly can import Braille.Open or a newly-built Chinese character file, promptly can import common Chinese character.

Characteristics are: except that each operation all has voice suggestion or response, can obtain corresponding Chinese converted contents in the input braille, as shown in Figure 2, be convenient to person of good sense (as: teacher) check and correction braille manuscript; Blind person and person of good sense's written communication.

2. read aloud Chinese-character text

To the Chinese character electronic document that has obtained, read aloud with phoneme synthesizing method.Open the Chinese character file, in the choice menus item " massage voice reading ", just can begin reading aloud of Chinese-character text, select this menu item will stop to read aloud once more.In addition can also read aloud automatically the menu in the menu bar of current cursor place.

Characteristics are: the blind person not only can listen and read electronic edition Chinese character document, also can read various forms by the OCR translation function simultaneously, as the Chinese character document of storages such as CD, books.

3. Voice Navigation

This system adopts the keyword recognition technology to realize Voice Navigation.Therefore, send when order can be with various close, more ambiguous statements.For example the user wants the a1.txt that opens file, and he may say:

1) a1.txt that opens file

2) file a1.txt is opened

3) open a1.txt

4) a1.txt is opened

These four kinds of saying connotations are identical, but as the signal of phonetic entry a great difference just arranged, and observing the common ground that these sayings can find out them is that a verb-" opening " all arranged, all with object-" filename " of a logical meaning.Also there is similar problem for sayings such as copy, deletions.This system finds out the object of one of verb crucial in the phonetic entry and important attribute thereof, finishes identification and affirmation to user input commands.The keyword recognition system generally is used in the situation of unspecified person, continuous speech.Employing is based on the keyword recognition method of HMM (hidden Markov model, a kind of model of describing the state transitions relation) framework, and its principle is:

At first with the voice flow segmentation of input, every section corresponding and a sentence or the voice paragraph that sentence length is suitable.Which keyword then, searches in each section and determined whether keyword, be if there is keyword also must determine.The input of system is made up of keyword input and the outer voice of antistop list, and the latter is called rubbish, can comprise non-key speech, non-language (sucking mouth sound, breathing sound etc.) and ground unrest three parts.System sets up a cover HMM model for each keyword, in like manner also will set up some cover HMM to rubbish.The feature vector sequence of any one section input voice is obtained the status switch corresponding with this sequence with the Viterbi algorithm, if in the state of experience the person that belongs to the keyword is arranged, can detect the keyword of correspondence.

4. voice system control.Be characterized in: the special-purpose subsystem of several blind persons is integrated with Voice Navigation.Judge the residing duty of present system, suitable operation is carried out in order according to the working environment analyzing speech.The controllable order of every menu and hot key all can exhale order to replace with mouth.Simultaneously can mouth exhale and close mouse, close orders such as phonetic entry.(beginning with particular key control phonetic entry) to avoid noise

Voice Navigation not only integrates a plurality of blind persons with subsystem, the interactive mode that makes things convenient for the close friend is provided for simultaneously these softwares, makes modern technologies such as the blind person can use a computer more freely, network, joins among the informationized society.

5. blind person's editing machine and braille printout

Blind person's editing machine is the editing machine that makes things convenient for the blind person to use, and it must have the basic function of general editing machine, and appropriate voice interactive function is provided.This editing machine is based on Keyboard Control, and promptly the blind person controls current working state by keyboard.The blind person is an indispensable ingredient (as described above) of blind person's editing machine with Braille input method and Chinese character input method.The method for designing of editing machine is: in input process, after the end of input, or after opening certain electronic document, the blind person can learn current cursor position by voice suggestion, with which which row mark of row.The blind person can listen and read, edit, revise document, as deleting, add, duplicate literal, paragraph etc. under the help of voice.When running into phonetically similar word and can not recognize, the blind person can use explanation function, by phrase differentiate be which phonetically similar word as: red, pronunciation Hong, when looking into the meaning of word, computing machine will be used the voice informing user: red, red flag red; Red is red.Same flood, pronunciation Hong will be apprised of flood, the flood of flood.Can obtain the translator of English of this speech if desired by Chinese-English dictionary; If English, can read, can explain that the Chinese of this English looks like by english Chinese dictionary as needs.At last, the blind person can select to read aloud continuously, listens and reads content in full.So the Core Feature of blind person's editing machine is: State Control, exercisable function difference under different states, as, can not arbitrarily delete under the file management state, cursor leaves the file operation district, and prompting or help user return; Read aloud; Keyboard or voice control by the order of various keyboard operations or phonetic entry, help the user to finish document and listen the task of reading, understand, writing.As shown in Figure 3.

1) State Control:

Monitor the cursor current location, avoid system to carry out illegal operation.

2) read aloud:

Report current cursor position (which row, which row); Read aloud cursor left side letter or Chinese character, cursor right side letter or Chinese character; Can make an explanation to Chinese character in case of necessity, centering, english make an explanation, translate; Read continuously from current location, stop.

3) read aloud automatically:

Automatically read when finishing cursor left, read automatically during cursor right, read the right when cursor up down is moved automatically, read the left side when cursor up down is moved automatically.

Native system can directly connect the braille printer, as: the INDEX BRAILLE series of products of the Index Embossers company of Sweden Sweden, carry out Braille to current braille electronic document and print.

6. the blind person uses email manager: as shown in Figure 4.

Under voice control, in the ordinary electronic E-Mail Manager, add Voice Navigation, massage voice reading, the blind Chinese, the blind translation function of the Chinese.This blind person can finish the basic function of writing, send and receive e-mail, set up common email managers such as address book with email manager.Can be the blind person and read aloud the mail of receiving, writing.Blind users relies on voice and system interaction to finish the operation of sending and receiving Email.For example: computing machine is informed user's " you have new mail " by sound; Inquiry " your receiving emails? ", " will read aloud mail? " Prompting " please import receiver's address ", " please import e-mail theme " etc.After each operation corresponding voice answer-back or voice suggestion are arranged all.

7. the blind person uses browser: as shown in Figure 5.

Under voice control, in generic browser, add Voice Navigation, massage voice reading, OCR conversion, the blind Chinese, the blind translation function of the Chinese.This blind person can finish the basic function of general browsers such as inquiry, reading with browser.Can be the blind person and read aloud web page contents.The user relies on voice and system interaction to finish operations such as keyword, network address input, inquiry.For example: computing machine is informed user " you arrived so-and-so webpage " by sound; Inquire " you want what is inquired about? " Prompting " please import keyword ", " please import network address " etc.After arriving named web page, put in order, read aloud web page contents according to webpage.Can skip by keyboard and finish to read aloud.

8. intelligent inference function

This belongs to medium-scale towards blind person's man-machine information interaction system, and the theme of application is the various common operations and the simple information inquiry of computing machine.Because main blind man uses, so real-time requires better, the friendly degree of use is higher.Consider these factors, take based on the semantic description system of case grammar and the analytical algorithm of mating based on robust mode.Form with case frame (Case Frame) is carried out semantic expressiveness to discourse content, a case frame comprises a notion and relevant attribute (being groove) thereof, (Recursiye TransitionNetworks, RTN) the possible linguistic form to these grooves is described with recursive transition network.When analyzing, use top-down RTNchart (chart) analytical algorithm that sentence is mated, as import sentence and the word outside the system dictionary occurred, give elimination and do not do analysis, for asyntactic composition in the input, directly skip during analysis, search can constitute the segment (being significant phrase) of notion.Carry out the search of Viterbi beam, obtain the result of ultimate analysis according to certain evaluation mark.Be mapped to semantic frame by the phrase that analyzes, so just obtained one or several such case frame and represented for sentence.Interaction content is carried out performance analysis, and the real-time update system state.

Simultaneity factor is in time made the theme prediction, estimates the next action of user, optimizes the knowledge base searching algorithm.Different interactive strategies is specified and carried out to the demand of analysis user.Predict and induce according to user's past behavior and current behavior, accelerate the realization of systematic search function, avoid because mouth exhales the identification of order and misconstruction etc. to make system enter endless loop.

The present invention has set up tens required comprehensive knowledge bases of the blind intertranslation of the Chinese, the theory that Chinese natural language is understood is applied in the braille technology for automatically treating first, finished one and can carry out the blind Chinese of Chinese, the blind automatic conversion of the Chinese, the input editing of collection braille Chinese, blind person are controlled in the intelligent computer systems towards Chinese blind person of one with Email sending and receiving management, voice system.With the artificial intelligence representation of knowledge and reasoning, the theme prediction, the content analysis scheduling theory is applied to the system state analysis, makes it have certain voice human-computer interaction function, and can utilize man-machine conversation, and system points out the user and induces, and is user-friendly.

Claims (1)

1, the intelligent Chinese computer system used of a kind of blind person mainly comprises the common personal computer that can surf the Net, and the point that transforms into standard braille ASCII character and output in order to Chinese character that computing machine is shown or Braille shows the device output unit; In order to receive image that scanner obtains and the optical character recognition device that converts computer version by identification to; The keyboard for blind person input and the editing machine of input Chinese character and Braille; Voice messaging or control command are conveyed into the voice input device of computing machine; Printer output device in order to the content output that computing machine is to be exported; In order to statement, phrase, speech or syllable are become the phonetic synthesis output unit of sound waveform and output; It is characterized in that, also comprise be arranged in the said computing machine in order to realize Chinese braille to Chinese character and Chinese character to the blind converter of the Chinese of the automatic conversion of Chinese braille; The comprehensive knowledge base of required various knowledge bases when being arranged in the said computing machine in order to the blind intertranslation of the management Chinese; Be arranged on the speech recognition device in order to said phonetic entry is circulated be changed to computer version or identify the key words in this phonetic entry stream in the said computing machine; Be arranged on the natural language generation device that has the Chinese sentence of intonation in the said computing machine in order to generation; Said optical character recognition device, keyboard for blind person input and editing machine, three kinds of input channels of voice input device constitute unified input interface; Show three kinds of output channels that device output unit, printer output device, phonetic synthesis output unit constitute by point; Be arranged in the said computing machine in order to controller overhead control between said three kinds of inputs, the output channel; Be arranged in the said computing machine in order to the man-machine interaction content is analyzed, in time make the theme prediction, estimate the next action of user, the inference system device of different interactive strategies is specified and carried out to the analysis user demand; The blind converter of the said Chinese comprises the automatic switch of Chinese braille to the automatic switch of Chinese character and Chinese character to Chinese braille, wherein, Chinese braille is discerned braille to the automatic switch of Chinese character in order to books printed in braille are scanned the back, or with keyboard with after the braille input, the notion of braille by phonetic is converted to Chinese character; Each link of said phonetic and Chinese character conversion, utilize the Chinese braille comprehensive knowledge base, phonetic in band transition probability weight adopts the searching method of state transition path to obtain N optimum in order to Chinese character conversion search graph, realizes by the automatic conversion of braille to Chinese character; Said Chinese character is done braille word link writing according to Chinese braille word link writing rule to Chinese-character text for elder generation to the automatic switch of Chinese braille, then speech is converted to braille; Said participle is one by one speech branch to be come write; Said write the two or more syllables of a word together are the characteristics according to braille, and the principle that is the right length by logicality and custom, the syllable of Chinese grammar, voice links up some speech to be write, and too disperses to avoid syllable structure, is convenient to touch and reads.

Images