CN1323003A - Intelligent Chinese computer system for the blind - Google Patents

Abstract

The present invention belongs to the field of mode identification and artificial intelligence technology. The computer system consists of network PC as well as microphone, sound box or earphone, scanner, blind's Braille display, printer and software modules in the host computer and related hardware. The present invention makes it possible for the blind to operate computer naturally and conveniently by means of listening, saying and touching. The interacting process is homizized and intelligent, and supplies tool for the blind to treat file and intercourse with those without eyesight obstruction and the present invention provides the teachers in blind school with useful tool.

Description

The intelligent Chinese computer system that the blind person uses

The invention belongs to pattern-recognition and field of artificial intelligence.Be particularly related to the intelligent computer systems design that Chinese blind person uses.

The blind person uses braille (touching the braille symbol of reading) to carry out attending classes and information interchange.In some developed countries, having worked out preferably, the blind person uses computing machine and operating platform thereof.Britain has developed the computing machine that the blind person uses, and each key of its keyboard is to be differed by size, shape, texture, and every key all has the interaction of multimedia information function of acoustic mechanism.Microsoft (Microsoft) expression, plan is cooperated with the Pause Data International of New Zealand dysopia technology manufacturer, but develops the electronic book reading machine of blind man and visually impaired person use.Ground such as Taiwan, Hong Kong also has corresponding braille computing machine (mainly being to have the blind person to put apparent device) to put goods on the market.Price is all very high, and a point shows device and wants 4000～5000 dollars, and general Chinese blind person can't afford.In China, in recent years, for the blind person can being used a computer and can reading the work that plain text has also been done some parts, under the subsidy of China Disabled Federation and China Blind Person Association is supported, develop braille word link writing system as Chinese braille bookstore; Reading machine for the blind was studied in the National Library of China under Dos operating system, be the common Chinese-character text of block letter is discerned by scanning input computer, converted the Chinese character of discerning to sound again and was exported by computing machine; Make the blind person can hear plain text; Department of Automation of Tsing-Hua University studied the blind person and used inputting method, helped word selection with sound, and the conversion of the Chinese character braille under Dos.

In addition, person of good sense's Chinese Character Recognition, speech recognition, speech synthesis technique have reached practical or approaching practical level.But, the intelligent Chinese computer system that does not also have the blind person to use in the world at present.

The objective of the invention is to overcome the deficiency of above-mentioned technology, proposed the intelligent Chinese computer system that a kind of blind person uses, the blind person is given full play to listen, say, touch ability when using a computer, more natural selectively, operational computations machine more easily.Compare with traditional man-machine information interaction means, this system has adopted multimodal interactive mode.The user both can use keyboard, blind person to put traditional interactive modes such as apparent device, display, also can adopt new interaction techniques such as voice and OCR simultaneously.Make reciprocal process hommization more, intellectuality.Give blind person's document process, exchange with the normal person, school for the blind's teachers ' teaching provides instrument.

The intelligent Chinese computer system that blind person of the present invention uses is made of hardware and software module, as shown in Figure 1; The required hardware of this system is mainly main frame, comprising: display, keyboard, sound card, network interface card or modulator-demodular unit etc., the common personal computer that can surf the Net.Microphone, audio amplifier or the earphone that links to each other with each interface of this main frame, scanner (ordinary flat formula or blind person's special use), blind person are with the apparent device of point, printer (blind person's printer or Chinese characters in current use printer); This system software is arranged in said main frame and the related hardware.Wherein mainly contain: OCR module, keyboard for blind person input and editor module, voice input module constitute three kinds of input channels; Show device output module, printer output module, phonetic synthesis output module, three kinds of output channels of formation by point; And the blind conversion module of the Chinese, comprehensive knowledge library module, input interface module, the natural-sounding that are connected between said each input channel and the output channel are discerned Understanding Module, natural language generator module, voice operation demonstrator module, inference system module, control interface module.

The function of each module and implementation method are described as follows among the present invention:

OCR module: adopt prior art.Both person of good sense's the Braille text that is printed on the Chinese-character text on the paper or is engraved on the paper can be sent into computing machine by optical scanner, automatically handle, comprise automatic identification to printed Chinese character, handwritten Chinese character, Braille, image file is converted into e-text, provides necessary condition for reading processing such as (reading aloud), editor.

Phonetic entry: facing to the microphone speech, voice messaging (control command etc.) is sent into computing machine, adopt prior art.

The apparent device output module of point: Chinese character or Braille that computing machine is shown transform into standard braille ASCII character, output to the apparent device of point that the blind person uses, the blind person can be read by touching, understand computing machine, reach and use a computer and itself and mutual purpose just in content displayed.

Table one: the table of comparisons of braille ASCII character and Braille sign indicating number (the braille code be by about 2 each three point of row, from left bank is 1,2,3 points from top to bottom, row is 4,5,6 points from top to bottom from the right side, is referred to as a position, and the binary value of braille code is from left to right to be followed successively by 1-6 point position)

ASCII character is worth 16 systems	ASCII character is worth 10 systems	Control character	Symbol	The Braille code, 2 systems	Braille code 16 systems
						20	?32		Short side
21	?33		Exclamation mark	011101	?1D
						22	?34		Double quotation marks	000010	?02
23	?35		Pound sign	001111	?0F
						24	?36		Dollar number	110101	?35
25	?37		Percentage sign	100101	?25
						26	?38		With number	111101	?3D
27	?39		Single quotation marks	001000	?08
						28	?40		Open parenthesis	111011	?3B
29	?41		Close symbol	011111	?1F
						2A	?42		Asterisk	100001	?21
2B	?43		Plus sige	001101	?0D
						2C	?44		Comma	000001	?01
2D	?45		Minus sign	001001	?09
						2E	?46		Round dot	000101	?05
2F	?47		Oblique fraction line	001100	?0C
						30	?48		?0	001011	?0B
31	?49		?1	010000	?10
						32	?50		?2	011000	?18
33	?51		?3	010010	?12
						34	?52		?4	010011	?13
35	?53		?5	010001	?11
						36	?54		?6	011010	?1A
37	?55		?7	011011	?1B
						38	?56		?8	011001	?19
39	?57		?9	001010	?0A
						3A	?58		Colon	100011	?23
3B	?59		Branch	000011	?03
						3C	?60		Is less than	110001	?31
3D	?61		Equal sign	111111	?3F
						3E	?62		Greater-than sign	001110	?0E

?3F	?63		Question mark	?100111	?27
						?40	?64	NUL (sky)	Circle a	?000100	?04
?41	?65	SOH (start of header)	A	?100000	?20
						?42	?66	STX (start of text)	B	?110000	?30
?43	?67	ETX (end of text)	C	?100100	?24
						?44	?68	EOT (end of transmission (EOT))	D	?100110	?26
?45	?69	ENQ (inquiry)	E	?100010	?22
						?46	?70	ACK (admitting)	F	?110100	?34
?47	?71	BEL (bell character (BEL))	G	?110110	?36
						?48	?72	BS (backspace)	H	?110010	?32
?49	?73	HT (horizontal tabulation)	I	?010100	?14
						?4A	?74	LF (line feed)	J	?010110	?16
?4B	?75	VT (vertical tab)	K	?101000	?28
						?4C	?76	FF (skipping)	L	?111000	?38
?4D	?77	CR (carriage return)	M	?101100	?2C
						?4E	?78	SO (displacement output)	N	?101110	?2E
?4F	?79	SI (displacement input)	O	?101010	?2A
						?50	?80	DLE (data link escape)	P	?111100	?3C
?51	?81	DC1 (device control 1)	Q	?111110	?3E
						?52	?82	DC2 (device control 2)	R	?111010	?3A
?53	?83	DC3 (device control 3)	S	?011100	?1C
						?54	?84	DC4 (device control 4)	T	?011110	?1E
?55	?85	NAK (negating)	U	?101001	?29
						?56	?86	SYN (synchronously)	V	?111001	?39
?57	?87	ETB (transmission block end)	W	?010111	?17
						?58	?88	CAN (calcellation)	X	?101101	?2D
?59	?89	EM (medium are with finishing)	Y	?101111	?2F
						?5A	?90	SUB (displacement)	Z	?101011	?2B
?5B	?91	ESC (escape)	Open bracket	?010101	?15
						?5C	?92	FS (file separator)	Fall oblique line	?110011	?33
?5D	?93	GS (group separater)	Close bracket	?110111	?37
						?5E	?94	RS (rs chacter)	Last pinnacle	?000110	?06
?5F	?95	US (unit separator)	Short-term	?000111	?07
						?60	?96		Single apostrophe	?000100	?04
?61	?97		A	?100000	?20
						?62	?98		B	?110000	?30
?63	?99		C	?100100	?24
						?64	?100		D	?100110	?26
?65	?101		E	?100010	?22
						?66	?102		F	?110100	?34
?67	?103		G	?110110	?36
						?68	?104		H	?110010	?32
?69	?105		I	?010100	?14
						?6A	?106		J	?010110	?16
?6B	?107		K	?101000	?28
						?6C	?108		L	?111000	?38
?6D	?109		M	?101100	?2C
						?6E	?110		N	?101110	?2E
?6F	?111		O	?101010	?2A
						?70	?112		P	?111100	?3C

?71	?113		?Q	?111110	?3E
						?72	?114		?R	?111010	?3A
?73	?115		?S	?011100	?1C
						?74	?116		?T	?011110	?1E
?75	?117		?U	?101001	?29
						?76	?118		?V	?111001	?39
?77	?119		?W	?010111	?17
						?78	?120		?X	?101101	?2D
?79	?121		?Y	?101111	?2F
						?7A	?122		?Z	?101011	?2B
?7B	?123		Open braces	?010101	?15
						?7C	?124		Two vertical lines	?110011	?33
?7D	?125		Close braces	?110111	?37
						?7E	?126		Tilde	?000110	?06
?7F	?127		Deletion	?000111	?07

The printer output module: the content that computing machine is to be exported outputs to blind person's printer or Chinese characters in current use printer, adopts prior art.

Phonetic synthesis module, audio frequency output module: statement, phrase, speech or syllable are become sound waveform, sound by loudspeaker or earphone.Adopt prior art.

The blind modular converter of the Chinese: comprise the automatic conversion of Chinese braille to the automatic conversion of Chinese character and Chinese character to Chinese braille.Wherein, Chinese braille to the Implementation of automatic transformation method of Chinese character is: with books printed in braille scanning back identification braille, or with keyboard with the braille input after, the notion of braille by phonetic is converted to Chinese character; Each link of said phonetic and Chinese character conversion, utilize the Chinese braille comprehensive knowledge base, phonetic in band transition probability weight adopts the viterbi searching method to obtain N optimum in order to Chinese character conversion search graph, realizes by the automatic conversion of braille to Chinese character.

Said Chinese braille comprehensive knowledge base: comprise that electronic dictionary, rule base and statistical information storehouse are (by the big rule of statistics

What the mould real corpus obtained shows the probability storehouse together in abutting connection with speech).

Above-mentioned Chinese braille comprises following concrete steps to the automatic switching method of Chinese character:

1) reads in the not whole continuous non-Braille symbol of converting text head;

Whether 2) current input point word symbol represents non-Chinese character meaning, if the expression Chinese character changes step 4; If table

Show non-Chinese character, in the viterbi search graph, search for the N-best path and select best path, obtain changeing

Change the result, and the non-Braille symbol that begins to read in is inserted into correspondence position;

3) transformation result of minute book sentence, the transformation result of the input point word symbol of the non-Chinese character meaning of record expression,

Empty the viterbi search graph, change step 5 over to;

4) search all Chinese character speech candidates that the braille symbol of current input can mate, and search at viterbi

The corresponding node of structure among the figure.

5) judge whether that all conversion finishes? if, output conversion back Chinese character result; If not, change step 1.

Chinese character to the Implementation of automatic transformation method of Chinese braille is:

At first Chinese-character text is made the braille word link writing, convert speech to braille then according to Chinese braille word link writing rule; Said participle is one by one speech branch to be come write; Said write the two or more syllables of a word together are the characteristics according to braille, and the principle that is the right length by logicality and custom, the syllable of Chinese grammar, voice links up some speech to be write, and too disperses to avoid syllable structure, is convenient to mould and reads.

Above-mentioned Chinese character specifically can may further comprise the steps to the automatic switching method of Chinese braille:

1) at first non-Chinese symbol is carried out pre-cutting and handles, read in one section continuous Chinese character string, use respectively the MM method and

The RMM method is carried out participle according to vocabulary;

2) relatively whether MM is identical, identical with the RMM word segmentation result, and the record word segmentation result changes step 1 over to;

3) when MM and RMM word segmentation result are inequality, the ambiguity tree of structure ambiguity field is searched for optimum participle

As a result, the record word segmentation result changes step 1 over to;

Do you judge that the text participle finishes? if, according to braille word link writing rule word segmentation result is made amendment, generate the Braille of word segmentation result correspondence.

The comprehensive knowledge library module: required various knowledge bases when being the blind intertranslation of the Chinese comprise:

(1) electronic dictionary for Braille: comprise the electronic dictionary of Chinese character (60,000 speech) to the electronic dictionary of braille, braille (60,000 speech) to Chinese character, Chinese word segmenting dictionary etc.

(2) rule base: comprise Chinese braille word link writing rule, morphological rule, phrase rule, syntactic rule etc.

(3) statistical information storehouse: in order to reflect the Chinese context relation, the adjacent speech of Chinese that obtains with several hundred million word real corpus statistics connects dependence statistical knowledge etc. with showing between probability storehouse, part of speech.

Blind person's common keyboard input interface module: the blind person imports Chinese character or Braille with the common computer keyboard by blind person's input method of Chinese character and braille input method.Adopt prior art.

Sound identification module: dual mode is arranged

(1) unspecified person, continuous speech recognition are prior arts.

(2) keyword voice identification: the key word recognition that will import in the voice flow is come out, and is convenient to speaker's semantic understanding.Be used to differentiate the computer command of the various sayings that the blind person sends.It is prior art.

Natural language generation module: according to the mutual content of control, as when needing voice suggestion or inquiry, produce and have Chinese sentence intonation, that the blind person easily understands.What adopt at present is the voice that record in advance according to content choice, plays.

Control module: be the overhead control between above several input, the output channel.The dialogue management layer is the core of system, it is organized the whole session process according to certain dialog strategy, is responsible for the communication between each module, make the reaction of system according to corresponding decision rule, so that man-machine interaction is normally carried out under Expected Results.

Control module is by state analyzer, and dialog manager and state storehouse are formed.

The state of storing in the state storehouse comprises system state and dialogue state.System state is described the situation (as: program name, ongoing operation, operation requirement etc.) that the current application program module starts and moves with certain data structure, has also comprised the pattern of the input and output of using simultaneously.Dialogue state has reflected the situation of current man-machine interaction process.Because the restriction of the form of system operation order, dialogue state is by (env; Act; Obj; Condition) case-frame represents.

Dialog manager is made corresponding dialogue action by dialogue state is analyzed, and perhaps finishes system acting, perhaps carries out corresponding system prompt.Dialog manager adopts present general slot-filling algorithm to realize corresponding dialog strategy, manages and dispatches in order to the process to dialogue.Prior art.

State analyzer is responsible for the multi-mode input of the system that accepts, and selects following action to carry out according to current system state: start dialog manager; Start the corresponding application module; Send message to application program module; Transfer the I/O control to application program module.The final state analyzer converts input to canonical form and is put in the system state storehouse.Prior art.

The present invention can realize following function:

1. the present invention's input has three passages: common keyboard input, OCR input and phonetic entry; Output has three

Passage: voice output, printout and point show device output.Input and output following several modes capable of being combined:

1) the braille computing machine is imported, and (the apparent device output of point, display are exported, printer is defeated in Chinese-character text output

Go out).The document that will have on the braille paper under voice suggestion helps becomes electronic edition by the OCR converter

Document or by keyboard input and editor's braille document, by blind Chinese translation function convert thereof into into

The Chinese character document.By normal printer or display output.Used module and order are: the OCR conversion,

Input interface, blind Chinese converter, natural-sounding understanding, speech recognition, phonetic synthesis, language are given birth to

One-tenth, comprehensive knowledge base, control interface, printer output.(blind person and person of good sense use alternately)

2) braille computing machine input, braille text output (printer output).Will be under voice suggestion helps

There is document on the braille paper to become the electronic edition document or by keyboard input and editor by the OCR converter

The braille document.Directly show device output by braille printer or point.Used module and order are: OCR

Conversion, input interface, control interface, natural language understanding, speech recognition, phonetic synthesis, language

Speech generates, point shows device output, braille printer output.(blind person and blind person use alternately)

3) Chinese-character text input, (point shows device output, printer output in braille output.Help in voice suggestion

To have document on the Chinese character paper down becomes the electronic edition document by the OCR converter or is imported by keyboard

And editor's Chinese character document, convert thereof into by the blind translation function of the Chinese and to be the braille document.Beat by braille

Seal machine or point show device output.Used module and order are: OCR conversion, input interface, control connect

Mouth, the blind conversion of the Chinese, natural language understanding, speech recognition, phonetic synthesis, language generation, point show

Device output, comprehensive knowledge base, braille printer output.(annotate: blind person's teaching and braille publishing are used)

4) Chinese-character text input, Chinese character output (display, printer output).Will under voice suggestion helps

Document on the existing Chinese character paper becomes the electronic edition document by the OCR converter or is imported and compiled by keyboard

Collect the Chinese character document.Directly by normal printer or display output.Used module and order are: OCR

Conversion, input interface, control interface, natural language understanding, speech recognition, phonetic synthesis, language

Speech generates, printer output.(blind person and person of good sense use alternately)

2. braille Chinese character auto-conversion function: the braille document is converted into the Chinese character document automatically.Used module is: blind

Chinese conversion, comprehensive knowledge base.

3. Chinese character braille auto-conversion function: the Chinese character document is automatically converted to the braille document.Used module is: the Chinese is blind

Conversion, comprehensive knowledge base.

4. the blind person listens and reads Chinese-character text (novel, magazine, newspaper, Chinese character mail), and used module and order are: OCR

Conversion, control interface, speech recognition, phonetic synthesis, language generation, natural language understanding.

5. the blind person uses email manager: but blind person's sending and receiving Email, and read aloud the mail of receiving and write

Mail.Relate to input of blind person's Voice Navigation, Braille or Chinese character and Chinese-character text output, document is read aloud

Function.Used module is: input interface, speech recognition, natural-sounding understanding, inference system, control connect

Mouth, natural language generation, phonetic synthesis, the blind conversion of the Chinese, comprehensive knowledge base, natural-sounding are understood, point shows

Device output, printer output.

6. the blind person uses browser: the various information on blind person's browse network.Use blind person's Voice Navigation, blind person's meter

The function of reading aloud that input of calculation machine and Chinese-character text output, blind person listen Chinese-character text.Used module is: input

Interface, speech recognition, natural-sounding understanding, inference system, control interface, natural language produce, voice

Synthetic, the blind conversion of the Chinese, comprehensive knowledge base, natural-sounding are understood, point shows device output, printer output.

7. braille file manager: the mode with the order bar helps braille managing queries file.Used module is: defeated

Incoming interface, inference system, control interface, natural language generation, phonetic synthesis, the blind conversion of the Chinese, comprehensively know

Know storehouse, natural-sounding understanding.

8. blind person's Voice Navigation: the blind person can use a computer and network freely.Every menu and hot key may command

Order all can exhale order to replace with mouth, simultaneously can mouth exhale and close mouse, close life such as phonetic entry

Order.Used module and order are: speech recognition, natural-sounding understanding, inference system, control interface, from

Right language produces, phonetic synthesis.

Characteristics of the present invention are: have multiple interactive mode, can select separately hardware configuration according to economic conditions and needs, that gives full play to the blind person when using a computer listens, says, touches ability.Make the blind person can be more natural selectively, operational computations machine more easily.Compare with traditional man-machine information interaction means, this system has adopted multimodal interactive mode.The user both can use keyboard, mouse, blind person to put traditional interactive modes such as apparent device, display, also can adopt new interaction techniques such as voice and OCR simultaneously.Make reciprocal process hommization more, intellectuality.Give blind person's document process, exchange with the normal person, school for the blind's teachers ' teaching provides instrument.

Brief Description Of Drawings:

Fig. 1 is that blind person's computer system of the present invention constitutes synoptic diagram.

Fig. 2 is embodiments of the invention Braille input synoptic diagram.

Fig. 3 is a present embodiment blind person editing machine synoptic diagram.

Fig. 4 uses the email manager synoptic diagram for the present embodiment blind person.

Fig. 5 uses the browser synoptic diagram for the present embodiment blind person.

The intelligent Chinese computer system that a kind of blind person that the present invention proposes uses is described as follows in conjunction with each drawings and Examples:

A kind of embodiment that the present invention proposes is the minimum system that the blind person uses, its hardware comprises: the common personal computer that can surf the Net, the suitable Intel Pentium of basic hardware configuration requirement: CPU II is more than 400, more than the internal memory 128M, more than the hard disk 4G, sound card, microphone, loudspeaker or earphone and the required basic configuration of general computing machine.Basic software comprises: operating system Microsoft Windows9x or Windows 2000.

The composition and the course of work of present embodiment each several part are described in detail as follows: 1. keyboard input: dual mode can be arranged

(1) Braille input: international standard braille keyboard, use FDS, six keys of JKL are corresponding braille one side respectively, promptly

From left to right, six points from top to bottom.In proper order: 3 points in the first left side, from top to bottom, the right, back

3 points, from top to bottom.The phonetic entry prompting is arranged in the process of input, make the blind person know and oneself hit

Under be which key, send out what sound.

(2) Chinese phonetic alphabet input method: can select western language, words spelling, words Two bors d's oeuveres etc.In the process of input language is arranged

The sound input prompt, which key under making the blind person know oneself to hit is, sends out what sound.Can be by language

The candidate of polyphone is selected in the sound prompting.

Open or a newly-built braille file, promptly can import the braille idea.Open or a newly-built Chinese character file,

Promptly can import common Chinese character.

Characteristics are: except that each operation all has voice suggestion or response, can obtain corresponding Chinese converted contents in the input braille, as shown in Figure 2, be convenient to person of good sense (as: teacher) check and correction braille manuscript; Blind person and person of good sense's written communication.

2. read aloud Chinese-character text

To the Chinese character electronic document that has obtained, read aloud with phoneme synthesizing method.Open the Chinese character file, in the choice menus item " massage voice reading ", just can begin reading aloud of Chinese-character text, select this menu item will stop to read aloud once more.In addition can also read aloud automatically the menu in the menu bar of current cursor place.

Characteristics are: the blind person not only can listen and read electronic edition Chinese character document, also can read various forms by the OCR translation function simultaneously, as the Chinese character document of storages such as CD, books.

3. Voice Navigation

This system adopts the keyword recognition technology to realize Voice Navigation.Therefore, send when order can be with various close, more ambiguous statements.For example the user wants the al.txt that opens file, and he may say:

1) al.txt that opens file

2) file al.txt is opened

3) open al.txt

4) it is identical al.txt to be opened these four kinds of saying connotations, but the signal as phonetic entry just has a great difference, observing the common ground that these sayings can find out them is all to have a verb one " to open ", all with object-" filename " of a logical meaning.Also there is similar problem for sayings such as copy, deletions.This system finds out the object of one of verb crucial in the phonetic entry and important attribute thereof, finishes identification and affirmation to user input commands.The keyword recognition system generally is used in the situation of unspecified person, continuous speech.Employing is based on the keyword recognition method of HMM framework, and its principle is:

At first with the voice flow segmentation of input, every section corresponding and sentence or sentence length be the voice paragraph considerably.Which keyword then, searches in each section and determined whether keyword, be if there is keyword also must determine.The input of system is made up of keyword input and the outer voice of antistop list, and the latter is called rubbish, can comprise non-key speech, non-language (sucking mouth sound, breathing sound etc.) and ground unrest three parts.System sets up a cover HMM model for each keyword, in like manner also will set up some cover HMM to rubbish.The feature vector sequence of any one section input voice is obtained the status switch corresponding with this sequence with the Viterbi algorithm, if in the state of experience the person that belongs to the keyword is arranged, can detect the keyword of correspondence.

4. voice system control.Be characterized in: the special-purpose subsystem of several blind persons is integrated with Voice Navigation.Judge the residing duty of present system, carry out suitable operation according to the analyzing speech order of working environment.The controllable order of every menu and hot key all can exhale order to replace with mouth.Simultaneously can mouth exhale and close mouse, close orders such as phonetic entry.(beginning with particular key control phonetic entry) to avoid noise

Voice Navigation not only integrates a plurality of blind persons with subsystem, the interactive mode that makes things convenient for the close friend is provided for simultaneously these softwares, makes modern technologies such as the blind person can use a computer more freely, network, joins among the informationized society.

5. blind person's editing machine and braille printout

Blind person's editing machine is the editing machine that makes things convenient for the blind person to use, and it must have the basic function of general editing machine, and voice interactive function rightly is provided.This editing machine is based on Keyboard Control, and promptly the blind person controls current working state by keyboard.The blind person is an indispensable ingredient (as described above) of blind person's editing machine with Braille input method and Chinese character input method.The method for designing of editing machine is: in input process, after the end of input, or after opening certain electronic document, the blind person can learn current cursor position by voice suggestion, with which which row mark of row.The blind person can listen and read, edit, revise document, as deleting, add, duplicate literal, paragraph etc. under the help of voice.When running into phonetically similar word and can not recognize, the blind person can use explanation function, by phrase differentiate be which phonetically similar word as: red, pronunciation Hong, when looking into the meaning of word, computing machine will be used the voice informing user: red, red flag red; Red is red.Same flood, pronunciation Hong will be apprised of flood, the flood of flood.Can obtain the translator of English of this speech if desired by Chinese-English dictionary; If English, can read, can explain that the Chinese of this English looks like by english Chinese dictionary as needs.At last, the blind person can select to read aloud continuously, listens and reads content in full.So the Core Feature of blind person's editing machine is: State Control, exercisable function difference under different states, as, can not arbitrarily delete under the file management state, cursor leaves the file operation district, and prompting or help user return; Read aloud; Keyboard or voice control by the order of various keyboard operations or phonetic entry, help the user to finish document and listen the task of reading, understand, writing.As shown in Figure 3.

1) State Control:

Monitor the cursor current location, avoid system to carry out illegal operation.

2) read aloud:

Report current cursor position (which row, which row); Read aloud cursor left side letter or Chinese character, the cursor right side

Letter or Chinese character; Can make an explanation to Chinese character in case of necessity, centering, english make an explanation, translate: from working as

Read continuously the front position, stops.

3) read aloud automatically:

Automatically read when finishing cursor left, read automatically during cursor right, read the right when cursor up down is moved automatically, light

Put on and read the left side when moving down automatically.

Native system can directly connect the braille printer, as: the INDEX BRAILLE series of products of the Index Embossers company of Sweden Sweden, carry out Braille to current braille electronic document and print.

6. the blind person uses email manager: as shown in Figure 4.

Under voice control, in the ordinary electronic E-Mail Manager, add Voice Navigation, massage voice reading, the blind Chinese, the blind translation function of the Chinese.This blind person can finish the basic function of writing, send and receive e-mail, set up common email managers such as address book with email manager.Can be the blind person and read aloud the mail of receiving, writing.Blind users relies on voice and system interaction to finish the operation of sending and receiving Email.For example: computing machine is informed user's " you have new mail " by sound; Inquiry " your receiving emails? ", " will read aloud mail? " Prompting " please import receiver's address ", " please import e-mail theme " etc.After each operation corresponding voice answer-back or voice suggestion are arranged all.

7. the blind person uses browser: as shown in Figure 5.

Under voice control, in generic browser, add Voice Navigation, massage voice reading, OCR conversion, the blind Chinese, the blind translation function of the Chinese.This blind person can finish the basic function of general browsers such as inquiry, reading with browser.Can be the blind person and read aloud web page contents.People user relies on voice and system interaction to finish operations such as keyword, network address input, inquiry.For example: computing machine is informed user " you arrived so-and-so webpage " by sound; Inquire " you want what is inquired about? " Prompting " please import keyword ", " please import network address " etc.After arriving named web page, put in order, read aloud also content of net according to webpage.Can skip by keyboard and finish to read aloud.

8. intelligent inference function

This belongs to medium-scale towards blind person's man-machine information interaction system, and the theme of application is the various common operations and the simple information inquiry of computing machine.Because main blind man uses, so real-time requires better, the friendly degree of use is higher.Consider these factors, take based on the semantic description system of case grammar and the analytical algorithm of mating based on robust mode.Form with case frame (Case Frame) is carried out semantic expressiveness to discourse content, a case frame comprises a notion and relevant attribute (being groove) thereof, (Recursive TransitionNetworks, RTN) the possible linguistic form to these grooves is described with recursive transition network.When analyzing, use top-down RTNchart (chart) analytical algorithm that sentence is mated, as import sentence and the word outside the system dictionary occurred, give elimination and do not do analysis, for asyntactic composition in the input, directly skip during analysis, search can constitute the segment (being significant phrase) of notion.Carry out the search of Viterbi beam, obtain the result of ultimate analysis according to certain evaluation mark.Be mapped to semantic frame by the phrase that analyzes, so just obtained one or several such case frame and represented for sentence.Interaction content is carried out performance analysis, and the real-time update system state.

Simultaneity factor is in time made the theme prediction, estimates the next action of user, optimizes the knowledge base searching algorithm.Different interactive strategies is specified and carried out to the demand of analysis user.Predict and induce according to user's past behavior and current behavior, accelerate the realization of systematic search function, avoid mouth to exhale mistakes such as the identification of order and explanation to make system enter endless loop.

The present invention has set up has tens required comprehensive knowledge bases of the blind intertranslation of the Chinese, the theory that Chinese natural language is understood is applied in the braille technology for automatically treating first, finished the blind Chinese of Chinese, the blind automatic conversion of the Chinese, the input editing of collection braille Chinese, blind person are controlled in the intelligent computer systems towards Chinese blind person of one with Email sending and receiving management, voice system.With the artificial intelligence representation of knowledge and reasoning, the theme prediction, the content analysis scheduling theory is applied to the system state analysis, makes it have certain voice human-computer interaction function, and can utilize man-machine conversation, and system points out the user and induces, and is user-friendly.

Claims (1)

1, the intelligent Chinese computer system used of a kind of blind person, constitute by hardware and software module, it is characterized in that, said hardware is mainly by display, keyboard, sound card, network interface card or modulator-demodular unit, the main frame that the common personal computer that can surf the Net is formed, microphone, audio amplifier or the earphone that links to each other with each interface of this main frame, scanner, blind person use the apparent device of point, printer; Said software module comprises: OCR module, keyboard for blind person input and editor module, voice input module constitute three kinds of input channels; Show device output module, printer output module, phonetic synthesis output module, three kinds of output channels of formation by point; And the blind conversion module of the Chinese, comprehensive knowledge library module, input interface module, the natural-sounding that are connected between said each input channel and the output channel are discerned Understanding Module, natural language generator module, voice operation demonstrator module, inference system module, control interface module; These whole software modules are arranged in said main frame and the related hardware.

Images