CN1506871A - Sign language translating system - Google Patents

Sign language translating system Download PDF

Info

Publication number
CN1506871A
CN1506871A CNA02154526XA CN02154526A CN1506871A CN 1506871 A CN1506871 A CN 1506871A CN A02154526X A CNA02154526X A CN A02154526XA CN 02154526 A CN02154526 A CN 02154526A CN 1506871 A CN1506871 A CN 1506871A
Authority
CN
China
Prior art keywords
sign language
signal
terminal computer
voice
language interpretation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA02154526XA
Other languages
Chinese (zh)
Inventor
徐晓毅
姜平
陈绩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CNA02154526XA priority Critical patent/CN1506871A/en
Publication of CN1506871A publication Critical patent/CN1506871A/en
Pending legal-status Critical Current

Links

Images

Abstract

The sign language translating system includes mainly a device connected to a preliminary processing circuit to convert the hand bending characteristic or the sign language action into electronic signal; and a terminal computer comprising phonetic system, display system and phonetic synthesis system. The device to convert the sign language action into electronic signal is several stalls, each of which has one sensor fixed to the finger joint to convert the finger bending characteristic into electronic signal. The preliminary processing circuit includes bridge circuits in the amount corresponding to that of sensors and converting the varying sensor resistance into electronic signal, one signal amplifier circuit, one A/D converter, and one preliminary monochip computer with serial port for obtaining digital quantity and containing with the terminal computer. The sensor is constituted via setting one strain gage onto one thin metal sheet.

Description

A kind of sign language interpretation system
Technical field
The present invention relates to the translation system that a kind of sign language with the deaf-mute converts voice or literal to, relate in particular to the portable sign language translation system that a kind of sign language with the deaf-mute converts voice or literal to.
Background technology
The deaf-mute can only express the wish of oneself to other people by its sign language, and the normal people of most physiology seldom has the sign language learned, and therefore, is not easy to understand deaf-mute's wish and requirement because of its physiological defective.Usually the phenomenon that can see is to exchange by paper and pen.But this method is very inconvenient, owing to can not write too many content, therefore exchanges content and is limited to.
Summary of the invention
The object of the present invention is to provide the portable sign language interpretation system of a kind of glove-type, sign language interpreter can be become voice that the normal person can hear or the literal that can see.
For achieving the above object, technical scheme of the present invention is as follows:
The bending change Feature Conversion of hand is become electric signal, and to this signal process shaping and amplification, A/D converts digital quantity to, carries out subsequent treatment by computing machine by the signal preposing signal process circuit.Signal preposing signal process circuit of the present invention mainly includes bridge circuit, signal amplification circuit, A/D change-over circuit and preposition single-chip microcomputer of input sensor signal.Subsequent process circuit to electric signal is a terminal computer, include voice system, liquid crystal display display system and speech synthesis system, wherein voice system is voice control synthesis chip, with the phonetic element fragmented storage recording in advance, classify, make in the sound bank storage unit.After preposition single-chip microcomputer is determined certain voice, the sound bank storage unit is carried out composition algorithm computing and relevant inquiring and call by chip controls.The speed of this operation satisfies normal person's normal word speed fully.
Utilize serial ports to carry out communication between signal preposing signal process circuit and the terminal computer, its communications protocol satisfies the requirement of a large amount of classification of transmission real time datas, converts voice that the normal person can understand to or displaying contents on display screen by terminal computer.
The present invention is that sensor is fixed on relative each joint of hand on the gloves with the device that the finger spelling action is transformed into electronic signal, as certain joint or the wrist joint place of finger, forms data glove.The structure of this sensor is strainometer to be arranged on the foil form, and this strainometer has that deflection is big, and fatigue lifetime is long, good linearity, the characteristics that temperature effect is little.Each strainometer is respectively as the part of bridge circuit separately, by the mode input signal preposing signal process circuit of difference.
The preposition single-chip microcomputer of signal preposing signal process circuit of the present invention obtains the valid data of described data glove by serial ports, and controls voice system and carry out the conversion of literal code to voice.The electric signal of this preposing signal process circuit is very little with the variable quantity of maximum actuation amplitude, its variable quantity absolute value is about 0.1mv, for the correct such tiny signal of perception, guarantee not to be subjected to the influence of various extraneous interference again, the signal condition amplifying circuit that the present invention adopts, its enlargement factor can reach more than 10,000 times, and temperature is floated less than 5ppm.
The core devices of A/D converter of the present invention is a high speed serialization A/D chip, and it requires one to be to guarantee real-time requirement of the present invention, the 2nd, and pin should be less, can reduce the volume of electronic installation.
Signal preposing signal process circuit of the present invention can be divided into two kinds:
A kind of is mancarried device, be located in wrist, is connected with data glove, and is connected with portable computer by a cable.Here said portable computer is a single-chip microcomputer.
A kind of is that this signal preposing signal process circuit is arranged on desk-top computer, connect with data glove by cable, and carry out analyzing and processing and pattern-recognition according to the data of data glove, and finally convert word, speech and complete sentence to, also can on screen, show or sound.
Voice system of the present invention, its core are voice control synthesis chip, and the phonetic element of recording in advance, classify, making is stored in the sound bank storage unit.After preposition single-chip microcomputer is determined certain voice, by chip controls the coding of voice or voice string is passed to chipspeech, finish the phonetic synthesis algorithm by chipspeech again and to the segmented request of sound bank storage unit with call.Various language can be stored in the sound bank of the present invention, as the language and the literal of domestic Han nationality and other ethnic group, various foreign language and literal can also be stored, as English, Japanese, German, French, Korean, Russian, Spanish etc.
Description of drawings
Fig. 1 is a principle of the invention synoptic diagram.
Fig. 2 is the data glove structural representation of embodiment.
Fig. 3 a, Fig. 3 b, Fig. 3 c are respectively sensor construction synoptic diagram in the data glove of embodiment.
Fig. 4 is the strain transducer bridge circuit of embodiment.
Fig. 5 is the electrical schematic diagram of the signal amplification circuit of embodiment.
Fig. 6 is the electrical schematic diagram of whole signal amplification circuit.
Fig. 7 is the chip synoptic diagram of A/D change-over circuit.
Fig. 8 is the electrical schematic diagram of the A/D change-over circuit of embodiment.
Fig. 9 is the computer interface circuit of embodiment.
Figure 10 is the serial ports output circuit of embodiment.
Figure 11 is a signal pre-process main flow chart.
Figure 12 is the serial ports interrupt service routine main flow chart of A/D converter and preposition single-chip microcomputer.
Figure 13 is preposition single-chip data acquisition process flow diagram.
Figure 14 carries out pattern-recognition for terminal computer, data vector is changed into the process flow diagram of action coding.
Figure 15 is the process flow diagram of terminal computer with initial consonant and simple or compound vowel of a Chinese syllable synthetic word, speech and phrase.
Figure 16 receives process flow diagram for the terminal computer serial data.
Figure 17 judges main flow chart for terminal computer sound.
Embodiment
See also Fig. 1.Basic technical scheme of the present invention is as follows:
When the sign language action of user by some agreement, the resistance value that is worn over the electronic installation on its hand changes, and the signal of this variation is amplified by signal amplifier, and is handled by preposition single-chip microcomputer after the A/D conversion; Signal after will being handled by communications protocol reaches terminal computer again and handles, and correspondingly sends out in modes such as literal or voice.Fig. 1 has shown ultimate principle of the present invention.The device that is used for the action of deaf-mute's finger spelling is transformed into electronic signal is a data glove.These gloves are fingerstall in this example, can certainly make the wrister form of wearing in the similar sports.The whippy Lycra lining of this fingerstall 1 usefulness is made, and is fixed with sensor 2 on the fingerstall 1, and sensor 2 is with lead-in wire 3.As shown in Figure 2.
In conjunction with consulting Fig. 3.Sensor 2 on the fingerstall 1 is to be fitted in a foil 1 as the strainometer among Fig. 3 a 2, and lead-in wire 3 connects signal pre-process plate.Sensor 2 of the present invention can also have Fig. 3 b and two kinds of forms of Fig. 3 c, promptly use the strainometer of two or more weak points to be attached on same the foil 1, can be as gathering the bending change amount in the second or the 3rd joint of finger among Fig. 3 a, also can gather the variable quantity in finger the second and the 3rd joint simultaneously, and obtain the crooked variable quantity that produces of whole finger as among Fig. 3 b the variable quantity in two joints being cascaded, the input preposing signal process circuit, also can be as among Fig. 3 c the bending change amount in each joint being picked out the input preposing signal process circuit separately.The strainometer 2 that this example adopts is the strainometer that Japan produces, and foil 1 is the sheet steel of 0.8mm.Fingerstall 1 in this example has 5, is worn over any 5 finger-joint places on hand respectively, and is all passable as for which joint that is worn over finger.When finger is crooked, the resistance value of strainometer 2 is changed, the resistance variations of generation is delivered to preposing signal process circuit.
Signal preposing signal process circuit of the present invention mainly includes bridge circuit, signal amplification circuit, A/D change-over circuit and the preposition single-chip microcomputer with the corresponding input sensor signal of fingerstall 1 quantity.
See also Fig. 4.In the present invention, 5 sensors 2 are imported the bridge circuit of strain transducer respectively as the part of bridge circuit separately by the mode of difference.Because the electric signal of strainometer 4 inputs is very little with the variable quantity of maximum actuation amplitude, its variable quantity absolute value is about 0.1mv, for the correct such tiny signal of perception, guarantee not to be subjected to the influence of various extraneous interference again, this example is used amplifier MAX4078, enlargement factor reaches more than 10,000 times, and temperature is floated less than 5ppm.Figure 5 shows that the amplifying circuit that a fingerstall connects.Fig. 6 is whole signal amplification circuit figure.
Amplifying signal connects the A/D change-over circuit, what this example adopted is a high speed serialization 88 passage A/D chip (MAX1110), it guarantees real-time requirement of the present invention on the one hand, on the other hand because the characteristics of serial chip are that pin is less, thereby has reduced electronic installation volume of the present invention.A/D change-over circuit chip is seen Fig. 7, and the electric principle of A/D change-over circuit is seen Fig. 8.
The numerical information that obtains after the A/D conversion reaches preposition single-chip microcomputer by serial ports, said preposition single-chip microcomputer is relative aftermentioned terminal computer, this preposition single-chip microcomputer mainly is the numerical information that the sensor 2 on all fingerstall 1 of circle collection obtains after preposing signal process circuit, A/D conversion, utilize valid data to carry out pattern-recognition, convert numerical information to code element that word, speech etc. are of practical significance, then these words, word combination are become complete sentence.This preposition single-chip microcomputer also will be controlled voice system and carry out the conversion of literal code to voice.Preposition single-chip microcomputer in this example is a slice 89C51 single-chip microcomputer, and this chip is characterized in that volume is little as shown in Figure 9, and powerful, computing velocity is fast, and clock frequency is up to 24MHz, the not enough lus of average instruction execution cycle.About the serial ports output circuit between A/D change-over circuit and the preposition single-chip microcomputer is seen Figure 10, the main flow of its work of signal preposing signal process circuit such as Figure 11 and shown in Figure 12.Signal preposing signal process circuit described in this example is a kind of mancarried device, be located in wrist, is connected with data glove, and is connected with terminal computer by a cable.
The another kind of method to set up of this signal preposing signal process circuit is arranged in the terminal computer, connects with data glove by cable, and carries out analyzing and processing and pattern-recognition according to the data of data glove.This set is a known technology in terminal computer, and this example is not done too much description.
This signal preposing signal process circuit is to adopt the RS232 communications protocol with the multicore flexible cable terminal computer that interconnects by preposition single-chip microcomputer, and this terminal computer is the same with preposition single-chip microcomputer, is a slice 89C51 single-chip microcomputer.Its communications protocol satisfies the requirement of a large amount of classification of transmission real time data.The computing machine that terminal connects includes voice system, liquid crystal display display system and speech synthesis system, wherein voice system is a voice control synthesis chip XF1M01, comprise phonetic synthesis algorithm and jumbo sound bank storage unit, the phonetic element of recording in advance, classify, making is stored in the sound bank storage unit.After preposition single-chip microcomputer is determined certain voice or voice string, by chip controls voice coding is passed to chipspeech, finish the phonetic synthesis algorithm and to the inquiry of sound bank storage unit with call by chipspeech, finally convert word, speech and complete sentence to, also can on screen, show or sound.The speed of this operation satisfies normal person's normal word speed fully.The present invention can use various language and literal, and this itself is the technology that this area colleague is easy to imagine, does not describe in detail at this.
5 strainometer signals on the described data glove of data acquisition software part scan round, the A/D chip by serial obtain 5 bit digital information after described 5 strainometer signals amplify through conditioning.5 numerical informations of this that collects are formed the vector data of 5 bytes successively at every turn.Then the requirement according to communications protocol after the described vector data pre-service is packaged into a complete packet.Serial port by preposition single-chip microcomputer sends to terminal computer at last, and its main flow is seen Figure 13.
Preposition single-chip microcomputer whenever receives a complete packet all will carry out decode operation, just according to the requirement of communications protocol packet be separated in the other direction to read through, and the valid data that decoding obtains temporarily leave in the storer.
The mode identificating software part is actual to be a kind of fuzzy algorithm.Be exactly that obscurity model building is all carried out in each action of deaf-mute's finger spelling in advance concretely.During pattern-recognition, computing machine will obtain decoded valid data automatically, be actually the data vector of 5 bytes, compare by the predetermined fuzzy factor and fuzzy vector model, thus the actual act of judgement deaf-mute finger spelling.Data vector also needs the process of a fuzzy recognition to actual confirming operation, and its essence is by each change fuzzy operator, makes that real action is noted effectively accurately.Finger spelling action of the present invention is exactly to store by being converted to voice coding after this fuzzy algorithm identification, sees Figure 14.
The language translation algorithm comprises deaf-mute's finger spelling input algorithm and language composition algorithm.
Deaf-mute's finger spelling input algorithm, concretely, it is individual character commonly used with the deaf-mute, be decomposed into initial consonant, simple or compound vowel of a Chinese syllable, sometimes also comprise the combination of tone, just become the finger spelling action group one to comprise the finger spelling action of representing initial consonant, the finger spelling of back to back expression simple or compound vowel of a Chinese syllable moves, and then represents the sequence of the finger spelling action of tone sometimes.Computing machine carries out analysis-by-synthesis to this action sequence, just by finger spelling input algorithm, converts these action sequences to individual character, converts voice signal to, drives sound-producing device, perhaps stores, and sees Figure 15.
The serial data of this terminal computer receives process flow diagram and sees Figure 16.Terminal computer software is realized Data Receiving, analysis, processing, and the function of speech utterance.The serial data form is unit with the byte, and per 9 bytes are one group, and No. 5 sensors of representing the front end data gloves are all over patrolling once, and last byte is a flag byte<0FFh 〉, represent that one group of data ends.
Every group of data that terminal computer receives will compare with the existing data in the character database, to determine the character representation of current data group, thereby form the character string of a definite sequence, terminal computer obtains the orthoepy of each individual character, the pronunciation equipment pronunciation that utilizes the sound control to connect by sound card at last according to the order characteristics of this character string in conjunction with sound database and dictionary table commonly used.The process flow diagram that this process sound is judged is seen Figure 17.

Claims (12)

1, a kind of sign language interpretation system, it mainly consists of:
Sign language action is transformed into the device of electronic signal, connects a signal preposing signal process circuit, become electric signal with bending change Feature Conversion with hand; One terminal computer includes language analysis system, speech synthesis system, display system and sonification system;
The device that wherein the sign language action is transformed into electronic signal is some fingerstall, and its corresponding hand joint is fixed with a sensor, becomes electric signal with the bending change Feature Conversion with hand;
This signal preposing signal process circuit includes:
With the corresponding bridge circuit of number of sensors, be converted to electric signal with the resistance value that sensor is changed;
One signal amplification circuit is to amplify above-mentioned signal;
One A/D converter becomes digital quantity with the electrical signal conversion of amplifying;
One preposition single-chip microcomputer obtains the digital quantity of A/D converter by serial ports, and by the serial ports terminal computer that interconnects, aforesaid sign language interpreter is become voice or literal;
Described sensor is a strainometer to be arranged on the foil form.
2, singlehanded sign language interpretation system as claimed in claim 1 is characterized in that, described fingerstall is to be worn over the left hand or the right hand.
3, sign language interpretation system as claimed in claim 1 is characterized in that, described hand joint is each joint of finger or wrist joint.
4, sign language interpretation system as claimed in claim 1 is characterized in that, described voice system is a voice control synthesis chip.
5, as claim 1 or 4 described sign language interpretation system, it is characterized in that, the core of described voice system is a chipspeech, it is to go to call the phonetic element that is stored in the sound bank storage unit by composition algorithm, after system determines certain voice or voice string, by voice coding chipspeech is controlled, chipspeech is finished relevant compose operation, and finally sends corresponding sound control signal.
6, sign language interpretation system as claimed in claim 1 is characterized in that, the display system of described terminal computer is a display screen.
7, sign language interpretation system as claimed in claim 1 is characterized in that, the sonification system of described terminal computer is a loudspeaker.
8, sign language interpretation system as claimed in claim 1 is characterized in that, described signal preposing signal process circuit be located in wrist, is connected with data glove, and is connected with terminal computer by a cable.
9, sign language interpretation system as claimed in claim 1 is characterized in that, described signal preposing signal process circuit is arranged in the terminal computer.
As claim 1,8 or 9 described sign language interpretation system, it is characterized in that 10, described terminal computer is a single-chip microcomputer or desktop computer.
As claim 1 or 5 described sign language interpretation system, it is characterized in that 11, described language and literal are various language and literal.
12, sign language interpretation system as claimed in claim 1 is characterized in that, described sign language is deaf-mute's sign language.
CNA02154526XA 2002-12-06 2002-12-06 Sign language translating system Pending CN1506871A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNA02154526XA CN1506871A (en) 2002-12-06 2002-12-06 Sign language translating system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNA02154526XA CN1506871A (en) 2002-12-06 2002-12-06 Sign language translating system

Publications (1)

Publication Number Publication Date
CN1506871A true CN1506871A (en) 2004-06-23

Family

ID=34235518

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA02154526XA Pending CN1506871A (en) 2002-12-06 2002-12-06 Sign language translating system

Country Status (1)

Country Link
CN (1) CN1506871A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101794528A (en) * 2010-04-02 2010-08-04 北京大学软件与微电子学院无锡产学研合作教育基地 Gesture language-voice bidirectional translation system
CN102362709A (en) * 2011-09-13 2012-02-29 苏州市伦琴工业设计有限公司 Interpreting glove for deaf and dumb sign language and interpreting method thereof
CN101609618B (en) * 2008-12-23 2012-05-30 浙江工业大学 Real-time hand language communication system based on special codes
CN101577062B (en) * 2008-12-30 2012-07-25 浙江工业大学 Space encoding-based method for realizing interconversion between sign language motion information and text message
CN105096696A (en) * 2015-07-31 2015-11-25 努比亚技术有限公司 Sign language translation apparatus and method based on intelligent bracelet
CN106157749A (en) * 2015-04-07 2016-11-23 陈斌 Sign language transducer
CN109032333A (en) * 2018-06-24 2018-12-18 佛山凯舒易智能科技有限公司 A kind of sign Language Recognition and method based on optical fiber perception

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609618B (en) * 2008-12-23 2012-05-30 浙江工业大学 Real-time hand language communication system based on special codes
CN101577062B (en) * 2008-12-30 2012-07-25 浙江工业大学 Space encoding-based method for realizing interconversion between sign language motion information and text message
CN101794528A (en) * 2010-04-02 2010-08-04 北京大学软件与微电子学院无锡产学研合作教育基地 Gesture language-voice bidirectional translation system
CN101794528B (en) * 2010-04-02 2012-03-14 北京大学软件与微电子学院无锡产学研合作教育基地 Gesture language-voice bidirectional translation system
CN102362709A (en) * 2011-09-13 2012-02-29 苏州市伦琴工业设计有限公司 Interpreting glove for deaf and dumb sign language and interpreting method thereof
CN102362709B (en) * 2011-09-13 2013-08-14 苏州市伦琴工业设计有限公司 Interpreting glove for deaf and dumb sign language and interpreting method thereof
CN106157749A (en) * 2015-04-07 2016-11-23 陈斌 Sign language transducer
CN105096696A (en) * 2015-07-31 2015-11-25 努比亚技术有限公司 Sign language translation apparatus and method based on intelligent bracelet
CN109032333A (en) * 2018-06-24 2018-12-18 佛山凯舒易智能科技有限公司 A kind of sign Language Recognition and method based on optical fiber perception

Similar Documents

Publication Publication Date Title
US5353377A (en) Speech recognition system having an interface to a host computer bus for direct access to the host memory
CN110364171A (en) A kind of audio recognition method, speech recognition system and storage medium
WO2020238045A1 (en) Intelligent speech recognition method and apparatus, and computer-readable storage medium
CN102831195B (en) Personalized speech gathers and semantic certainty annuity and method thereof
CN109063624A (en) Information processing method, system, electronic equipment and computer readable storage medium
CN111695338A (en) Interview content refining method, device, equipment and medium based on artificial intelligence
CN107221344A (en) A kind of speech emotional moving method
CN111460175A (en) SNOMED-CT-based medical noun dictionary construction and expansion method
CN1506871A (en) Sign language translating system
JP2980026B2 (en) Voice recognition device
JPH0372997B2 (en)
WO2023272616A1 (en) Text understanding method and system, terminal device, and storage medium
Jelinek A real-time, isolated-word, speech recognition system for dictation transcription
WO1983002190A1 (en) A system and method for recognizing speech
KR100372850B1 (en) Apparatus for interpreting and method thereof
CN115019820A (en) Touch sensing and finger combined sounding deaf-mute communication method and system
CN212675912U (en) Automatic voice recognition system based on FPGA
CN111506764A (en) Audio data screening method, computer device and storage medium
CN108538152A (en) A kind of portable device for the exchange of deaf-mute's normal voice
CN113643692B (en) PLC voice recognition method based on machine learning
CN112668664B (en) Intelligent voice-based conversational training method
CN1206581C (en) Mixed input method
CN212461141U (en) Embedded intelligent voice controller
KR20010018532A (en) User Interface method using Hand-written character recognition and Speech Recognition Synchronous
CN2724095Y (en) Device for intelligent collection, language inter translation and voice synthesis

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication