CN201440733U - Mobile speech communication terminal suitable for person with language barrier - Google Patents

Mobile speech communication terminal suitable for person with language barrier Download PDF

Info

Publication number
CN201440733U
CN201440733U CN2009201663228U CN200920166322U CN201440733U CN 201440733 U CN201440733 U CN 201440733U CN 2009201663228 U CN2009201663228 U CN 2009201663228U CN 200920166322 U CN200920166322 U CN 200920166322U CN 201440733 U CN201440733 U CN 201440733U
Authority
CN
China
Prior art keywords
text information
sign language
chip
chipset
call
Prior art date
Application number
CN2009201663228U
Other languages
Chinese (zh)
Inventor
陈绍君
龚云云
Original Assignee
深圳市同洲电子股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市同洲电子股份有限公司 filed Critical 深圳市同洲电子股份有限公司
Priority to CN2009201663228U priority Critical patent/CN201440733U/en
Application granted granted Critical
Publication of CN201440733U publication Critical patent/CN201440733U/en

Links

Abstract

The utility model provides a mobile speech communication terminal suitable for a person with language barrier. A camera lens captures a sign language image, a sign language resolving chip set resolves the sign language image into a picture track and matches the picture track with that of standard sign language in a storage device so as to obtain vague text information corresponding to the matched picture track, a text information error-correcting chip set utilizes grammar and word combination parameters in the storage device to carry out error correction on the vague text information, a CPU (central processing unit) converts the correct text information into an audio signal and sends the audio signal to the opposite mobile speech communication terminal, and then the audio signal sent by the opposite mobile speech communication terminal is received and is output to a display screen for displaying after being converted into the text information. By applying the above mobile speech communication terminal, the person with language barrier can use sign language to carry out synchronous and correct speech communication with a user at an opposite terminal of the speech communication.

Description

一种适用于语言障碍人士的移动通话终端 One for speech impaired call mobile terminal

技术领域 FIELD

[0001] 本实用新型涉及移动通信技术,特别涉及一种适用于语言障碍人士的移动通话终 [0001] The present invention relates to mobile communication technologies, particularly to a language suitable for a mobile call final impaired

丄山顺。 Shang Shan Shun.

背景技术 Background technique

[0002] 随着移动通信技术的迅速发展,移动通话终端成为人们日常生活中进行沟通交流的主要工具。 [0002] With the rapid development of mobile communication technology, mobile call terminals become the main tool of communication in daily life.

[0003] 目前普遍应用的移动通话终端都是将本端使用者的声音信号进行收音,然后转换为适于移动通信网络传输的信号后传输到对端,再由对端移动通话终端进行反向转换生成声音信号播放给对端使用者,显然这种普遍应用的移动通话终端并不适于语言障碍人士使用。 [0003] There is currently widely used is a mobile terminal call audio signal of the local radio user is then converted to a signal suitable for transmission a mobile communication network for transmission to the terminal, and then the reverse side of the mobile terminal call generates a sound signal converted to the end user to play, apparently talking mobile terminal, the ubiquitous not suitable language impaired users.

[0004] 专利申请号为200610121174.9的中国专利申请提出了一种解决方案,语言障碍 [0004] Patent Application No. 200610121174.9 of Chinese patent application proposes a solution to the language barrier

人士将要通话的内容以短信息的方式输入本端的移动通话终端,一路由本端的移动通话终 The person to call the contents of a short message input by way of the local mobile terminal call, all the way to end the call by the mobile terminal

端将该内容以短信息的方式发送到对端的移动通话终端,另一路由本端的移动通话终端中 The content transmission side in the manner of a short message to the mobile terminal of the calling side, the other end of the route of this call is a mobile terminal

的短信息音频转换器转换成音频信号发送到对端的移动通话终端,当对端的使用者也为语 SMS audio converter to convert the audio signal to the mobile terminal to end the call, when the user language is also peer

言障碍人士时可以通过阅读短消息来了解通话内容,当对端的使用者为正常人士时可以通 I can understand the conversation by reading a short speech disabilities message, when the end user is a normal person can pass

过收听音频来了解通话内容。 I had to learn to listen to audio conversations. 虽然该方案也解决了语言障碍人士使用移动通话终端进行通 Although the program also solves the language barrier people to use the mobile terminal through the call

话的问题,但是通话时仍需逐字输入文字信息,特别是在通话内容较多时将非常麻烦。 The question, then, but still need to enter verbatim text message during a call, especially when there are many phone conversations would be very troublesome.

[0005] 专利申请号为200510047240. 8的中国专利提出了一种为语言障碍人员提供数据 [0005] Patent Application No. 200510047240.8 of Chinese patent teaches a method of providing data to the language barrier staff

转换和传输的装置,该装置应用于电视接收机,存储有手语影像对应的文本信息和语音信 Conversion and transmission apparatus, the apparatus is applied to a television receiver, stores a sign language video and text information corresponding to the speech signal

息,当通过摄像部件接收到使用者的手语影像时,根据使用者的请求将该手语影像转换为 Message, when receiving a sign language video imaging means by the user, at the request of the user image into the sign language

文本信息或语音信息,还可以进一步将转换后的文本信息和语音信息传输给诸如移动通话 Text information and voice information or text information after transmitting voice information, it may further be converted to a call such as a mobile

终端或电脑等终端进行显示或播放。 Computer terminal or other device for display or playback. 虽然该方案给出了将手语影像转换为文本信息或语音 Although the program gives the image to convert sign language into text or voice message

信息,相比于直接输入文字信息来说方便了使用,但是首先该装置设置于电视接收机中,如 Information, compared to the direct input of character information is easy to use, but first the device is provided in the television receiver, such as

果通信双方的使用者位于不同地点,不仅双方都需要准备内置有该装置的电视接收机,同 If the communication between the two sides users in different locations, both sides need to be prepared not only built-in television receiver of the device, with

时还需准备其他接收信息的终端,同样非常麻烦且双方的交流也不同步,其次该装置将手 Need to prepare the terminal to receive other information, it is also very cumbersome and exchanges between the two sides is not synchronized, followed by the device will hand

语影像转换为文本信息或语音信息时仅简单的利用了预存的对应关系,假设使用者的手语 Language image into a text message or voice information simply by using the pre-stored correspondence relationship, assuming the user's sign language

输入不规范或者个别词语输入顺序错误,则将直接转换出错误的文本信息或语音信息,转 Input or irregular sequence of individual words in an input error, the error is directly converted text information or voice information, transferred

换的准确度较低。 In other low accuracy.

实用新型内容 SUMMARY

[0006] 本实用新型提供一种适用于语言障碍人士的移动通话终端,使得语言障碍人士可以使用手语与通话对端的使用者进行同步、准确的通话交流。 [0006] The present invention provides a terminal for mobile calls the speech impaired, speech impaired may be such that the sign language of the call end user synchronization and accurate exchange of call. [0007] 本实用新型的技术方案是这样实现的: [0007] The technical solution of the present invention is implemented as follows:

[0008] —种适用于语言障碍人士的移动通话终端,关键在于,该移动通话终端包括:中央处理器、显示屏、存储器、摄像头、手语解析芯片组和文本信息纠错芯片组;[0009] 所述摄像头与手语解析芯片组连接,将捕捉的手语影像传输给手语影像解析芯片组; [0008] - species suitable for the mobile terminal talking speech impaired, the key is, calls the mobile terminal comprising: a central processor, a display, a memory, camera, sign language parsing and text information correction chipset chipset; [0009] the camera and the sign language parsing chipset connected, the captured image is transmitted to the sign language of the sign language video analytic chipset;

[0010] 所述手语解析芯片组与存储器和文本信息纠错芯片组分别连接,将摄像头捕捉的手语影像解析为图形轨迹、并与存储器中预先存储的规范手语的图形轨迹进行匹配,得出相匹配的图形轨迹所对应的模糊文本信息传输给文本信息纠错芯片组; [0010] The sign language parsing chipset and the memory and the text information correction chipset are connected, the camera captures a sign language image analyze a graphic trajectory, and matched to regulate sign language graphics trajectory memory previously stored derived phase Fuzzy matching text pattern corresponding trajectory information to the text information correction chipset;

[0011] 所述文本信息纠错芯片组与存储器和中央处理器分别连接,利用存储器中预先存储的语法及组词参数对所述模糊文本信息进行纠错,得到准确文本信息传输给中央处理器; [0011] The text information correction chipset connected to a memory and a central processor, respectively, of the blur correction text information using syntax and parameters set of words stored in the memory in advance, accurate text information to the central processor ;

[0012] 所述中央处理器与摄像头和显示屏分别连接,在通话连接建立后向摄像头发送触发信号;将所述准确文本信息转换为音频信号向对端移动通话终端发送;接收对端移动通话终端发送的音频信号,转换为文本信息后输出到显示屏进行显示。 [0012] The central processor and the display are connected to the camera and sends the trigger signal to the camera after a call connection is established; the precise text information is converted to the audio signal transmitting calls to the peer mobile terminal; mobile terminal receiving the call after the audio signal sent by the terminal into text information to the screen display. [0013] 较佳地,所述手语解析芯片组包括:解析芯片和匹配芯片; [0013] Preferably, said analytic Chipset sign language comprising: matching analysis chip and the chip;

[0014] 所述解析芯片与摄像头和匹配芯片分别连接,将摄像头捕捉的手语影像解析为多个包含手型轮廓和运动趋势的图形轨迹,并按顺序传输给匹配芯片; [0014] The analysis chip are respectively connected with the camera chip and matching, the camera captures images parsed into sign language track pattern comprising a plurality of hand movement and trends shaped profile, matched to the chip sequence, in transmission;

[0015] 所述匹配芯片与存储器和文本信息纠错芯片组分别连接,在存储器预先存储的图 [0015] The memory chip matched text information error correction and chipset are connected, stored in advance in the memory of FIG.

形轨迹中分别查找到与解析芯片解析出的多个图形轨迹相匹配的图形轨迹,再从存储器中 Shaped track respectively parsed to find the pattern analysis chip plurality of tracks to match the graph track, then from the memory

读出相匹配的图形轨迹所对应模糊文本信息传输给文本信息纠错芯片组。 Read match pattern corresponding to the blur trajectory of the text information to the text information correction chipset.

[0016] 较佳地,所述文本信息纠错芯片组包括:参数读取芯片和纠错执行芯片; [0016] Preferably, the text information correction chipset comprising: a correction parameter read and execute chip chips;

[0017] 所述参数读取芯片与存储器和纠错执行芯片分别连接,从所述存储器中读取预先 [0017] The memory chip and the read parameter and performs error correction chip are connected, read from the memory in advance

存储的语法及组词参数传输给纠错执行芯片; Grammar and word parameters stored set of transmission to execute error correction chip;

[0018] 所述纠错执行芯片与手语解析芯片组和中央处理器分别连接,利用所述语法及组 [0018] The error correcting sign language parsing and execution chip chipset and a central processor connected by a set of syntax and

词参数,对所述模糊文本信息进行纠错,得到准确文本信息传输给中央处理器。 Word parameter, the blur correction text information, the text to give accurate information to the central processor.

[0019] 较佳地,该移动通话终端中还包括:麦克风、音频处理器和扬声器; [0019] Preferably, the calling mobile terminal further comprises: a microphone, a speaker, and an audio processor;

[0020] 所述中央处理器进一步与麦克风和音频处理器分别连接,接收从所述显示屏输入 [0020] The central processor is further connected to a microphone and an audio processor to receive input from the display screen

的模式选择信号,当所述模式选择信号指示当前为语言障碍工作模式通话时,继续执行所 A mode selection signal, when the mode selection signal indicating that the current operation mode call to the language barrier, to continue the

述向摄像头发送触发信号的操作;当所述模式选择信号指示当前为正常工作模式通话时, Said camera sends a signal to trigger operation; when the mode selection signal indicating that the current call is the normal operating mode,

向麦克风和音频处理器发送触发信号; Sending a trigger signal to the microphone and audio processor;

[0021] 所述麦克风与音频处理器连接,收集外部声音信号传输给音频处理器; [0021] The audio processor connected to the microphone for collecting external sound signals to the audio processor;

[0022] 所述音频处理器与扬声器连接,将所述麦克风收集的声音信号进行模数转换后传 [0022] The audio processor is connected to the rear speakers, the sound signal collected by the microphone of the analog to digital conversion transmission

输给对端移动通话终端;接收对端移动通话终端发送的音频信号,在进行数模转换后传输 Lost call peer mobile terminal; transmissions received after digital-analog conversion of the audio signal transmitted from the mobile terminal side call, performing

到扬声器播放。 To the speaker.

[0023] 可见,如果通话发起端为语言障碍人士,则通话发起端的移动通话终端可将语言障碍人士的手语影像转换为文本信息、再转换为音频信号传输给对端移动通话终端;如果通话接收端为语言障碍人士,通话接收端的移动通话终端可将收到的音频信号转换为文本信息进行显示;如果通话发起端为正常人士,则通话发起端的移动通话终端可直接将声音信号转换后传输给对端移动通话终端;如果通话接收端为正常人士,则通话接收端的移动通话终端可直接将音频信号转换为声音播放。 [0023] visible, if the call originating terminal is speech impaired, then the call initiation end mobile call terminal may convert the speech impaired sign language image into text information, and then converted into an audio signal transmitted to the mobile call terminal end; if a call received end is a speech impaired, an audio signal into the call receiving end mobile call terminals can be received as text information displayed; if the call originating terminal is a normal person, the call originating mobile call terminal end directly to the sound signal transmitted to the peer mobile terminal call; if the call receiving end is a normal person, the call reception side mobile terminal can directly call audio signal into sound playback. 这样本实用新型中的移动通话终端就实现了语言障碍人士使用手语与对端使用者进行同步通话交流的目的,另一方面,本实用新型提供的移动通话终端在对手语影像进行转换的过程中,不仅利用了预存的规范手语对应的模糊文本信息,还利用预存的语法及组词参数对模糊文本信息进行纠错,从而使通信双方通话交流的内容更加准确。 Thus the present invention is to achieve a mobile terminal talking speech impaired and the object of the sign language of the end user performs call communication synchronization, on the other hand, the present invention provides a mobile terminal call sign language image conversion process not only the use of fuzzy text sign language specification information stored in the corresponding group of words and grammar also make use of stored parameters of fuzzy text information for error correction, so that both parties call communication exchange is much more accurate.

附图说明 BRIEF DESCRIPTION

[0024] 图1为本实用新型中适用于语言障碍人士的移动通话终端的结构示意图。 [0024] FIG. 1 is suitable for the present invention in a schematic structural diagram of a mobile terminal call the speech impaired. 具体实施方式 Detailed ways

[0025] 为使本实用新型的目的和优点更加清楚,下面结合附图和实施例对本实用新型作进一步的详细说明。 [0025] To make the objects and advantages of the invention more apparent, the accompanying drawings and the following detailed description of embodiments of the present invention further embodiment.

[0026] 图1为本实用新型中适用于语言障碍人士的移动通话终端的结构示意图,该移动通话终端包括:存储器11、摄像头12、手语解析芯片组13、文本信息纠错芯片组14、中央处理器15和显示屏16。 Diagram of a mobile call terminal structure of those obstacles novel in [0026] FIG 1 is practical with language, the mobile calling terminal comprises: a memory 11, the camera 12, the sign language analysis chip set 13, the text information correction chip set 14, a central The processor 15 and the display screen 16.

[0027] 其中,存储器11中存储有语法及组词参数和规范手语的图形轨迹及其对应的模糊文本信息。 [0027] wherein, the memory 11 stores the graphic trace and the parameters set of words and grammar specification and Fuzzy sign language corresponding to the text information. 上述规范手语的图形轨迹包括手型轮廓和运动趋势,每一个常用词(例如"我们"等)存储为一幅图片,当然这仅为图形轨迹及其存储方式的一种具体举例,并不排除在其他应用场景下采用其他方式。 The above specification, including sign language hand-shaped profile graph track trends and movements, each common words (such as "we", etc.) is stored as a picture, of course, this is only a graphical way of storing and track specific example, does not preclude other ways in other scenarios. 上述语法及组词参数,包括语法及组词的常用规则,例如: 陈述句的组词顺序为主语、谓语和宾语;常用词等。 This syntax and parameters set of words, including the commonly used group of words and grammar rules, for example: main groups declarative language word sequence, predicates and objects; common words and so on.

[0028] 摄像头12由中央处理器15触发,捕捉手语影像传输给手语影像解析芯片组13。 [0028] The camera 12 is triggered by the central processor 15, image capture sign language parsing is transmitted to the sign language video chipset 13. [0029] 手语解析芯片组13将摄像头12捕捉的手语影像解析为图形轨迹、并与存储器11 中存储的图形轨迹进行匹配,得出相匹配的图形轨迹所对应的模糊文本信息传输给文本信息纠错芯片组14。 [0029] The sign language parsing chipset 13 sign language image analyze the camera 12 to capture the graphic trace and match the graph tracks stored in the memory 11, blurred text messaging derived matched pattern trace corresponds to a text information correction wrong chipset 14. 该手语解析芯片组13包括:解析芯片131和匹配芯片132。 The sign language analysis chip set 13 comprising: parsing chip 131 and chip 132 match. [0030] 其中,解析芯片131将摄像头12捕捉的手语影像解析为多个包含手型轮廓和运动趋势的图形轨迹,由于使用者在输入手语时,每两个常用词之间会有一定的停顿,因此摄像头12捕捉并传输给解析芯片131的手语影像也在每两个常用词之间存在一定的停顿,这里的解析就可以依据这些停顿点进行。 [0030] wherein the analysis chip 131 parses the sign language imaging camera 12 to capture a plurality of tracks comprises a hand-shaped profile pattern and movement trend, since the user when the input sign language, there will be some common word pause between each two also exist between each two common words, so the camera captures 12 and transferred to the analysis chip 131 sign language video must pause here to resolve these pauses can be carried out in accordance with points. 解析芯片131将分解后的图形轨迹按顺序传输给匹配芯片132。 The track pattern analysis chip 131 is transmitted to the decomposed matching chip 132 sequentially.

[0031] 匹配芯片132在存储器11存储的图形轨迹中分别查找到与解析芯片131解析出的多个图形轨迹相匹配的图形轨迹,再从存储器11中读出相匹配的图形轨迹所对应的模糊文本信息传输给文本信息纠错芯片组14。 [0031] chip 132, respectively, to find matches in the graphics memory 11 stores the trajectory in the parsed analytical chip 131 and a plurality of graphical trace trace pattern matches, then read out from the memory 11 matches the pattern corresponding to the blur trajectory text information to the text information correction chipset 14. 匹配芯片132在确定存储器11中与解析芯片131分解出的图形轨迹匹配的图形轨迹时,首先依照完全匹配的原则,如果没有查找到,则其次依据最接近匹配原则,将最接近的那一幅已存储的图形轨迹作为匹配的图形轨迹。 Chip 132 match when determining the memory 11 and analysis chip 131 decomposed graphics pattern matching locus trajectory firstly accordance with the principle of exact match, if not found, then followed by the closest match according to the principle, that one closest to graph tracks stored as a pattern matched trajectory. [0032] 文本信息纠错芯片组14,利用存储器11中存储的语法及组词参数对模糊文本信息进行纠错,得到准确文本信息传输给中央处理器15。 [0032] Text information error correction chipset 14, and parameter syntax words stored in the memory 11 using a set of fuzzy text information correction, accurate text information to the central processor 15. 文本信息纠错芯片组14包括:参数读取芯片141和纠错执行芯片142。 Text information correction chipset 14 includes: a parameter reading chip 141 and the chip 142 performs the error correction.

[0033] 其中,参数读取芯片141从存储器11中读取语法及组词参数。 [0033] wherein the read parameter read chip 141 and the group of words grammatical parameters from the memory 11. [0034] 纠错执行芯片142利用参数读取芯片141读取的语法及组词参数,对模糊文本信息进行纠错,得到准确文本信息后传输给中央处理器15。 After [0034] chip 142 performs the error correction using the set of words and grammar specifications of chip 141 reads the read error correction fuzzy text information, the text to give accurate information to the central processor 15. 例如:假设使用者手语输入时将宾语和谓语的位置颠倒,则纠错执行芯片142可以依据语法规则将这两个词的位置颠倒过来,再假设使用者手语输入某个词时并不规范,使得手语解析芯片组13解析成了另外一个使整句话完全不通顺的词,则纠错执行芯片142可以使用词语联想从常用词的词库中挑选出正确的词,这样就及时纠正了模糊文本信息中的错误。 For example: Suppose the user inputs the position of sign language predicate and object reversed, the error correction chip 142 can perform these two words reversed position in accordance with the rules of grammar, and then assume that the user is not standardized sign language input a word, 13 sign language makes analytical analysis chip set became another so that the entire sentence is completely fluent word, then perform error correction chip 142 may use word association to pick out the correct word from a dictionary of common words, so that you promptly correct the fuzzy the error message text.

[0035] 中央处理器15,在通话连接建立后触发摄像头12 ;将准确文本信息转换为音频信号向对端移动通话终端发送;接收对端移动通话终端发送的音频信号,转换为文本信息后输出到显示屏16进行显示。 [0035] The central processor 15, the trigger after a call connection is established the camera 12; the exact text information is converted to the audio signal transmitting peer mobile call terminal to; after receiving the audio signal from the mobile calling terminal transmitting end, is converted into text information output the display screen 16 for display.

[0036] 为了使本实用新型中的移动通话终端既适用于语言障碍人士、也能具备正常情况下的通话功能,该移动通话终端还包括通常移动通话终端都会具备的麦克风17、音频处理器18和扬声器19,其中麦克风17收集外部声音信号传输给音频处理器18 ;音频处理器18 将麦克风收集的声音信号进行模数转换后传输给对端移动通话终端,将对端移动通话终端发送的音频信号进行数模转换后传输给扬声器19进行播放。 [0036] In order that the present invention is applicable to both a mobile terminal talking speech impaired, but also includes a call function under normal circumstances, the mobile terminal further comprises a calling terminal calls the mobile will typically have a microphone 17, an audio processor 18 and a speaker 19, a microphone 17 which collect external sound signals to the audio processor 18; an audio processor 18 the sound signal collected by the microphone is transmitted to the analog-digital conversion after the end of the call the mobile terminal, the mobile audio call will be sent by the terminal end after transmitting the signal digital-analog conversion to the speaker 19 for playback. 在这种情况下,移动通话终端将具有两种工作模式,即正常工作模式和语言障碍工作模式,显示屏16上可以提供给使用者模式选择应用界面,中央处理器15在接收使用者输入的模式选择信号后,如果该模式选择信号指示当前为语言障碍工作模式,则继续执行触发摄像头12的操作,如果该模式选择信号指示当前为正常工作模式,则触发麦克风17和音频处理器18执行操作。 In this case, the calling mobile terminal will have two modes, namely a normal operating mode and language barriers operating mode, the display 16 may be provided to the user application interface selection mode, the CPU 15 receives a user input after the mode selection signal, if the mode selection signal indicating that the current language barriers mode, proceed to operation triggers the camera 12, if the mode selection signal indicative of the current to the normal operating mode, the microphone 17 and the audio processor 18 performs operations triggering . [0037] 本实用新型中的手语解析芯片组13和文本信息纠错芯片组14,都可以采用通常的如集成芯片的实体部件实现,将预编制的算法程序烧录在这些实体部件中后,即可以按照本实用新型中介绍的方式进行工作。 [0037] The present invention in sign language analysis chip set 13 and text information correction chipset 14, as generally all the physical components of the integrated chip implemented, then the algorithm preprogrammed components burn these entities may be used, That work can be carried out in the manner described in the present invention. 与通常的移动通话终端不同,本实用新型中的移动通话终端的主板在设计时为手语解析芯片组13和文本信息纠错芯片组14预留了贴片位置,并通过布线设计完成它们与其他部件的连接。 Normal mobile call terminal different from the motherboard in the present invention, a mobile terminal in a call sign language designed to parse text information error correction and chipset chipset 1314 reserved patch position and complete them with other through a wiring design connection member.

[0038] 下面介绍本实用新型提供的移动通话终端的工作过程,以通信双方均为语言障碍人士为例,此时通话双方均将移动通话终端的工作模式选择为语言障碍工作模式,以下描述中为了简洁,将通话发起一方称为发起方终端,将通话接收一方称为接收方终端,该工作过程包括以下步骤: [0038] The following describes the working process of the call the mobile terminal of the present invention provides for the communication parties are speech impaired, for example, each time the two sides talking mobile call terminal operating mode is selected as the language barrier mode of operation, the following description for brevity, the party initiating the call originating party called terminal, the called party receives the call receiving terminal, the working process comprising the steps of:

[0039] 1)发起方终端的摄像头采集语言障碍人士的手语影像; [0039] 1) the originator terminal of sign language video camera capture impaired language;

[0040] 2)发起方终端的手语解析芯片组将手语影像解析为图形轨迹,并得到存储器中与该图形轨迹匹配的预存图形轨迹所对应的模糊文本信息; [0040] 2) the originator terminal chipset sign language parsing parsed into sign language image trace pattern, and with a text memory in the fuzzy match with the track pattern prestored graphics information corresponding to the track;

[0041] 3)发起方终端的文本信息纠错芯片组对上述模糊文本信息执行纠错后得到准确文本信息; [0041] 3) the originator terminal text information correction chipset accurate text information to text information after performing the blurring correction;

[0042] 4)发起方终端的中央处理器将上述准确文本信息转换为音频信号后发送给接收方终端; [0042] 4) originator terminal central processing unit converts the text information is accurately transmitted to the receiver terminal after the audio signal;

[0043] 5)接收方终端的中央处理器将上述音频信号转换为文本信息,并在接收方终端的显示屏上显示该文本信息。 [0043] 5) the central processor of the receiving terminal for the audio signal into the text information, and displays the text message on the display recipient terminal.

[0044] 如果通信双方一方为语言障碍人士,另一方为正常人士,则正常人士一方将移动通话终端的工作模式选择为正常工作模式,语言障碍人士一方仍将移动通话终端的工作模式选择为语言障碍工作模式。 [0044] If one of the two communication parties who language barrier, and the other is a normal person, who is one of the normal call of the mobile terminal operating mode is selected as the normal operation mode, speech impaired one will call the mobile terminal operation mode is selected as the language barriers to operating mode.

[0045] 当发起方指语言障碍人士这一方时,上述工作过程中的其他步骤相同,仅第5)步中,接收方终端的中央处理器触发接收方终端中的音频处理器对接收到的音频信号进行数模转换,然后通过扬声器播放给正常人士。 [0045] When the initiator means speech impaired this side, the other steps are the same as the above-described working process, only the 5) step, the central processor of the receiving terminal triggers the receiver terminal in the received audio processor The audio signal digital-analog conversion, and then played through the speaker to normal people.

[0046] 当发起方指正常人士这一方时,工作过程包括以下步骤:[0047] 1)发起方终端的中央处理器触发麦克风正常人士通话内容的声音信号进行收 [0046] When the initiator refers to the normal source side, the working process comprising the steps of: a sound signal [0047] 1) the originator terminal CPU normal person conversation microphone trigger is received

[0048] 2)发起方终端的音频处理器将麦克风收集的声音信号进行模数转换,然后发送给接收方终端; [0048] 2) a sound signal originating party terminal microphone collected the audio processor analog to digital conversion, and then transmitted to the receiving terminal;

[0049] 3)接收方终端的中央处理器将发起方终端发送的音频信号转换为文本信息,并在显示屏上显示给语言障碍人士。 [0049] 3) a central processor receiving terminal will initiate audio signal into the transmission side terminal into text information and displayed to the person on the display language barriers.

[0050] 综上所述,以上仅为本实用新型的较佳实施例而已,并非用于限定本实用新型的保护范围。 [0050] In summary, the present invention above is only preferred embodiments and not intended to limit the scope of protection of the present invention. 凡在本实用新型的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本实用新型的保护范围之内。 Any modification within the spirit and principle of the invention, the, equivalent substitutions, improvements should be included within the scope of protection of the present invention.

Claims (4)

  1. 一种适用于语言障碍人士的移动通话终端,其特征在于,该移动通话终端包括:中央处理器、显示屏、存储器、摄像头、手语解析芯片组和文本信息纠错芯片组;所述摄像头与手语解析芯片组连接,将捕捉的手语影像传输给手语影像解析芯片组;所述手语解析芯片组与存储器和文本信息纠错芯片组分别连接,将摄像头捕捉的手语影像解析为图形轨迹、并与存储器中预先存储的规范手语的图形轨迹进行匹配,得出相匹配的图形轨迹所对应的模糊文本信息传输给文本信息纠错芯片组;所述文本信息纠错芯片组与存储器和中央处理器分别连接,利用存储器中预先存储的语法及组词参数对所述模糊文本信息进行纠错,得到准确文本信息传输给中央处理器;所述中央处理器与摄像头和显示屏分别连接,在通话连接建立后向摄像头发送触发信号;将所述准确文 Impaired language suitable for a mobile terminal call, wherein the calling mobile terminal comprising: a central processor, a display, a memory, camera, sign language parsing and text information correction chipset chipset; the camera and sign language analytical chipset connected, the captured sign language video transmission to a sign language image analytic chipset; the sign language parsing chipset and the memory and the text information correction chipset connected, sign language image analyze captured by the camera for the graph track, and the memory sign language specification track pattern prestored matching, text messaging blur pattern derived locus matches corresponding to the text information correction chipset; the text information and the error correction chipset and memory are connected to the central processor , using the syntax and parameters set of words stored in the memory in advance of the blur correction text information, the text to give accurate information to the central processor; the central processor with the camera and the display screen are connected, after the call connection is established sending a trigger signal to the camera; the exact text 信息转换为音频信号向对端移动通话终端发送;接收对端移动通话终端发送的音频信号,转换为文本信息后输出到显示屏进行显示。 Information is converted to the audio signal transmitting calls to the peer mobile terminal; the mobile terminal receiving an audio signal transmitted from the call terminal, and outputs the converted text information to be displayed to a display screen.
  2. 2. 如权利要求1所述的移动通话终端,其特征在于,所述手语解析芯片组包括:解析芯片和匹配芯片;所述解析芯片与摄像头和匹配芯片分别连接,将摄像头捕捉的手语影像解析为多个包含手型轮廓和运动趋势的图形轨迹,并按顺序传输给匹配芯片;所述匹配芯片与存储器和文本信息纠错芯片组分别连接,在存储器预先存储的图形轨迹中分别查找到与解析芯片解析出的多个图形轨迹相匹配的图形轨迹,再从存储器中读出相匹配的图形轨迹所对应模糊文本信息传输给文本信息纠错芯片组。 Sign language image analyze the camera captured the analysis-chip with the camera and matching chip connections, respectively; analysis chip and a matching chip: as claimed in mobile calling terminal according to claim 1, wherein said sign language parsing chipset comprising pattern comprising a plurality of tracks shaped profile and hand movement trend, according to a transmission order matching chip; matching the text information and the memory chip and the chip set are connected to error correction, to find the trace pattern previously stored in the memory, respectively, to a analysis chip parsed plurality of graphical pattern matches the tracks trajectory, then the read match pattern corresponding to the blur trajectory of the text information to the text information from the memory in the error correction chip.
  3. 3. 如权利要求l所述的移动通话终端,其特征在于,所述文本信息纠错芯片组包括:参数读取芯片和纠错执行芯片;所述参数读取芯片与存储器和纠错执行芯片分别连接,从所述存储器中读取预先存储的语法及组词参数传输给纠错执行芯片;所述纠错执行芯片与手语解析芯片组和中央处理器分别连接,利用所述语法及组词参数,对所述模糊文本信息进行纠错,得到准确文本信息传输给中央处理器。 The parameter read chip memory chips and error correction performed; correction parameter read and execute chip chips: The mobile terminal talking to claim l, wherein said text information comprises error correction chipset They are connected to read word parameter set syntax, and transmitted from the pre-stored in the memory chip to the error correction performed; chip and performs the error correcting sign language parsing chipset and a central processor connected by a group of words and grammar parameter of the blur correction text information, the text to give accurate information to the central processor.
  4. 4. 如权利要求1所述的移动通话终端,其特征在于,该移动通话终端中还包括:麦克风、音频处理器和扬声器;所述中央处理器进一步与麦克风和音频处理器分别连接,接收从所述显示屏输入的模式选择信号,当所述模式选择信号指示当前为语言障碍工作模式通话时,继续执行所述向摄像头发送触发信号的操作;当所述模式选择信号指示当前为正常工作模式通话时,向麦克风和音频处理器发送触发信号;所述麦克风与音频处理器连接,收集外部声音信号传输给音频处理器; 所述音频处理器与扬声器连接,将所述麦克风收集的声音信号进行模数转换后传输给对端移动通话终端;接收对端移动通话终端发送的音频信号,在进行数模转换后传输到扬声器播放。 The mobile terminal calls according to claim 1, wherein the mobile call terminal further comprises: a microphone, an audio processor and a speaker; said central processor is further connected to the microphone and audio processor, respectively, received from the the display mode selection input signal, when the mode selection signal indicates the current operation mode call to the language barrier, continuing to execute the transmission operation to the camera trigger signal; if the current mode is the normal operation mode selection signal indicates the during a call, sending a trigger signal to the microphone and audio processor; connecting the microphone and audio processor for collecting external sound signals to the audio processor; the audio processor is connected with the speaker, the sound signal collected by the microphone after analog to digital conversion transmit to the peer mobile terminal call; audio signal receiving peer mobile terminal call transmission, after performing digital-analog conversion to the transmission speaker.
CN2009201663228U 2009-07-31 2009-07-31 Mobile speech communication terminal suitable for person with language barrier CN201440733U (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2009201663228U CN201440733U (en) 2009-07-31 2009-07-31 Mobile speech communication terminal suitable for person with language barrier

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2009201663228U CN201440733U (en) 2009-07-31 2009-07-31 Mobile speech communication terminal suitable for person with language barrier

Publications (1)

Publication Number Publication Date
CN201440733U true CN201440733U (en) 2010-04-21

Family

ID=42545388

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2009201663228U CN201440733U (en) 2009-07-31 2009-07-31 Mobile speech communication terminal suitable for person with language barrier

Country Status (1)

Country Link
CN (1) CN201440733U (en)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9712666B2 (en) 2013-08-29 2017-07-18 Unify Gmbh & Co. Kg Maintaining audio communication in a congested communication channel
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
US9915545B2 (en) 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
US10024678B2 (en) 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US10024679B2 (en) 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US10069965B2 (en) 2013-08-29 2018-09-04 Unify Gmbh & Co. Kg Maintaining audio communication in a congested communication channel
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10395555B2 (en) * 2015-03-30 2019-08-27 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing optimal braille output based on spoken and sign language
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9712666B2 (en) 2013-08-29 2017-07-18 Unify Gmbh & Co. Kg Maintaining audio communication in a congested communication channel
US10069965B2 (en) 2013-08-29 2018-09-04 Unify Gmbh & Co. Kg Maintaining audio communication in a congested communication channel
US10024679B2 (en) 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9915545B2 (en) 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
US10024678B2 (en) 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US10391631B2 (en) 2015-02-27 2019-08-27 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US10395555B2 (en) * 2015-03-30 2019-08-27 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing optimal braille output based on spoken and sign language
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions

Similar Documents

Publication Publication Date Title
US6934366B2 (en) Relay for personal interpreter
CN102939791B (en) Having for hearing, speech and visual disturbances human handheld communication aid
US7433818B2 (en) Subscriber terminal for providing speech-text encoding and telephony service
TWI249729B (en) Voice browser dialog enabler for a communication system
KR100819928B1 (en) Apparatus for speech recognition of wireless terminal and method of thereof
US20080059200A1 (en) Multi-Lingual Telephonic Service
US20100223046A1 (en) Methods and systems for a sign language graphical interpreter
US20100185434A1 (en) Methods, devices, and computer program products for providing real-time language translation capabilities between communication terminals
US8351581B2 (en) Systems and methods for intelligent call transcription
US7277858B1 (en) Client/server rendering of network transcoded sign language content
US8325883B2 (en) Method and system for providing assisted communications
CN102017513B (en) Method for real time network communication as well as method and system for real time multi-lingual communication
CN101202994B (en) Method and device assistant to user for body-building
US20070112571A1 (en) Speech recognition at a mobile terminal
US20040267527A1 (en) Voice-to-text reduction for real time IM/chat/SMS
US7774194B2 (en) Method and apparatus for seamless transition of voice and/or text into sign language
JP2015516091A (en) Method and terminal for updating voiceprint feature model
US6748053B2 (en) Relay for personal interpreter
US20090326939A1 (en) System and method for transcribing and displaying speech during a telephone call
EP3410257A1 (en) Method for quickly starting application service, and terminal
US20040117191A1 (en) Correlating video images of lip movements with audio signals to improve speech recognition
CN102117614A (en) Personalized text-to-speech synthesis and personalized speech feature extraction
CN1764266A (en) Apparatus and method for automatically changing communication mode in mobile communication terminal
JP2003163745A (en) Telephone set, interactive responder, interactive responding terminal, and interactive response system
CN1639696A (en) System and method for concurrent multimodal communication session persistence

Legal Events

Date Code Title Description
C14 Grant of patent or utility model
EXPY Termination of patent right or utility model