WO2017161741A1 - Method and device for communicating information with deaf-mutes, smart terminal - Google Patents

Method and device for communicating information with deaf-mutes, smart terminal Download PDF

Info

Publication number
WO2017161741A1
WO2017161741A1 PCT/CN2016/088485 CN2016088485W WO2017161741A1 WO 2017161741 A1 WO2017161741 A1 WO 2017161741A1 CN 2016088485 W CN2016088485 W CN 2016088485W WO 2017161741 A1 WO2017161741 A1 WO 2017161741A1
Authority
WO
WIPO (PCT)
Prior art keywords
sign language
gesture
voice signal
library
matching
Prior art date
Application number
PCT/CN2016/088485
Other languages
French (fr)
Chinese (zh)
Inventor
赵永贵
Original Assignee
乐视控股（北京）有限公司
乐视致新电子科技（天津）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 乐视控股（北京）有限公司, 乐视致新电子科技（天津）有限公司 filed Critical 乐视控股（北京）有限公司
Publication of WO2017161741A1 publication Critical patent/WO2017161741A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures

Definitions

  • the embodiments of the present invention relate to the technical field of intelligent terminals, and in particular, to a method, device, and intelligent terminal for information exchange between deaf and dumb people.
  • Mobile phones as an important and popular communication tool in modern times, especially the arrival of mobile Internet technology, have made the traditional mobile phone function machine has been replaced by a smart machine with powerful data processing functions.
  • the intelligent machine can also rely on the mobile Internet for information acquisition and information exchange.
  • the smartphone For most users who have no language and hearing impairments, the smartphone is very convenient for information acquisition and communication, and can be used without barriers.
  • the inventors have found that at least the following problems exist in the prior art: for a deaf-mute person with language and hearing impairment, when the information is exchanged, the standard language cannot be listened to because of the inability to pronounce normally. It can be carried out by sending and receiving text messages. Therefore, the performance of the mobile phone is not fully utilized, and the convenience provided by the deaf-mute people in information exchange is poor.
  • the purpose of the embodiments of the present application is to provide a method, a device, and an intelligent terminal for communicating information with a deaf person, which are used to solve the defect that the performance of the smart terminal is not fully exerted when the deaf-mute person performs information exchange in the prior art, and is enhanced.
  • Deaf-mute people use smart terminals for information exchange convenience.
  • the embodiment of the present application provides a method for information exchange with a deaf person, which includes:
  • the presentation selects standard sign language information corresponding to the voice signal from the sign language library.
  • the receiving a voice signal, and matching the voice signal in a pre-established sign language library includes:
  • selecting, according to a matching result of the voice signal in a pre-established sign language library, selecting standard sign language information corresponding to the voice signal from the sign language library includes:
  • the presenting that selecting the standard sign language information corresponding to the voice signal from the sign language library includes:
  • the displaying, by using the sign language library, the standard sign language information that matches the digital signal corresponding to the voice signal comprises: graphically displaying the selection from the sign language library The standard sign language information matched by the digital signal corresponding to the voice signal.
  • the method further includes: determining whether to start the hoarse interaction, and if yes, performing the receiving of the standard voice signal; otherwise, ending.
  • the embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
  • the embodiment of the present application provides a method for information exchange with a deaf person, which includes:
  • Each of the sign language gesture images in the opponent gesture image sequence performs gesture feature recognition and positioning
  • the capturing a sign language gesture and collecting the captured sign language gesture to form a sign language gesture image sequence includes:
  • the sign language gesture is captured and the captured sign language gesture is captured, a sequence of sign language gesture images is formed, and the sequence of sign language gesture images is converted into a digital signal.
  • the matching and the located gesture feature are matched in the sign language library to generate a digital signal:
  • a matching and generating a speech signal is performed in the sign language library according to the recognized and positioned gesture feature on the digital signal corresponding to the sign language gesture image.
  • performing gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images includes:
  • Each sign language gesture image gesture decomposition in the opponent gesture image sequence is performed, and gesture feature recognition and positioning are performed on the gesture-decomposed sign language gesture image.
  • the method further includes: determining whether to start the hoarse interaction, and if yes, performing the capturing of the start sign language gesture; otherwise, ending.
  • the embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
  • An embodiment of the present application provides an apparatus for performing information exchange with a deaf person, including:
  • a receiving module configured to receive a voice signal, and match the voice signal in a pre-established sign language library
  • a matching module configured to select a standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
  • a presentation module configured to present, from the sign language library, select standard sign language information corresponding to the voice signal.
  • the embodiment of the present application provides a method for information exchange with a deaf person, which includes:
  • An acquisition module configured to capture a sign language gesture and collect the captured sign language gesture to form a sequence of sign language gesture images
  • a positioning module configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images
  • a matching module configured to perform in the sign language library based on the recognized and positioned gesture features Equipped with a digital signal
  • a sending module configured to send a digital signal generated based on a matching result in the sign language library.
  • An embodiment of the present application provides an intelligent terminal, including a presentation module and a hardware processor, where the hardware processor is configured to receive a voice signal, and match the voice signal in a pre-established sign language library; a matching result of the speech signal in the sign language library, and selecting standard sign language information corresponding to the speech signal from the sign language library; the rendering module is configured to present a selection from the sign language library to correspond to the speech signal Standard sign language information.
  • the embodiment of the present application provides a smart terminal, including a camera, a hardware processor, and a radio frequency module, where the camera captures a sign language gesture and collects the captured sign language gesture to form a sign language gesture image sequence; the hardware processor is configured as a hand gesture Each of the sign language gesture images in the image sequence performs gesture feature recognition and positioning; and generates a digital signal according to the identified and positioned gesture features in the sign language library; the radio frequency module is configured to send the generated according to the matching result in the sign language library. Digital signal.
  • FIG. 1 is a schematic flow chart of a method for information exchange with a deaf person according to Embodiment 1 of the present application;
  • FIG. 2 is a schematic flowchart of a method for performing information exchange with a deaf person according to Embodiment 2 of the present application;
  • FIG. 3 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 3 of the present application;
  • FIG. 4 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 4 of the present application;
  • FIG. 5 is a schematic structural diagram of a fifth intelligent terminal according to Embodiment 5 of the present application.
  • FIG. 6 is a schematic structural diagram of a smart terminal according to Embodiment 6 of the present application.
  • the voice signal is matched in a pre-established sign language library, and standard sign language information corresponding to the voice signal is selected from the sign language library, so that the voice signal can be corresponding to the voice signal.
  • the standard sign language information is presented to the deaf person as the receiver; or, the sign language gesture is captured and the captured sign language gesture is captured to form a sequence of sign language gesture images, and the gesture feature recognition and positioning is performed for each sign language gesture image and matched in the sign language library.
  • the voice signal is generated to generate a voice signal, and the sign language of the deaf person is presented to the receiver's non-listening and language-disabled users, thereby realizing the information interaction between the two parties, and overcoming the performance of the smart terminal when the deaf-mute person performs information exchange.
  • the defects have enhanced the convenience of deaf-mute people using intelligent terminals for information exchange.
  • the smart terminal may include a smart phone, and may also include a smart tablet, a smart TV, a smart bracelet, and the like.
  • FIG. 1 is a schematic flow chart of a method for information exchange with a deaf person according to Embodiment 1 of the present application; in this embodiment, a user without language and hearing impairment calls a deaf-mute person with language and hearing impairment as an example, and the calling party is For users without language and hearing impairment, the listener is a deaf person with language and hearing impairment.
  • This embodiment describes the technical solution from the listener side. As shown in Figure 1, it includes:
  • S101 Receive a voice signal, and match the voice signal in a pre-established sign language library.
  • the speech signal is from a user without language and hearing impairment
  • the sign language library includes standard sign language information.
  • the sign language library is established by standardizing all the sign language used by the deaf person in daily use. Specifically, it is possible to collect sign language systems used by different countries or regions, such as the "vocabulary sign language” in the United States and the "scrabble sign language” in China, and an independent sign language library can be established for different sign language systems.
  • the sign language library is built specifically, the sign language model of different sign language in different sign language systems can be determined first, based on the comparison of the speech signal and the different sign language models, so as to achieve the matching process.
  • the receiving the voice signal in step S101, and matching the voice signal in the pre-established sign language library may specifically include: receiving the voice signal, converting the voice signal into The digital signal matches the digital signal corresponding to the voice signal in a pre-established sign language library.
  • the matching process implements a process of determining the correspondence between the speech signal and the standardized sign language in the sign language library.
  • the sign language system and the sign language system may be first determined.
  • the corresponding sign language library searches for a standardized sign language corresponding to the voice signal in the determined sign language library. Specifically, since in the sign language library, each sign language pattern can be matched to a corresponding binary code, a corresponding speech signal is generated according to the binary code.
  • the selecting the standard sign language information corresponding to the voice signal from the sign language library according to the matching result of the voice signal in the pre-established sign language library comprises: And matching the digital signal corresponding to the voice signal, and selecting standard sign language information matching the digital signal corresponding to the voice signal from the sign language library.
  • the presentation selects standard sign language information corresponding to the voice signal from the sign language library.
  • the presenting that selecting the standard sign language information corresponding to the voice signal from the sign language library comprises: displaying standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library.
  • the displaying, by using the sign language library, the standard sign language information that matches the digital signal corresponding to the voice signal comprises: graphically displaying, selecting, from the sign language library, a digital signal matching corresponding to the voice signal Standard sign language information.
  • sign language can be graphically displayed on the display of the smart terminal.
  • the method further includes: determining whether to start the hoarse interaction, and if yes, performing the receiving of the standard voice signal; otherwise, ending.
  • a quick start button can be set on the smart terminal, and the trigger of the quick start button is used to determine whether the deaf mode is activated.
  • an APP may be created according to the method described in FIG. 1 above, and the APP is installed and started on the smart terminal.
  • the above voice signals may include signals transmitted through a carrier network such as GSM/CDMA, and may also include signals transmitted through the mobile Internet, such as by WeChat.
  • the embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
  • the computer readable recording medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer).
  • a machine-readable medium includes read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (eg, carrier waves) , infrared signals, digital signals, etc.).
  • FIG. 2 is a schematic flowchart of a method for exchanging information with a deaf person according to a second embodiment of the present application; in this embodiment, a user with language and hearing impairment dials a telephone to a deaf-mute person without language and hearing impairment as an example, and the calling party is For users with language and hearing impairments, the listener is a deaf person with no language and hearing impairment.
  • This embodiment describes the technical solution from the listener side. As shown in Figure 2, it includes:
  • the smart terminal is configured with a front or rear camera.
  • the camera captures and collects the sign language gesture to form the gesture image. Sequence, including gesture images at successive points in time.
  • the capturing the sign language gesture and collecting the captured sign language gesture to form the sign language gesture image sequence in step S201 includes:
  • the sign language gesture is captured and the captured sign language gesture is captured, a sequence of sign language gesture images is formed, and the sequence of sign language gesture images is converted into a digital signal.
  • performing gesture feature recognition and positioning on each of the sign language gesture images in the sequence of the opponent gesture image in step S202 includes:
  • Each sign language gesture image gesture decomposition in the opponent gesture image sequence is performed, and gesture feature recognition and positioning are performed on the gesture-decomposed sign language gesture image.
  • the invariant features on the finger include at least: a shape , skin color, texture, edge information.
  • an annotated speech signal is established in the gesture library.
  • the sign language system and the sign language library corresponding to the sign language system can be determined first. , find in the identified sign language library A standard speech signal corresponding to a sign language signal.
  • the matching and the located gesture feature are matched in the sign language library to generate a digital signal:
  • a matching and generating a speech signal is performed in the sign language library according to the recognized and positioned gesture feature on the digital signal corresponding to the sign language gesture image.
  • the matched voice signal is sent to the receiver, and played on the receiving terminal.
  • the method may further include: determining whether to start the hoarse interaction, and if yes, performing the capturing of the startup sign language gesture; otherwise, ending.
  • a quick start button can be set on the smart terminal, and the trigger of the quick start button is used to determine whether the deaf mode is activated.
  • an APP may be created according to the method described in FIG. 1 above, and the APP is installed and started on the smart terminal.
  • the above voice signals may include signals transmitted through a carrier network such as GSM/CDMA, and may also include signals transmitted through the mobile Internet, such as by WeChat.
  • step S201 may be performed on the sender side, and steps S202-204 may be performed on the receiver side.
  • the embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
  • the computer readable recording medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer).
  • a machine-readable medium includes read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (eg, carrier waves) , infrared signals, digital signals, etc.).
  • FIG. 3 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 3 of the present application; as shown in FIG. 3, the method includes:
  • the receiving module 301 is configured to receive a voice signal, and match the voice signal in a pre-established sign language library;
  • the matching module 302 is configured to select a standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
  • the presentation module 303 is configured to present, from the sign language library, select standard sign language information corresponding to the voice signal.
  • the receiving module is further configured to receive a voice signal, convert the voice signal into a digital signal, and match the digital signal corresponding to the voice signal in a pre-established sign language library.
  • the matching module is further configured to select, according to a matching result of the digital signal corresponding to the voice signal, a digital signal corresponding to the voice signal from the sign language library. Standard sign language information.
  • the presentation module is further configured to display standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library.
  • the presentation module is further configured to graphically display, from the sign language library, standard sign language information that matches a digital signal corresponding to the voice signal.
  • the method further includes: a determining module configured to determine whether to initiate the deaf-mute interaction, and if yes, executing the initiating receiving module to receive the standard voice signal; otherwise, ending.
  • FIG. 4 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 4 of the present application; as shown in FIG. 4, the method includes:
  • the acquiring module 401 is configured to capture a sign language gesture and collect the captured sign language gesture to form a sequence of sign language gesture images;
  • the positioning module 402 is configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images;
  • the matching module 403 is configured to generate a digital signal by matching in the sign language library according to the recognized and located gesture features;
  • Sending module 404 configured to send a digital letter generated according to a matching result in the sign language library number.
  • the collection module 401 is further configured to capture a sign language gesture and collect the captured sign language gesture to form a sign language gesture image sequence, and convert the sign language gesture image sequence into a digital signal.
  • the matching module 403 is further configured to perform matching to generate a voice signal in the sign language library according to the digital signal on the corresponding sign language gesture image according to the recognized and located gesture feature.
  • the positioning module 402 is further configured to decompose each of the sign language gesture image gestures in the sequence of hand gesture gesture images, and perform gesture feature recognition and positioning on the gesture-decomposed sign language gesture image.
  • the method further includes a judging module (not shown) configured to determine whether to start the deaf-mute interaction, and if so, initiating the acquisition module to capture the sign language gesture; otherwise, ending.
  • a judging module (not shown) configured to determine whether to start the deaf-mute interaction, and if so, initiating the acquisition module to capture the sign language gesture; otherwise, ending.
  • the smart terminal 500 includes a presentation module 501 and a hardware processor 502, and the hardware processor 502 is configured to receive a voice signal in a pre-established sign language library. Matching the voice signal; and selecting standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library; the presentation module 501 is configured The standard sign language information corresponding to the voice signal is selected from the sign language library for presentation.
  • the camera 601 may include a camera 601, a hardware processor 602, and a radio frequency module 603.
  • the camera 601 captures a sign language gesture and collects the captured sign language gesture to form a sign language. a sequence of gesture images;
  • the hardware processor 602 is configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images, and generate a digital signal by matching in the sign language library according to the recognized and positioned gesture features.
  • the gesture library includes a standard voice signal;
  • the radio frequency module 603 is configured to transmit a digital signal generated according to a matching result in the sign language library.
  • the camera can be replaced by a handwriting screen of the smart terminal for capturing a sign language gesture and collecting the captured sign language gesture to form a sequence of sign language gesture images, which are not illustrated in detail.
  • the device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, ie may be located A place, or it can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Library & Information Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Telephonic Communication Services (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A method and device for communicating information with deaf-mutes, and a smart terminal. The method comprises: receiving a voice signal, and matching the voice signal in a pre-established sign language library (S101); selecting standard sign language information corresponding to the voice signal from the sign language library according to the matching result of the voice signal in the pre-established sign language library (S102); and presenting the standard sign language information corresponding to the voice signal and selected from the sign language library (S103). By matching a voice signal in a pre-established sign language library and selecting standard sign language information corresponding to the voice signal from the sign language library, the standard sign language information corresponding to the voice signal can be presented to deaf-mutes who are receivers, thereby overcoming the defect that the performance of a smart terminal is not fully used during information communication with deaf-mutes and improving the convenience for the deaf-mutes to achieve information communication using the smart terminal.

Description

聋哑人进行信息交流的方法、装置及智能终端Method, device and intelligent terminal for information exchange between deaf and mute people
本申请要求于2016年3月23日提交中国专利局、申请号为201610168501.X，发明名称为“一种聋哑人进行信息交流的方法、装置及智能终端”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on March 23, 2016, the Chinese Patent Office, the application number is 201610168501.X, and the invention name is “a method, device and intelligent terminal for information exchange between deaf and dumb people”. The entire contents of this application are incorporated herein by reference.
技术领域Technical field
本申请实施例涉及智能终端技术领域，尤其涉及一种聋哑人进行信息交流的方法、装置及智能终端。The embodiments of the present invention relate to the technical field of intelligent terminals, and in particular, to a method, device, and intelligent terminal for information exchange between deaf and dumb people.
背景技术Background technique
手机作为现代重要而普及的交流工具，尤其是移动互联网技术的到来，使得传统意义上的手机即功能机已经被逐渐的具有强大数据处理功能的智能机代替。智能机除了可以实现传统手机的基本通信功能比如发短信、打电话等，还可以依托于移动互联网进行信息获取以及信息交流。Mobile phones, as an important and popular communication tool in modern times, especially the arrival of mobile Internet technology, have made the traditional mobile phone function machine has been replaced by a smart machine with powerful data processing functions. In addition to the basic communication functions of traditional mobile phones, such as texting, making phone calls, etc., the intelligent machine can also rely on the mobile Internet for information acquisition and information exchange.
对于大多数无语言和听觉障碍的用户来说，智能手机极大的方便了信息的获取和交流，可以无障碍的进行使用。在实现本申请过程中，发明人发现现有技术中至少存在如下问题：对于有语言和听觉障碍的聋哑人来说，在进行信息交流时，由于无法正常发音，无法收听标准的语言，只能通过收发短信来进行，因此，使得手机的性能没有充分的发挥，对聋哑人在信息交流时提供的便捷性较差。For most users who have no language and hearing impairments, the smartphone is very convenient for information acquisition and communication, and can be used without barriers. In the process of implementing the present application, the inventors have found that at least the following problems exist in the prior art: for a deaf-mute person with language and hearing impairment, when the information is exchanged, the standard language cannot be listened to because of the inability to pronounce normally. It can be carried out by sending and receiving text messages. Therefore, the performance of the mobile phone is not fully utilized, and the convenience provided by the deaf-mute people in information exchange is poor.
发明内容Summary of the invention
本申请实施例的目的在于提供一种与聋哑人进行信息交流的方法、装置及智能终端，用以解决现有技术中聋哑人在进行信息交流时智能终端性能没有充分发挥的缺陷，增强聋哑人利用智能终端进行信息交流的便捷性。The purpose of the embodiments of the present application is to provide a method, a device, and an intelligent terminal for communicating information with a deaf person, which are used to solve the defect that the performance of the smart terminal is not fully exerted when the deaf-mute person performs information exchange in the prior art, and is enhanced. Deaf-mute people use smart terminals for information exchange convenience.
本申请实施例采用的技术方案如下：The technical solutions adopted by the embodiments of the present application are as follows:
本申请实施例提供一种与聋哑人进行信息交流的方法，其包括：The embodiment of the present application provides a method for information exchange with a deaf person, which includes:
接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；Receiving a voice signal, and matching the voice signal in a pre-established sign language library;
根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手 语库中选择与所述语音信号对应标准手语信息；According to the matching result of the speech signal in a pre-established sign language library, from the hand Selecting standard sign language information corresponding to the voice signal in the language library;
呈现从所述手语库中选择与所述语音信号对应标准手语信息。The presentation selects standard sign language information corresponding to the voice signal from the sign language library.
具体地，在本申请的一实施例中，所述接收语音信号，在预先建立的手语库中对所述语音信号进行匹配包括：Specifically, in an embodiment of the present application, the receiving a voice signal, and matching the voice signal in a pre-established sign language library includes:
接收语音信号，将所述语音信号转换成数字信号，在预先建立的手语库中对所述语音信号对应的数字信号进行匹配。Receiving a voice signal, converting the voice signal into a digital signal, and matching the digital signal corresponding to the voice signal in a pre-established sign language library.
具体地，在本申请的一实施例中，所述根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息包括：Specifically, in an embodiment of the present application, selecting, according to a matching result of the voice signal in a pre-established sign language library, selecting standard sign language information corresponding to the voice signal from the sign language library includes:
根据对所述语音信号对应的数字信号的匹配结果，从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。Selecting standard sign language information matching the digital signal corresponding to the voice signal from the sign language library according to a matching result of the digital signal corresponding to the voice signal.
具体地，在本申请的一实施例中，所述呈现从所述手语库中选择与所述语音信号对应标准手语信息包括：Specifically, in an embodiment of the present application, the presenting that selecting the standard sign language information corresponding to the voice signal from the sign language library includes:
显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。Displaying standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library.
具体地，在本申请的一实施例中，所述显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息包括：以图形方式显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。Specifically, in an embodiment of the present application, the displaying, by using the sign language library, the standard sign language information that matches the digital signal corresponding to the voice signal comprises: graphically displaying the selection from the sign language library The standard sign language information matched by the digital signal corresponding to the voice signal.
具体地，在本申请的一实施例中，还包括：判断是否启动聋哑交互，如果是，则执行启动标准语音信号的接收；否则，结束。Specifically, in an embodiment of the present application, the method further includes: determining whether to start the hoarse interaction, and if yes, performing the receiving of the standard voice signal; otherwise, ending.
本申请实施例提供一种在其上记录有用于执行上述方法的程序的计算机可读记录介质。The embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
本申请实施例提供一种与聋哑人进行信息交流的方法，其包括：The embodiment of the present application provides a method for information exchange with a deaf person, which includes:
捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；Capturing a sign language gesture and collecting the captured sign language gesture to form a sequence of sign language gesture images;
对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；Each of the sign language gesture images in the opponent gesture image sequence performs gesture feature recognition and positioning;
根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；Generating a digital signal based on the identified and located gesture features in the sign language library;
发送根据在手语库中匹配结果生成的数字信号。 Sends a digital signal generated based on the matching result in the sign language library.
具体地，在本申请的一实施例中，所述捕获手语手势并采集捕获到的手语手势形成手语手势图像序列包括：Specifically, in an embodiment of the present application, the capturing a sign language gesture and collecting the captured sign language gesture to form a sign language gesture image sequence includes:
捕获手语手势并采集捕获到的手语手势，形成手语手势图像序列，将所述手语手势图像序列转换成数字信号。The sign language gesture is captured and the captured sign language gesture is captured, a sequence of sign language gesture images is formed, and the sequence of sign language gesture images is converted into a digital signal.
具体地，在本申请的一实施例中，所述根据识别和定位到的手势特征在手语库中进行匹配生成数字信号：Specifically, in an embodiment of the present application, the matching and the located gesture feature are matched in the sign language library to generate a digital signal:
根据识别和定位到的手势特征在对应手语手势图像上的数字信号，在手语库中进行匹配生成语音信号。A matching and generating a speech signal is performed in the sign language library according to the recognized and positioned gesture feature on the digital signal corresponding to the sign language gesture image.
具体地，在本申请的一实施例中，所述对手语手势图像序列中每个手语手势图像进行手势特征识别和定位包括：Specifically, in an embodiment of the present application, performing gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images includes:
对手语手势图像序列中每个手语手势图像手势分解，在手势分解了的手语手势图像上进行手势特征识别和定位。Each sign language gesture image gesture decomposition in the opponent gesture image sequence is performed, and gesture feature recognition and positioning are performed on the gesture-decomposed sign language gesture image.
具体地，在本申请的一实施例中，还包括：判断是否启动聋哑交互，如果是，则执行启动手语手势的捕获；否则，结束。Specifically, in an embodiment of the present application, the method further includes: determining whether to start the hoarse interaction, and if yes, performing the capturing of the start sign language gesture; otherwise, ending.
本申请实施例提供一种在其上记录有用于执行上述方法的程序的计算机可读记录介质。The embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
本申请实施例提供一种与聋哑人进行信息交流的装置，其包括：An embodiment of the present application provides an apparatus for performing information exchange with a deaf person, including:
接收模块，配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；a receiving module configured to receive a voice signal, and match the voice signal in a pre-established sign language library;
匹配模块，配置为根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；a matching module, configured to select a standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
呈现模块，配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。And a presentation module configured to present, from the sign language library, select standard sign language information corresponding to the voice signal.
本申请实施例提供一种与聋哑人进行信息交流的方法，其包括：The embodiment of the present application provides a method for information exchange with a deaf person, which includes:
采集模块，配置为捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；An acquisition module configured to capture a sign language gesture and collect the captured sign language gesture to form a sequence of sign language gesture images;
定位模块，配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；a positioning module configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images;
匹配模块，配置为根据识别和定位到的手势特征在手语库中进行匹 配生成数字信号；A matching module configured to perform in the sign language library based on the recognized and positioned gesture features Equipped with a digital signal;
发送模块，配置为发送根据在手语库中匹配结果生成的数字信号。A sending module configured to send a digital signal generated based on a matching result in the sign language library.
本申请实施例提供一种智能终端，其包括呈现模块以及硬件处理器，所述硬件处理器配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；以及根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；所述呈现模块配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。An embodiment of the present application provides an intelligent terminal, including a presentation module and a hardware processor, where the hardware processor is configured to receive a voice signal, and match the voice signal in a pre-established sign language library; a matching result of the speech signal in the sign language library, and selecting standard sign language information corresponding to the speech signal from the sign language library; the rendering module is configured to present a selection from the sign language library to correspond to the speech signal Standard sign language information.
本申请实施例提供一种智能终端，其包括摄像头、硬件处理器以及射频模块，所述摄像头捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；所述硬件处理器配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；以及根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；所述射频模块配置为发送根据在手语库中匹配结果生成的数字信号。The embodiment of the present application provides a smart terminal, including a camera, a hardware processor, and a radio frequency module, where the camera captures a sign language gesture and collects the captured sign language gesture to form a sign language gesture image sequence; the hardware processor is configured as a hand gesture Each of the sign language gesture images in the image sequence performs gesture feature recognition and positioning; and generates a digital signal according to the identified and positioned gesture features in the sign language library; the radio frequency module is configured to send the generated according to the matching result in the sign language library. Digital signal.
本申请实施例的技术方案具有以下优点：The technical solution of the embodiment of the present application has the following advantages:
通过在预先建立的手语库中对所述语音信号进行匹配，从所述手语库中选择与所述语音信号对应标准手语信息，从而可以将与所述语音信号对应标准手语信息展现给作为接收方的聋哑人；或者，捕获手语手势并采集捕获到的手语手势形成手语手势图像序列，对每个手语手势图像进行手势特征识别和定位并在手语库中进行匹配生成语音信号从而生成语音信号，将聋哑人的手语展现给接收方的无听力和语言障碍用户，从而实现了双方的信息交互，克服了聋哑人在进行信息交流时智能终端性能没有充分发挥的缺陷，增强了聋哑人利用智能终端进行信息交流的便捷性。Selecting the standard sign language information corresponding to the voice signal from the sign language library by matching the voice signal in a pre-established sign language library, so that the standard sign language information corresponding to the voice signal can be presented to the receiving party. Or deaf-mute; or, capture sign language gestures and capture the captured sign language gestures to form a sequence of sign language gesture images, perform gesture feature recognition and localization on each sign language gesture image, and perform matching in the sign language library to generate a speech signal to generate a speech signal, The deaf-mute sign language is presented to the recipient's hearing-free and speech-disabled users, thereby realizing the information interaction between the two parties, overcoming the defects that the deaf-mute people do not fully exert their performance when performing information exchange, and enhances the deaf-mute person. The convenience of information exchange using smart terminals.
附图说明DRAWINGS
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其 他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is some embodiments of the present application, and those skilled in the art can obtain the same according to these drawings without any creative work. His drawings.
图1为本申请实施例一与聋哑人进行信息交流的方法流程示意图；1 is a schematic flow chart of a method for information exchange with a deaf person according to Embodiment 1 of the present application;
图2为本申请实施例二与聋哑人进行信息交流的方法流程示意图；2 is a schematic flowchart of a method for performing information exchange with a deaf person according to Embodiment 2 of the present application;
图3为本申请实施例三与聋哑人进行信息交流的装置结构示意图；3 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 3 of the present application;
图4为本申请实施例四与聋哑人进行信息交流的装置结构示意图；4 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 4 of the present application;
图5为本申请实施例五智能终端结构示意图；5 is a schematic structural diagram of a fifth intelligent terminal according to Embodiment 5 of the present application;
图6为本申请实施例六智能终端结构示意图。FIG. 6 is a schematic structural diagram of a smart terminal according to Embodiment 6 of the present application.
具体实施方式detailed description
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
本申请下述实施例中，通过在预先建立的手语库中对所述语音信号进行匹配，从所述手语库中选择与所述语音信号对应标准手语信息，从而可以将与所述语音信号对应标准手语信息展现给作为接收方的聋哑人；或者，捕获手语手势并采集捕获到的手语手势形成手语手势图像序列，对每个手语手势图像进行手势特征识别和定位并在手语库中进行匹配生成语音信号从而生成语音信号，将聋哑人的手语展现给接收方的无听力和语言障碍用户，从而实现了双方的信息交互，克服了聋哑人在进行信息交流时智能终端性能没有充分发挥的缺陷，增强了聋哑人利用智能终端进行信息交流的便捷性。In the following embodiments, the voice signal is matched in a pre-established sign language library, and standard sign language information corresponding to the voice signal is selected from the sign language library, so that the voice signal can be corresponding to the voice signal. The standard sign language information is presented to the deaf person as the receiver; or, the sign language gesture is captured and the captured sign language gesture is captured to form a sequence of sign language gesture images, and the gesture feature recognition and positioning is performed for each sign language gesture image and matched in the sign language library. The voice signal is generated to generate a voice signal, and the sign language of the deaf person is presented to the receiver's non-listening and language-disabled users, thereby realizing the information interaction between the two parties, and overcoming the performance of the smart terminal when the deaf-mute person performs information exchange. The defects have enhanced the convenience of deaf-mute people using intelligent terminals for information exchange.
下述实施例中，智能终端除了可以包括智能手机，还可以包括智能平板、智能电视、智能手环等等智能硬件设备。In the following embodiments, the smart terminal may include a smart phone, and may also include a smart tablet, a smart TV, a smart bracelet, and the like.
图1为本申请实施例一与聋哑人进行信息交流的方法流程示意图；本实施例中，无语言和听觉障碍的用户向有语言和听觉障碍的聋哑人拨打电话为例，拨打方为无语言和听觉障碍的用户，收听方为有语言和听觉障碍的聋哑人，本实施例从收听方一侧进行技术方案的说明。如图1所示，其包括： 1 is a schematic flow chart of a method for information exchange with a deaf person according to Embodiment 1 of the present application; in this embodiment, a user without language and hearing impairment calls a deaf-mute person with language and hearing impairment as an example, and the calling party is For users without language and hearing impairment, the listener is a deaf person with language and hearing impairment. This embodiment describes the technical solution from the listener side. As shown in Figure 1, it includes:
S101、接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；S101. Receive a voice signal, and match the voice signal in a pre-established sign language library.
本实施例中，语音信号来自无语言和听觉障碍的用户，手语库中包括标准的手语信息，手语库的建立通过把聋哑人在日常使用过程中所有使用到的手语，并进行标准化得到。具体地，可以收集不同国家或地区的使用的手语体系，比如有美国的“词汇性手语”、中国的“拼字性手语”，可以为不同手语体系建立独立的手语库。在具体建立手语库时，可以首先确定一个不同手语体系不同手语的手语模型，基于语音信号与不同手语模型的比对，从而实现匹配的过程。In this embodiment, the speech signal is from a user without language and hearing impairment, and the sign language library includes standard sign language information. The sign language library is established by standardizing all the sign language used by the deaf person in daily use. Specifically, it is possible to collect sign language systems used by different countries or regions, such as the "vocabulary sign language" in the United States and the "scrabble sign language" in China, and an independent sign language library can be established for different sign language systems. When the sign language library is built specifically, the sign language model of different sign language in different sign language systems can be determined first, based on the comparison of the speech signal and the different sign language models, so as to achieve the matching process.
具体地，本实施例中或其他任意实施例中，步骤S101中接收语音信号，在预先建立的手语库中对所述语音信号进行匹配可以具体包括：接收语音信号，将所述语音信号转换成数字信号，在预先建立的手语库中对所述语音信号对应的数字信号进行匹配。Specifically, in this embodiment or any other embodiment, the receiving the voice signal in step S101, and matching the voice signal in the pre-established sign language library may specifically include: receiving the voice signal, converting the voice signal into The digital signal matches the digital signal corresponding to the voice signal in a pre-established sign language library.
具体地，接收的语音信号可以是射频信号，通过对该射频信号进行解调、信道分离、解密、信道解码、语音解码等方式得到数字信号。Specifically, the received voice signal may be a radio frequency signal, and the digital signal is obtained by performing demodulation, channel separation, decryption, channel decoding, voice decoding, and the like on the radio frequency signal.
S102、根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；S102. Select, according to a matching result of the voice signal in a pre-established sign language library, standard sign language information corresponding to the voice signal from the sign language library;
本实施例中或其他任意实施例中，匹配的过程实现了语音信号与手语库中标准化手语的对应关系确定过程，当进行语音信号与标准化手语的对应时，可以首先确定手语体系以及该手语体系对应的手语库，在确定的手语库中查找与语音信号对应的标准化手语。具体地中，由于在手语库中，每一个手语图形可以匹配到对应一个二进制码，根据该二进制码生成对应的语音信号。In this embodiment or any other embodiment, the matching process implements a process of determining the correspondence between the speech signal and the standardized sign language in the sign language library. When the correspondence between the speech signal and the standardized sign language is performed, the sign language system and the sign language system may be first determined. The corresponding sign language library searches for a standardized sign language corresponding to the voice signal in the determined sign language library. Specifically, since in the sign language library, each sign language pattern can be matched to a corresponding binary code, a corresponding speech signal is generated according to the binary code.
本实施例中或其他任意实施例中，所述根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息包括：根据对所述语音信号对应的数字信号的匹配结果，从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。In this embodiment or any other embodiment, the selecting the standard sign language information corresponding to the voice signal from the sign language library according to the matching result of the voice signal in the pre-established sign language library comprises: And matching the digital signal corresponding to the voice signal, and selecting standard sign language information matching the digital signal corresponding to the voice signal from the sign language library.
S103、呈现从所述手语库中选择与所述语音信号对应标准手语信息。 S103. The presentation selects standard sign language information corresponding to the voice signal from the sign language library.
本实施例中，所述呈现从所述手语库中选择与所述语音信号对应标准手语信息包括：显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。具体地，所述显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息包括：以图形方式显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。In this embodiment, the presenting that selecting the standard sign language information corresponding to the voice signal from the sign language library comprises: displaying standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library. Specifically, the displaying, by using the sign language library, the standard sign language information that matches the digital signal corresponding to the voice signal comprises: graphically displaying, selecting, from the sign language library, a digital signal matching corresponding to the voice signal Standard sign language information.
对于视力良好的聋哑人，可以将手语以图形方式显示在智能终端的显示屏上。For deaf-mute people with good vision, sign language can be graphically displayed on the display of the smart terminal.
具体地，在本申请的一实施例中，还包括：判断是否启动聋哑交互，如果是，则执行启动标准语音信号的接收；否则，结束。Specifically, in an embodiment of the present application, the method further includes: determining whether to start the hoarse interaction, and if yes, performing the receiving of the standard voice signal; otherwise, ending.
具体地，可以在智能终端上设定一快捷启动按键，通过该快捷启动按键的触发来判断是否启动了聋哑交互模式。Specifically, a quick start button can be set on the smart terminal, and the trigger of the quick start button is used to determine whether the deaf mode is activated.
但是，需要说明的是，也可以根据上述图1所述的方法制作一APP，在智能终端上安装并启动该APP。However, it should be noted that an APP may be created according to the method described in FIG. 1 above, and the APP is installed and started on the smart terminal.
上述语音信号可以包括通过运营商网络如GSM/CDMA进行发送的信号，也可以包括通过移动互联网进行传输的信号，比如通过微信等。The above voice signals may include signals transmitted through a carrier network such as GSM/CDMA, and may also include signals transmitted through the mobile Internet, such as by WeChat.
本申请实施例提供一种在其上记录有用于执行上述方法的程序的计算机可读记录介质。The embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
所述计算机可读记录介质包括用于以计算机(例如计算机)可读的形式存储或传送信息的任何机制。例如，机器可读介质包括只读存储器(ROM)、随机存取存储器(RAM)、磁盘存储介质、光存储介质、闪速存储介质、电、光、声或其他形式的传播信号(例如，载波、红外信号、数字信号等)等。The computer readable recording medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer). For example, a machine-readable medium includes read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (eg, carrier waves) , infrared signals, digital signals, etc.).
图2为本申请实施例二与聋哑人进行信息交流的方法流程示意图；本实施例中，有语言和听觉障碍的用户向无语言和听觉障碍的聋哑人拨打电话为例，拨打方为有语言和听觉障碍的用户，收听方为无语言和听觉障碍的聋哑人，本实施例从收听方一侧进行技术方案的说明。如图2所示，其包括：2 is a schematic flowchart of a method for exchanging information with a deaf person according to a second embodiment of the present application; in this embodiment, a user with language and hearing impairment dials a telephone to a deaf-mute person without language and hearing impairment as an example, and the calling party is For users with language and hearing impairments, the listener is a deaf person with no language and hearing impairment. This embodiment describes the technical solution from the listener side. As shown in Figure 2, it includes:
S201、捕获手语手势并采集捕获到的手语手势形成手语手势图像序 列；S201, capturing a sign language gesture and collecting the captured sign language gesture to form a sign language gesture image sequence Column
本实施例中或者其他任意实施例中，在智能终端上均配置有前置或者后置摄像头，当聋哑人在使用手语进行信息表达的过程中，通过摄像头捕获并采集手语手势从而形成手势图像序列，包括在连续时间点上的手势图像。In this embodiment or any other embodiment, the smart terminal is configured with a front or rear camera. When the deaf person uses the sign language to express the information, the camera captures and collects the sign language gesture to form the gesture image. Sequence, including gesture images at successive points in time.
本实施例中或者其他任意实施例中，步骤S201中所述捕获手语手势并采集捕获到的手语手势形成手语手势图像序列包括：In this embodiment or any other embodiment, the capturing the sign language gesture and collecting the captured sign language gesture to form the sign language gesture image sequence in step S201 includes:
捕获手语手势并采集捕获到的手语手势，形成手语手势图像序列，将所述手语手势图像序列转换成数字信号。The sign language gesture is captured and the captured sign language gesture is captured, a sequence of sign language gesture images is formed, and the sequence of sign language gesture images is converted into a digital signal.
另外，对于支持手写的智能手机来说，可以捕获聋哑人在手机触控屏幕上手写输入的手语手势，比如在微信上手写出手语。In addition, for smartphones that support handwriting, you can capture the sign language gestures handwritten by deaf people on the touch screen of the mobile phone, such as handwriting on WeChat.
S202、对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；S202: Performing gesture feature recognition and positioning for each sign language gesture image in the opponent speech image sequence;
本实施例中或者其他任意实施例中，步骤S202中所述对手语手势图像序列中每个手语手势图像进行手势特征识别和定位包括：In this embodiment, or any other embodiment, performing gesture feature recognition and positioning on each of the sign language gesture images in the sequence of the opponent gesture image in step S202 includes:
对手语手势图像序列中每个手语手势图像手势分解，在手势分解了的手语手势图像上进行手势特征识别和定位。Each sign language gesture image gesture decomposition in the opponent gesture image sequence is performed, and gesture feature recognition and positioning are performed on the gesture-decomposed sign language gesture image.
本实施例中，通过在一系列的图像上寻找不变特征，并利用这些特征来定位入脸，具体包括模板匹配法和基于外观形状的方法，所述手指上的不变特征至少包括：形状、肤色、纹理、边缘信息。In this embodiment, by searching for invariant features on a series of images and using the features to locate the face, specifically including a template matching method and a method based on the appearance shape, the invariant features on the finger include at least: a shape , skin color, texture, edge information.
或者，制作标准模板(固定模板)或将模板先行参数化(可变模板)，计算输入图像与模板之间的相关值，这个相关值通常都是独立计算手指的轮廓、指尖各自的匹配程度后得出的综合描述，最后再根据相关值和预先设定的阈值来确定图像中是否存在手指。Or, make a standard template (fixed template) or parameterize the template first (variable template), calculate the correlation value between the input image and the template, this correlation value is usually independent calculation of the contour of the finger, the degree of matching of the fingertips After the comprehensive description, finally determine whether there are fingers in the image according to the correlation value and the preset threshold.
S203、根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；S203. Perform matching in the sign language library according to the recognized and positioned gesture features to generate a digital signal.
与上述图1类似，根据不同手语手势与语音信号的对应关系，在手势库中建立了标注语音信号，当进行手语与语音信号的对应时，可以首先确定手语体系以及该手语体系对应的手语库，在确定的手语库中查找 与手语信号对应的标准语音信号。Similar to FIG. 1 above, according to the correspondence between different sign language gestures and speech signals, an annotated speech signal is established in the gesture library. When the correspondence between the sign language and the speech signal is performed, the sign language system and the sign language library corresponding to the sign language system can be determined first. , find in the identified sign language library A standard speech signal corresponding to a sign language signal.
具体地，在本申请的一实施例中，所述根据识别和定位到的手势特征在手语库中进行匹配生成数字信号：Specifically, in an embodiment of the present application, the matching and the located gesture feature are matched in the sign language library to generate a digital signal:
根据识别和定位到的手势特征在对应手语手势图像上的数字信号，在手语库中进行匹配生成语音信号。A matching and generating a speech signal is performed in the sign language library according to the recognized and positioned gesture feature on the digital signal corresponding to the sign language gesture image.
S204、发送根据在手语库中匹配结果生成的数字信号。S204. Send a digital signal generated according to a matching result in the sign language library.
本实施例或者其他任意实施例中，将匹配到的语音信号发送到接收方，并在接收方得智能终端上进行播放。In this embodiment or any other embodiment, the matched voice signal is sent to the receiver, and played on the receiving terminal.
本实施例或其他任意实施例中，还可以包括：判断是否启动聋哑交互，如果是，则执行启动手语手势的捕获；否则，结束。In this embodiment or any other embodiment, the method may further include: determining whether to start the hoarse interaction, and if yes, performing the capturing of the startup sign language gesture; otherwise, ending.
具体地，可以在智能终端上设定一快捷启动按键，通过该快捷启动按键的触发来判断是否启动了聋哑交互模式。Specifically, a quick start button can be set on the smart terminal, and the trigger of the quick start button is used to determine whether the deaf mode is activated.
但是，需要说明的是，也可以根据上述图1所述的方法制作一APP，在智能终端上安装并启动该APP。However, it should be noted that an APP may be created according to the method described in FIG. 1 above, and the APP is installed and started on the smart terminal.
上述语音信号可以包括通过运营商网络如GSM/CDMA进行发送的信号，也可以包括通过移动互联网进行传输的信号，比如通过微信等。The above voice signals may include signals transmitted through a carrier network such as GSM/CDMA, and may also include signals transmitted through the mobile Internet, such as by WeChat.
需要说明的是，在另外一实施例中，上述步骤S201可以在发送方侧执行，而步骤S202-204可以在接收方侧执行。It should be noted that, in another embodiment, the above step S201 may be performed on the sender side, and steps S202-204 may be performed on the receiver side.
本申请实施例提供一种在其上记录有用于执行上述方法的程序的计算机可读记录介质。The embodiment of the present application provides a computer readable recording medium on which a program for executing the above method is recorded.
所述计算机可读记录介质包括用于以计算机(例如计算机)可读的形式存储或传送信息的任何机制。例如，机器可读介质包括只读存储器(ROM)、随机存取存储器(RAM)、磁盘存储介质、光存储介质、闪速存储介质、电、光、声或其他形式的传播信号(例如，载波、红外信号、数字信号等)等。The computer readable recording medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer). For example, a machine-readable medium includes read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (eg, carrier waves) , infrared signals, digital signals, etc.).
图3为本申请实施例三与聋哑人进行信息交流的装置结构示意图；如图3所示，其包括：3 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 3 of the present application; as shown in FIG. 3, the method includes:
接收模块301，配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配； The receiving module 301 is configured to receive a voice signal, and match the voice signal in a pre-established sign language library;
匹配模块302，配置为根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；The matching module 302 is configured to select a standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
呈现模块303，配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。The presentation module 303 is configured to present, from the sign language library, select standard sign language information corresponding to the voice signal.
本实施例中或其他任意实施例中，所述接收模块进一步配置为接收语音信号，将所述语音信号转换成数字信号，在预先建立的手语库中对所述语音信号对应的数字信号进行匹配。In this embodiment or any other embodiment, the receiving module is further configured to receive a voice signal, convert the voice signal into a digital signal, and match the digital signal corresponding to the voice signal in a pre-established sign language library. .
本实施例中或其他任意实施例中，所述匹配模块进一步配置为根据对所述语音信号对应的数字信号的匹配结果，从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。In this embodiment or any other embodiment, the matching module is further configured to select, according to a matching result of the digital signal corresponding to the voice signal, a digital signal corresponding to the voice signal from the sign language library. Standard sign language information.
本实施例中或其他任意实施例中，所述呈现模块进一步配置为显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。In this embodiment or any other embodiment, the presentation module is further configured to display standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library.
本实施例中或其他任意实施例中，所述呈现模块进一步配置为以图形方式显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。In this embodiment or any other embodiment, the presentation module is further configured to graphically display, from the sign language library, standard sign language information that matches a digital signal corresponding to the voice signal.
本实施例中或其他任意实施例中，还包括判断模块，配置为判断是否启动聋哑交互，如果是，则执行启动接收模块，接收标准语音信号；否则，结束。In this embodiment or any other embodiment, the method further includes: a determining module configured to determine whether to initiate the deaf-mute interaction, and if yes, executing the initiating receiving module to receive the standard voice signal; otherwise, ending.
图4为本申请实施例四与聋哑人进行信息交流的装置结构示意图；如图4所示，其包括：4 is a schematic structural diagram of an apparatus for performing information exchange with a deaf person according to Embodiment 4 of the present application; as shown in FIG. 4, the method includes:
采集模块401：配置为捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；The acquiring module 401 is configured to capture a sign language gesture and collect the captured sign language gesture to form a sequence of sign language gesture images;
定位模块402：配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；The positioning module 402 is configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images;
匹配模块403：配置为根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；The matching module 403 is configured to generate a digital signal by matching in the sign language library according to the recognized and located gesture features;
发送模块404：配置为发送根据在手语库中匹配结果生成的数字信 号。Sending module 404: configured to send a digital letter generated according to a matching result in the sign language library number.
本实施例或其他任意实施例中，所述采集模块401进一步配置为捕获手语手势并采集捕获到的手语手势，形成手语手势图像序列，将所述手语手势图像序列转换成数字信号。In this embodiment or any other embodiment, the collection module 401 is further configured to capture a sign language gesture and collect the captured sign language gesture to form a sign language gesture image sequence, and convert the sign language gesture image sequence into a digital signal.
本实施例或其他任意实施例中，所述匹配模块403进一步配置为根据识别和定位到的手势特征在对应手语手势图像上的数字信号，在手语库中进行匹配生成语音信号。In this embodiment or any other embodiment, the matching module 403 is further configured to perform matching to generate a voice signal in the sign language library according to the digital signal on the corresponding sign language gesture image according to the recognized and located gesture feature.
本实施例或其他任意实施例中，所述定位模块402进一步配置为对手语手势图像序列中每个手语手势图像手势分解，在手势分解了的手语手势图像上进行手势特征识别和定位。In this embodiment or any other embodiment, the positioning module 402 is further configured to decompose each of the sign language gesture image gestures in the sequence of hand gesture gesture images, and perform gesture feature recognition and positioning on the gesture-decomposed sign language gesture image.
本实施例或其他任意实施例中，还包括判断模块(图中未示出)，配置为判断是否启动聋哑交互，如果是，则启动采集模块捕获手语手势；否则，结束。In this embodiment or any other embodiment, the method further includes a judging module (not shown) configured to determine whether to start the deaf-mute interaction, and if so, initiating the acquisition module to capture the sign language gesture; otherwise, ending.
图5为本申请实施例五智能终端结构示意图；如图5所示，智能终端500包括呈现模块501以及硬件处理器502，所述硬件处理器502配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；以及根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；所述呈现模块501配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。5 is a schematic structural diagram of a smart terminal according to Embodiment 5 of the present application; as shown in FIG. 5, the smart terminal 500 includes a presentation module 501 and a hardware processor 502, and the hardware processor 502 is configured to receive a voice signal in a pre-established sign language library. Matching the voice signal; and selecting standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library; the presentation module 501 is configured The standard sign language information corresponding to the voice signal is selected from the sign language library for presentation.
图6为本申请实施例六智能终端结构示意图；如图6所示，其可以包括摄像头601、硬件处理器602以及射频模块603，所述摄像头601捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；所述硬件处理器602配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位，以及根据识别和定位到的手势特征在手语库中进行匹配生成数字信号，所述手势库包括标准的语音信号；所述射频模块603配置为发送根据在手语库中匹配结果生成的数字信号。6 is a schematic structural diagram of a smart terminal according to Embodiment 6 of the present application; as shown in FIG. 6, the camera 601 may include a camera 601, a hardware processor 602, and a radio frequency module 603. The camera 601 captures a sign language gesture and collects the captured sign language gesture to form a sign language. a sequence of gesture images; the hardware processor 602 is configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images, and generate a digital signal by matching in the sign language library according to the recognized and positioned gesture features. The gesture library includes a standard voice signal; the radio frequency module 603 is configured to transmit a digital signal generated according to a matching result in the sign language library.
另外一实施例中，摄像头可被智能终端的手写屏幕代替，用于捕获手语手势并采集捕获到的手语手势形成手语手势图像序列，详细不再附图说明。 In another embodiment, the camera can be replaced by a handwriting screen of the smart terminal for capturing a sign language gesture and collecting the captured sign language gesture to form a sequence of sign language gesture images, which are not illustrated in detail.
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, ie may be located A place, or it can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, optical discs, etc., include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。 Finally, it should be noted that the above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced by the equivalents. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (17)

  1. 一种与聋哑人进行信息交流的方法，其特征在于，包括：A method for communicating information with deaf people, characterized in that it comprises:
    接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；Receiving a voice signal, and matching the voice signal in a pre-established sign language library;
    根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；Selecting standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
    呈现从所述手语库中选择与所述语音信号对应标准手语信息。The presentation selects standard sign language information corresponding to the voice signal from the sign language library.
  2. 根据就权利要求1所述的方法，其特征在于，所述接收语音信号，在预先建立的手语库中对所述语音信号进行匹配包括：The method of claim 1 wherein said receiving a speech signal and matching said speech signal in a pre-established sign language library comprises:
    接收语音信号，将所述语音信号转换成数字信号，在预先建立的手语库中对所述语音信号对应的数字信号进行匹配。Receiving a voice signal, converting the voice signal into a digital signal, and matching the digital signal corresponding to the voice signal in a pre-established sign language library.
  3. 根据权利要求2所述的方法，其特征在于，所述根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息包括：The method according to claim 2, wherein the selecting the standard sign language information corresponding to the voice signal from the sign language library according to the matching result of the voice signal in the pre-established sign language library comprises:
    根据对所述语音信号对应的数字信号的匹配结果，从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。Selecting standard sign language information matching the digital signal corresponding to the voice signal from the sign language library according to a matching result of the digital signal corresponding to the voice signal.
  4. 根据权利要求1所述的方法，其特征在于，所述呈现从所述手语库中选择与所述语音信号对应标准手语信息包括：The method according to claim 1, wherein the presenting the selection of the standard sign language information corresponding to the voice signal from the sign language library comprises:
    显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。Displaying standard sign language information that matches a digital signal corresponding to the voice signal from the sign language library.
  5. 根据权利要求4所述的方法，其特征在于，所述显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息包括：以图形方式显示从所述手语库中选择与所述语音信号对应的数字信号匹配的标准手语信息。The method according to claim 4, wherein said displaying standard sign language information matching said digital signal corresponding to said voice signal from said sign language library comprises: graphically displaying selection from said sign language library Standard sign language information that matches the digital signal corresponding to the speech signal.
  6. 根据权利要求1-5任一项所述的方法，其特征在于，还包括：判断是否启动聋哑交互，如果是，则执行启动标准语音信号的接收；否则，结束。The method of any of claims 1-5, further comprising: determining whether to initiate a deaf-mute interaction, and if so, performing to initiate reception of the standard speech signal; otherwise, ending.
  7. 一种与聋哑人进行信息交流的方法，其特征在于，包括：A method for communicating information with deaf people, characterized in that it comprises:
    捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；Capturing a sign language gesture and collecting the captured sign language gesture to form a sequence of sign language gesture images;
    对手语手势图像序列中每个手语手势图像进行手势特征识别和定 位；Gesture feature recognition and determination for each sign language gesture image in a sequence of opponent gesture images Bit
    根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；Generating a digital signal based on the identified and located gesture features in the sign language library;
    发送根据在手语库中匹配结果生成的数字信号。Sends a digital signal generated based on the matching result in the sign language library.
  8. 根据权利要求7所述的方法，其特征在于，所述捕获手语手势并采集捕获到的手语手势形成手语手势图像序列包括：The method according to claim 7, wherein the capturing the sign language gesture and collecting the captured sign language gesture to form the sign language gesture image sequence comprises:
    捕获手语手势并采集捕获到的手语手势，形成手语手势图像序列，将所述手语手势图像序列转换成数字信号。The sign language gesture is captured and the captured sign language gesture is captured, a sequence of sign language gesture images is formed, and the sequence of sign language gesture images is converted into a digital signal.
  9. 根据权利要求8所述的方法，其特征在于，所述根据识别和定位到的手势特征在手语库中进行匹配生成数字信号：The method of claim 8 wherein said matching in said sign language based on the identified and located gesture features generates a digital signal:
    根据识别和定位到的手势特征在对应手语手势图像上的数字信号，在手语库中进行匹配生成语音信号。A matching and generating a speech signal is performed in the sign language library according to the recognized and positioned gesture feature on the digital signal corresponding to the sign language gesture image.
  10. 根据权利要求7所述的方法，其特征在于，所述对手语手势图像序列中每个手语手势图像进行手势特征识别和定位包括：The method according to claim 7, wherein the gesture feature recognition and positioning of each of the sign language gesture images in the sequence of hand gesture gesture images comprises:
    对手语手势图像序列中每个手语手势图像手势分解，在手势分解了的手语手势图像上进行手势特征识别和定位。Each sign language gesture image gesture decomposition in the opponent gesture image sequence is performed, and gesture feature recognition and positioning are performed on the gesture-decomposed sign language gesture image.
  11. 根据权利要求7所述的方法，其特征在于，还包括：判断是否启动聋哑交互，如果是，则执行启动手语手势的捕获；否则，结束。The method of claim 7, further comprising: determining whether to initiate a deaf-mute interaction, and if so, performing a capture of the initiating sign language gesture; otherwise, ending.
  12. 一种与聋哑人进行信息交流的装置，其特征在于，包括：A device for communicating information with a deaf person, characterized in that it comprises:
    接收模块，配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；a receiving module configured to receive a voice signal, and match the voice signal in a pre-established sign language library;
    匹配模块，配置为根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；a matching module, configured to select a standard sign language information corresponding to the voice signal from the sign language library according to a matching result of the voice signal in a pre-established sign language library;
    呈现模块，配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。And a presentation module configured to present, from the sign language library, select standard sign language information corresponding to the voice signal.
  13. 一种与聋哑人进行信息交流的装置，其特征在于，包括：A device for communicating information with a deaf person, characterized in that it comprises:
    采集模块，配置为捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；An acquisition module configured to capture a sign language gesture and collect the captured sign language gesture to form a sequence of sign language gesture images;
    定位模块，配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位； a positioning module configured to perform gesture feature recognition and positioning for each of the sign language gesture images in the sequence of hand gesture gesture images;
    匹配模块，配置为根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；a matching module configured to perform a matching in the sign language library according to the recognized and positioned gesture features to generate a digital signal;
    发送模块，配置为发送根据在手语库中匹配结果生成的数字信号。A sending module configured to send a digital signal generated based on a matching result in the sign language library.
  14. 一种智能终端，其特征在于，包括呈现模块以及硬件处理器，所述硬件处理器配置为接收语音信号，在预先建立的手语库中对所述语音信号进行匹配；以及根据在预先建立的手语库中对所述语音信号的匹配结果，从所述手语库中选择与所述语音信号对应标准手语信息；所述呈现模块配置为呈现从所述手语库中选择与所述语音信号对应标准手语信息。An intelligent terminal, comprising: a presentation module and a hardware processor, the hardware processor configured to receive a voice signal, to match the voice signal in a pre-established sign language library; and according to a pre-established sign language Selecting a standard sign language information corresponding to the voice signal from the sign language library; the rendering module is configured to select a standard sign language corresponding to the voice signal from the sign language library information.
  15. 一种智能终端，其特征在于，包括摄像头、硬件处理器以及射频模块，所述摄像头捕获手语手势并采集捕获到的手语手势形成手语手势图像序列；所述硬件处理器配置为对手语手势图像序列中每个手语手势图像进行手势特征识别和定位；以及根据识别和定位到的手势特征在手语库中进行匹配生成数字信号；所述射频模块配置为发送根据在手语库中匹配结果生成的数字信号。An intelligent terminal, comprising: a camera, a hardware processor, and a radio frequency module, wherein the camera captures a sign language gesture and collects the captured sign language gesture to form a sequence of sign language gesture images; the hardware processor is configured as a sequence of hand gesture gesture images Each of the sign language gesture images performs gesture feature recognition and positioning; and generates a digital signal by matching in the sign language library according to the recognized and positioned gesture features; the radio frequency module is configured to transmit the digital signal generated according to the matching result in the sign language library .
  16. 一种在其上记录有用于执行权利要求1所述方法的程序的计算机可读记录介质。A computer readable recording medium having recorded thereon a program for executing the method of claim 1.
  17. 一种在其上记录有用于执行权利要求7所述方法的程序的计算机可读记录介质。 A computer readable recording medium having recorded thereon a program for executing the method of claim 7.
PCT/CN2016/088485 2016-03-23 2016-07-04 Method and device for communicating information with deaf-mutes, smart terminal WO2017161741A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610168501.XA CN105868282A (en) 2016-03-23 2016-03-23 Method and apparatus used by deaf-mute to perform information communication, and intelligent terminal
CN201610168501.X 2016-03-23

Publications (1)

Publication Number Publication Date
WO2017161741A1 true WO2017161741A1 (en) 2017-09-28

Family

ID=56625530

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/088485 WO2017161741A1 (en) 2016-03-23 2016-07-04 Method and device for communicating information with deaf-mutes, smart terminal

Country Status (2)

Country Link
CN (1) CN105868282A (en)
WO (1) WO2017161741A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109858043A (en) * 2018-12-19 2019-06-07 上海立势达机电科技有限公司 A kind of sign language and speech language Bidirectional intelligent translating equipment
CN110008839A (en) * 2019-03-08 2019-07-12 西安研硕信息技术有限公司 A kind of intelligent sign language interactive system and method for adaptive gesture identification
CN110111651A (en) * 2018-02-01 2019-08-09 周玮 Intelligent language interactive system based on posture perception
CN112256827A (en) * 2020-10-20 2021-01-22 平安科技（深圳）有限公司 Sign language translation method and device, computer equipment and storage medium
CN116805272A (en) * 2022-10-29 2023-09-26 武汉行已学教育咨询有限公司 Visual education teaching analysis method, system and storage medium

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106653051A (en) * 2016-12-09 2017-05-10 天脉聚源(北京)传媒科技有限公司 Video deaf-mute mode expression method and apparatus
CN106789593B (en) * 2017-01-13 2019-01-11 山东师范大学 A kind of instant message processing method, server and system merging sign language
US10854110B2 (en) 2017-03-03 2020-12-01 Microsoft Technology Licensing, Llc Automated real time interpreter service
CN108629241B (en) * 2017-03-23 2022-01-14 华为技术有限公司 Data processing method and data processing equipment
CN107798964A (en) * 2017-11-24 2018-03-13 郑军 The sign language intelligent interaction device and its exchange method of a kind of Real time identification gesture
CN110555329A (en) * 2018-05-31 2019-12-10 苏州欧力机器人有限公司 Sign language translation method, terminal and storage medium
CN108960158A (en) * 2018-07-09 2018-12-07 珠海格力电器股份有限公司 Intelligent sign language translation system and method
CN111461005B (en) * 2020-03-31 2023-11-28 腾讯科技（深圳）有限公司 Gesture recognition method and device, computer equipment and storage medium
CN113851029B (en) * 2021-07-30 2023-09-05 阿里巴巴达摩院(杭州)科技有限公司 Barrier-free communication method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594434A (en) * 2009-06-16 2009-12-02 中兴通讯股份有限公司 The sign language processing method and the sign language processing mobile terminal of portable terminal
CN101661330A (en) * 2009-09-22 2010-03-03 深圳华为通信技术有限公司 Method for converting sign language and terminal thereof
CN101794528A (en) * 2010-04-02 2010-08-04 北京大学软件与微电子学院无锡产学研合作教育基地 Gesture language-voice bidirectional translation system
US20140171036A1 (en) * 2009-11-18 2014-06-19 Gwendolyn Simmons Method of communication
CN103956167A (en) * 2014-05-06 2014-07-30 北京邮电大学 Visual sign language interpretation method and device based on Web

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101605158A (en) * 2008-06-13 2009-12-16 鸿富锦精密工业（深圳）有限公司 Mobile phone dedicated for deaf-mutes
CN101539994B (en) * 2009-04-16 2012-07-04 西安交通大学 Mutually translating system and method of sign language and speech
US8640021B2 (en) * 2010-11-12 2014-01-28 Microsoft Corporation Audience-based presentation and customization of content
CN102497513A (en) * 2011-11-25 2012-06-13 中山大学 Video virtual hand language system facing digital television

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594434A (en) * 2009-06-16 2009-12-02 中兴通讯股份有限公司 The sign language processing method and the sign language processing mobile terminal of portable terminal
CN101661330A (en) * 2009-09-22 2010-03-03 深圳华为通信技术有限公司 Method for converting sign language and terminal thereof
US20140171036A1 (en) * 2009-11-18 2014-06-19 Gwendolyn Simmons Method of communication
CN101794528A (en) * 2010-04-02 2010-08-04 北京大学软件与微电子学院无锡产学研合作教育基地 Gesture language-voice bidirectional translation system
CN103956167A (en) * 2014-05-06 2014-07-30 北京邮电大学 Visual sign language interpretation method and device based on Web

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110111651A (en) * 2018-02-01 2019-08-09 周玮 Intelligent language interactive system based on posture perception
CN109858043A (en) * 2018-12-19 2019-06-07 上海立势达机电科技有限公司 A kind of sign language and speech language Bidirectional intelligent translating equipment
CN109858043B (en) * 2018-12-19 2023-10-27 上海立势达机电科技有限公司 Gesture language and voice language bidirectional intelligent translation device
CN110008839A (en) * 2019-03-08 2019-07-12 西安研硕信息技术有限公司 A kind of intelligent sign language interactive system and method for adaptive gesture identification
CN110008839B (en) * 2019-03-08 2023-01-03 陕西联森电子科技有限公司 Intelligent sign language interaction system and method for self-adaptive gesture recognition
CN112256827A (en) * 2020-10-20 2021-01-22 平安科技（深圳）有限公司 Sign language translation method and device, computer equipment and storage medium
CN116805272A (en) * 2022-10-29 2023-09-26 武汉行已学教育咨询有限公司 Visual education teaching analysis method, system and storage medium

Also Published As

Publication number Publication date
CN105868282A (en) 2016-08-17

Similar Documents

Publication Publication Date Title
WO2017161741A1 (en) Method and device for communicating information with deaf-mutes, smart terminal
US11580983B2 (en) Sign language information processing method and apparatus, electronic device and readable storage medium
KR101786944B1 (en) Speaker displaying method and videophone terminal therefor
CN104753766B (en) Expression sending method and device
WO2016169095A1 (en) Terminal alarm method and apparatus
CN109309751B (en) Voice recording method, electronic device and storage medium
WO2017072534A2 (en) Communication system and method
US20190251972A1 (en) Speech processing method and device
CN108960158A (en) Intelligent sign language translation system and method
CN109257498B (en) Sound processing method and mobile terminal
US11321280B2 (en) Multimedia file sharing method and terminal device
CN111508531B (en) Audio processing method and device
TW201807565A (en) Voice-based information sharing method, device, and mobile terminal
WO2016187910A1 (en) Voice-to-text conversion method and device, and storage medium
CN107452383B (en) Information processing method, server, terminal and information processing system
CN102655544A (en) Method for issuing communication, and communication terminal
CN107888965A (en) Image present methods of exhibiting and device, terminal, system, storage medium
CN108763475B (en) Recording method, recording device and terminal equipment
WO2016202277A1 (en) Message sending method and mobile terminal
CN113055529B (en) Recording control method and recording control device
CN106791226B (en) Call fault detection method and system
CN110111795B (en) Voice processing method and terminal equipment
CN111382598B (en) Identification method and device and electronic equipment
CN108491471B (en) Text information processing method and mobile terminal
CN109887492A (en) A kind of data processing method, device and electronic equipment

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16895092

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16895092

Country of ref document: EP

Kind code of ref document: A1