CN106254960A - A kind of video call method for communication disorders and system - Google Patents
A kind of video call method for communication disorders and system Download PDFInfo
- Publication number
- CN106254960A CN106254960A CN201610769730.7A CN201610769730A CN106254960A CN 106254960 A CN106254960 A CN 106254960A CN 201610769730 A CN201610769730 A CN 201610769730A CN 106254960 A CN106254960 A CN 106254960A
- Authority
- CN
- China
- Prior art keywords
- sign language
- video
- module
- video calling
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000030251 communication disease Diseases 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims description 27
- 230000006870 function Effects 0.000 claims abstract description 46
- 238000006243 chemical reaction Methods 0.000 claims abstract description 13
- 238000004891 communication Methods 0.000 claims abstract description 11
- 238000012856 packing Methods 0.000 claims abstract description 10
- 230000005540 biological transmission Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 206010011878 Deafness Diseases 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/426—Internal components of the client ; Characteristics thereof
- H04N21/42653—Internal components of the client ; Characteristics thereof for processing graphics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Telephonic Communication Services (AREA)
Abstract
The present invention provides a kind of video call system for communication disorders, described system to include: video calling originating end, sign language language identification server, video calling miscellaneous function server and video calling destination end；Described sign language language identification server, video calling miscellaneous function server are connected with video calling originating end and video calling destination end by communication network；After the participant of communication disorders is used sign language language to exchange by described video calling originating end, by sign language language identification server, it is text subtile information by sign language language conversion；Video calling originating end video the most at last, audio frequency and the packing of text subtile data, and then by video calling miscellaneous function server, video call data delivered to video calling destination end.Present invention achieves the video calling participant of communication disorders, it is possible to participate in video calling.
Description
Technical field
The present invention relates to set-top box technique field, particularly relate to a kind of video call method for communication disorders and be
System.
Background technology
Video calling is often referred to hold based on the Internet and mobile Internet (3G the Internet), by real-time between intelligent terminal
Transmit voice and a kind of communication mode of image (bust of user, photo, article etc.) of people.Video calling prevailing transmission
It is image and sound.Special population, may be in the face of some special difficulties when participating in video calling.Special population (deaf mute)
Use sign language exchanges, and they cannot the most effectively link up before normal video calling participant.
Summary of the invention
One of the technical problem to be solved in the present invention, is to provide a kind of video call system for communication disorders, real
Show the video calling participant of communication disorders, it is possible to carrying out video calling with linking up normal person, person provides for communication disorders
Convenient.
One of problem of the present invention is achieved in that a kind of video call system for communication disorders, described system bag
Include: video calling originating end, sign language language identification server, video calling miscellaneous function server and video calling target
End；Described sign language language identification server, video calling miscellaneous function server by communication network and video calling originating end and
Video calling destination end connects；
After the participant of communication disorders is used sign language language to exchange by described video calling originating end, by sign language language
Speech identifies server, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video calling originating end video the most at last, audio-frequency information and the packing of text subtile data, and then pass through video
Video call data is delivered to video calling destination end by call miscellaneous function server.
Further, described video calling originating end is provided with in the middle of hardware driving, operating system module, video calling
Part module, Sign Language Recognition engine, sign language turns captioning module, captions turn sound module, video/audio/captions coding packetization module
And video calling transport module；
Described hardware driving is that the software interface of device hardware is abstract；
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user's figure
As information, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, by written for the conversion of sign language implication
Word caption information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream,
Then Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
Further, described hardware driving includes that processor driving, communication interface driving, audio driven and video are compiled firmly
Code drives.
Further, described Sign Language Recognition engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module,
Sign Language Recognition implements module and Sign Language Recognition management module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.
Further, the operation principle of described Sign Language Recognition engine: the people of communication disorders is carried out video pictures collection；Again
The image binaryzation pretreatment that will gather；And it is semantic with identification, sign language segmentation, sign language Semantic mapping and sign language to carry out sign language tracking
Lard speech with literary allusions word, thus complete gesture identification.
Further, described video video phone system carries out video calling operation particularly as follows: described video calling is initiated
End gathers the video pictures of participant, and then by video pictures to sign language language identification server process；Sign Language Recognition is mainly entered
The following operation of row: call Sign Language Recognition engine and identify sign language information；Sign language turns captioning module by being converted into by sign language information
Text subtile information；Call captions and turn sound module, caption information is converted to acoustic information；By caption information and acoustic information
Returning to video calling originating end, the multi-medium data of video calling is compiled by video calling originating end by video/audio/captions
Code packetization module is packed, and then calls the video calling transport module of video calling middleware module, by video calling
Data pass through video calling miscellaneous function server transport to video calling destination end.
The two of this technical problem to be solved in the present invention, are to provide a kind of video call method for communication disorders,
Achieve the video calling participant of communication disorders, it is possible to carry out video calling with linking up normal person, provide for communication disorders person
Convenience.
The two of problem of the present invention are achieved in that a kind of video call method for communication disorders, it is characterised in that:
Described method need to provide video calling originating end, sign language language identification server, video calling miscellaneous function server and regard
Frequently call target end；
The participant of communication disorders uses sign language language to exchange, by sign language language identification at video calling originating end
Server, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video call terminal video the most at last, audio-frequency information and the packing of text subtile data, and then led to by video
Video call data is delivered to video calling destination end by words miscellaneous function server.
Further, described video calling originating end is provided with in the middle of hardware driving, operating system module, video calling
Part module, Sign Language Recognition engine, sign language turns captioning module, captions turn sound module, video/audio/captions coding packetization module
And video calling transport module；
Described hardware driving is that the software interface of device hardware is abstract；
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user's figure
As information, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, by written for the conversion of sign language implication
Word caption information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream,
Then Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
Further, described method is further particularly as follows: described video video phone system carries out video calling operation tool
Body is: described video calling originating end gathers the video pictures of participant, and then is serviced to sign language language identification by video pictures
Device processes；Sign Language Recognition is substantially carried out following operation: calls Sign Language Recognition engine and identifies sign language information；Sign language turns captioning module
By sign language information being converted into text subtile information；Call captions and turn sound module, caption information is converted to acoustic information；
Caption information and acoustic information return to video calling originating end, and video calling originating end is by the multi-medium data of video calling
Packed by video/audio/captions coding packetization module, and then the video calling calling video calling middleware module passes
Defeated module, by the data of video calling by video calling miscellaneous function server transport to video calling destination end.
Further, described hardware driving includes that processor driving, communication interface driving, audio driven and video are compiled firmly
Code drives.
Further, described Sign Language Recognition engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module,
Sign Language Recognition implements module and Sign Language Recognition management module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.
Further, the operation principle of described Sign Language Recognition engine: the people of communication disorders is carried out video pictures collection；Again
The image binaryzation pretreatment that will gather；And it is semantic with identification, sign language segmentation, sign language Semantic mapping and sign language to carry out sign language tracking
Lard speech with literary allusions word, thus complete gesture identification.
Present invention have the advantage that the present invention makes the video calling participant of communication disorders, use sign language language to carry out
Exchange, by sign language language identification server, is text subtile information by sign language language conversion.Video call terminal regards the most at last
Frequently, audio frequency and caption data packing, and then by video calling miscellaneous function server, video call data is delivered to video and leads to
Words destination end.It is achieved thereby that the video calling participant of communication disorders, it is possible to carry out video calling, for ditch with linking up normal person
Logical obstacle person provides conveniently.
Accompanying drawing explanation
The present invention is further illustrated the most in conjunction with the embodiments.
Fig. 1 is the system overall framework figure of the present invention.
Fig. 2 is the structural representation of each module in video call terminal of the present invention.
Fig. 3 is the fundamental diagram of Sign Language Recognition of the present invention.
Fig. 4 is the inventive method operating process schematic diagram.
Detailed description of the invention
Referring to shown in Fig. 1 to Fig. 3, video call terminal is interconnected by Base communication net (the Internet etc.).Video
Call comprises the outside sign language language identification server strengthening call function and video calling miscellaneous function server.Server merit
The division of energy is to divide on function logic, not divides from physical logic, i.e. sign language language identification server and video calling
Miscellaneous function server is likely to be present on same station server main frame.The efficient combination of the participation main body of video calling is:
Communication disorders participant and communication disorders participant (need not special handling)；Link up normal participant and link up normal participant
(need not special handling)；Communication disorders participant and the normal participant of communication (needing special handling).
A kind of video call system for communication disorders of the present invention, described system includes: video calling originating end (
As be that the participant of communication disorders uses), sign language language identification server, video calling miscellaneous function server and video lead to
Words destination end (is usually linked up normal participant to use)；Described sign language language identification server, video calling miscellaneous function
Server is connected with video calling originating end and video calling destination end by communication network；
After the participant of communication disorders is used sign language language to exchange by described video calling originating end, by sign language language
Speech identifies server, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video calling originating end video the most at last, audio-frequency information and the packing of text subtile data, and then pass through video
Video call data is delivered to video calling destination end by call miscellaneous function server.
In the present invention, described video calling originating end is provided with in hardware driving, operating system module, video calling
Between part module, Sign Language Recognition engine, sign language turns captioning module, captions turn sound module, video/audio/captions coding packing mould
Block and video calling transport module；
Described hardware driving is that the software interface of device hardware is abstract；Described hardware driving includes that processor drives, communicates
Interface driver, audio driven and video hard coded drive.
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user's figure
As information, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, by written for the conversion of sign language implication
Word caption information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream,
Then Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
Described Sign Language Recognition engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module, Sign Language Recognition are in fact
Execute module and Sign Language Recognition management module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；I.e. configuration makes
With which kind of Sign Language Recognition server (oneself or the most third-party)
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.Sign language is known
The upgrading of other engine engine for convenience, maintenance and expansion, optimal enforcement is to be deployed in video calling miscellaneous function server
On.Sign Language Recognition engine distribution is on video calling miscellaneous function server；Sign Language Recognition interface (API) is deployed in video calling
In client.Sign Language Recognition provider management module, is responsible for and safeguards the concrete real of multiple Sign Language Recognition interface (API)
Executing, these are embodied as being likely located on third party's Sign Language Recognition server.Choosing is responsible in Sign Language Recognition Service Operation policy module
Select the embodiment of final Sign Language Recognition interface.
Wherein, the operation principle of described Sign Language Recognition engine: the people of communication disorders is carried out video pictures collection；To adopt again
The image binaryzation pretreatment of collection；And carry out sign language follow the trail of lard speech with literary allusions with identification, sign language segmentation, sign language Semantic mapping and sign language semanteme
Word, thus complete gesture identification.
As shown in Figure 4, described video video phone system carries out video calling operation particularly as follows: described video calling is initiated
End gathers the video pictures of participant, and then by video pictures to sign language language identification server process；Sign Language Recognition is mainly entered
The following operation of row: call Sign Language Recognition engine and identify sign language information；Sign language turns captioning module by being converted into by sign language information
Text subtile information；Call captions and turn sound module, caption information is converted to acoustic information；By caption information and acoustic information
Returning to video calling originating end, the multi-medium data of video calling is compiled by video calling originating end by video/audio/captions
Code packetization module is carried out pack (video/audio/captions), and then calls the video calling transmission mould of video calling middleware module
Block, by the data of video calling by video calling miscellaneous function server transport to video calling destination end.
Referring to shown in Fig. 2 to Fig. 4, a kind of video call method for communication disorders of the present invention, described method needs
Video calling originating end, sign language language identification server, video calling miscellaneous function server and video calling target are provided
End；
The participant of communication disorders uses sign language language to exchange, by sign language language identification at video calling originating end
Server, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video call terminal video the most at last, audio-frequency information and the packing of text subtile data, and then led to by video
Video call data is delivered to video calling destination end by words miscellaneous function server.
Described video calling originating end is provided with hardware driving, operating system module, video calling middleware module, hands
Language identification engine, sign language turn captioning module, captions turn sound module, video/audio/captions encode packetization module and video leads to
Words transport module；
Described hardware driving is that the software interface of device hardware is abstract；Described hardware driving includes that processor drives, communicates
Interface driver, audio driven and video hard coded drive.
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user's figure
As information, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, by written for the conversion of sign language implication
Word caption information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream,
Then Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
In the present invention, described method is further particularly as follows: described video video phone system carries out video calling operation
Particularly as follows: described video calling originating end gathers the video pictures of participant, and then video pictures is taken to sign language language identification
Business device processes；Sign Language Recognition is substantially carried out following operation: calls Sign Language Recognition engine and identifies sign language information；Sign language turns captions mould
Block by being converted into text subtile information by sign language information；Call captions and turn sound module, caption information is converted to sound letter
Breath；Caption information and acoustic information return to video calling originating end, and video calling originating end is by the multimedia of video calling
Data are packed by video/audio/captions coding packetization module, and then the video calling video calling middleware module leads to
Words transport module, by the data of video calling by video calling miscellaneous function server transport to video calling destination end.
Described Sign Language Recognition engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module, Sign Language Recognition are in fact
Execute module and Sign Language Recognition management module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.Sign language is known
The upgrading of other engine engine for convenience, maintenance and expansion, optimal enforcement is to be deployed in video calling miscellaneous function server
On.Sign Language Recognition engine distribution is on video calling miscellaneous function server；Sign Language Recognition interface (API) is deployed in video calling
In client.Sign Language Recognition provider management module, is responsible for and safeguards the concrete real of multiple Sign Language Recognition interface (API)
Executing, these are embodied as being likely located on third party's Sign Language Recognition server.Choosing is responsible in Sign Language Recognition Service Operation policy module
Select the embodiment of final Sign Language Recognition interface.
Wherein, the operation principle of described Sign Language Recognition engine: the people of communication disorders is carried out video pictures collection；To adopt again
The image binaryzation pretreatment of collection；And carry out sign language follow the trail of lard speech with literary allusions with identification, sign language segmentation, sign language Semantic mapping and sign language semanteme
Word, thus complete gesture identification.
In a word, the present invention makes the video calling participant of communication disorders, uses sign language language to exchange, by sign language language
Speech identifies server, is text subtile information by sign language language conversion.Video call terminal video the most at last, audio frequency and captions number
According to packing, and then by video calling miscellaneous function server, video call data delivered to video calling destination end.Thus it is real
Show the video calling participant of communication disorders, it is possible to carrying out video calling with linking up normal person, person provides for communication disorders
Convenient.
Although the foregoing describing the detailed description of the invention of the present invention, but those familiar with the art should managing
Solving, our described specific embodiment is merely exemplary rather than for the restriction to the scope of the present invention, is familiar with this
The technical staff in field, in the equivalent modification made according to the spirit of the present invention and change, should be contained the present invention's
In scope of the claimed protection.
Claims (12)
1. the video call system for communication disorders, it is characterised in that: described system includes: video calling originating end,
Sign language language identification server, video calling miscellaneous function server and video calling destination end；Described sign language language identification
Server, video calling miscellaneous function server are connected with video calling originating end and video calling destination end by communication network；
After the participant of communication disorders is used sign language language to exchange by described video calling originating end, known by sign language language
Other server, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video calling originating end video the most at last, audio-frequency information and the packing of text subtile data, and then pass through video calling
Video call data is delivered to video calling destination end by miscellaneous function server.
A kind of video call system for communication disorders the most according to claim 1, it is characterised in that: described video leads to
Be provided with hardware driving in words originating end, operating system module, video calling middleware module, Sign Language Recognition engine, sign language turn
Captioning module, captions turn sound module, video/audio/captions coding packetization module and video calling transport module；
Described hardware driving is that the software interface of device hardware is abstract；
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user images letter
Breath, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, sign language implication be converted into word word
Curtain information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream, then
Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
A kind of video call system for communication disorders the most according to claim 2, it is characterised in that: described hardware drives
Move and include that processor driving, communication interface driving, audio driven and video hard coded drive.
A kind of video call system for communication disorders the most according to claim 2, it is characterised in that: described sign language is known
Other engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module, Sign Language Recognition implement module and sign language knowledge
Don't bother about reason module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.
A kind of video call system for communication disorders the most according to claim 2, it is characterised in that: described sign language is known
The operation principle of other engine: the people of communication disorders is carried out video pictures collection；The image binaryzation pretreatment that will gather again；And
Carry out sign language to follow the trail of and lard speech with literary allusions word with identification, sign language segmentation, sign language Semantic mapping and sign language semanteme, thus complete gesture identification.
A kind of video call system for communication disorders the most according to claim 2, it is characterised in that: described video regards
Frequently phone system carries out video calling operation particularly as follows: the video pictures of described video calling originating end collection participant, and then
By video pictures to sign language language identification server process；Sign Language Recognition is substantially carried out following operation: call Sign Language Recognition engine
Identify sign language information；Sign language turns captioning module by sign language information is converted into text subtile information；Call captions and turn sound
Module, is converted to acoustic information by caption information；Caption information and acoustic information are returned to video calling originating end, and video leads to
The multi-medium data of video calling is packed by video/audio/captions coding packetization module, and then is called by words originating end
The data of video calling are serviced by the video calling transport module of video calling middleware module by video calling miscellaneous function
Device is transferred to video calling destination end.
7. the video call method for communication disorders, it is characterised in that: described method need to provide video calling originating end,
Sign language language identification server, video calling miscellaneous function server and video calling destination end；
The participant of communication disorders uses sign language language to exchange at video calling originating end, is serviced by sign language language identification
Device, is text subtile information by sign language language conversion, and text subtile information is converted into digitized audio message；
Described video call terminal video the most at last, audio-frequency information and the packing of text subtile data, so auxiliary by video calling
Help function server that video call data is delivered to video calling destination end.
A kind of video call method for communication disorders the most according to claim 7, it is characterised in that: described video leads to
Be provided with hardware driving in words originating end, operating system module, video calling middleware module, Sign Language Recognition engine, sign language turn
Captioning module, captions turn sound module, video/audio/captions coding packetization module and video calling transport module；
Described hardware driving is that the software interface of device hardware is abstract；
Described operating system module is the basis that equipment runs other softwares；
Described video calling middleware module, realizes the general name of the repertoire interface of video calling by software；
Described Sign Language Recognition engine, is used for identifying sign language information；
Described sign language turns captioning module, and the gesture information of collection is converted into text subtile information, including gathering user images letter
Breath, gesture identification, gesture information and specific action comparison, identify corresponding sign language implication, sign language implication be converted into word word
Curtain information；
Described captions turn sound module, for transferring word to sound；
Described video/audio/captions coding packetization module, have identified gesture information, and changes into audio stream and caption stream, then
Video stream information, audio stream and caption stream three road stream are repacked；
Described video calling transport module, the i.e. function of the transmission of video calling middleware module.
A kind of video call method for communication disorders the most according to claim 8, it is characterised in that: described method is entered
One step is particularly as follows: described video video phone system carries out video calling operation particularly as follows: described video calling originating end collection
The video pictures of participant, and then by video pictures to sign language language identification server process；Sign Language Recognition is substantially carried out following
Operation: call Sign Language Recognition engine and identify sign language information；Sign language turns captioning module by sign language information is converted into word word
Curtain information；Call captions and turn sound module, caption information is converted to acoustic information；Caption information and acoustic information are returned to
Video calling originating end, the multi-medium data of video calling is packed by video calling originating end by video/audio/captions coding
Module is packed, and then calls the video calling transport module of video calling middleware module, the data of video calling is led to
Cross video calling miscellaneous function server transport to video calling destination end.
A kind of video call method for communication disorders the most according to claim 8, it is characterised in that: described hardware
Driving includes that processor driving, communication interface driving, audio driven and video hard coded drive.
11. a kind of video call methods for communication disorders according to claim 8, it is characterised in that: described sign language
Identification engine includes: Sign Language Recognition interface, Sign Language Recognition Service Operation policy module, Sign Language Recognition implement module and sign language
Identify management module；
Described Sign Language Recognition interface has been the definition of required interface on Sign Language Recognition function logic；
Described Sign Language Recognition Service Operation policy module, selects the embodiment of final Sign Language Recognition interface；
Described Sign Language Recognition implements module, for the enforcement to specific embodiment；
Described Sign Language Recognition management module, is responsible for and safeguards being embodied as of multiple Sign Language Recognition interface.
12. a kind of video call methods for communication disorders according to claim 8, it is characterised in that: described sign language
Identify the operation principle of engine: the people of communication disorders is carried out video pictures collection；The image binaryzation pretreatment that will gather again；
And carry out sign language and follow the trail of and lard speech with literary allusions word with identification, sign language segmentation, sign language Semantic mapping and sign language semanteme, thus complete gesture identification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610769730.7A CN106254960A (en) | 2016-08-30 | 2016-08-30 | A kind of video call method for communication disorders and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610769730.7A CN106254960A (en) | 2016-08-30 | 2016-08-30 | A kind of video call method for communication disorders and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106254960A true CN106254960A (en) | 2016-12-21 |
Family
ID=58080544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610769730.7A Pending CN106254960A (en) | 2016-08-30 | 2016-08-30 | A kind of video call method for communication disorders and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106254960A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106943740A (en) * | 2017-04-25 | 2017-07-14 | 合肥充盈信息科技有限公司 | A kind of gesture language-voice game interactive system |
| WO2018001088A1 (en) * | 2016-06-30 | 2018-01-04 | 中兴通讯股份有限公司 | Method and apparatus for presenting communication information, device and set-top box |
| CN111144367A (en) * | 2019-12-31 | 2020-05-12 | 重庆百事得大牛机器人有限公司 | Auxiliary semantic recognition method based on gesture recognition |
| CN113923471A (en) * | 2021-12-10 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Interaction method, device, equipment and storage medium |
| WO2023066023A1 (en) * | 2021-10-20 | 2023-04-27 | 中兴通讯股份有限公司 | Gesture-based communication method and apparatus, storage medium, and electronic apparatus |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101453611A (en) * | 2007-12-07 | 2009-06-10 | 希姆通信息技术（上海）有限公司 | Method for video communication between the deaf and the normal |
| CN101527092A (en) * | 2009-04-08 | 2009-09-09 | 西安理工大学 | Computer assisted hand language communication method under special session context |
| CN104125548A (en) * | 2013-04-27 | 2014-10-29 | 中国移动通信集团公司 | Method of translating conversation language, device and system |
| KR20150086902A (en) * | 2014-01-21 | 2015-07-29 | 박삼기 | Finger-language translation providing system for deaf person |
| CN105100482A (en) * | 2015-07-30 | 2015-11-25 | 努比亚技术有限公司 | Mobile terminal and system for realizing sign language identification, and conversation realization method of the mobile terminal |
-
2016
- 2016-08-30 CN CN201610769730.7A patent/CN106254960A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101453611A (en) * | 2007-12-07 | 2009-06-10 | 希姆通信息技术（上海）有限公司 | Method for video communication between the deaf and the normal |
| CN101527092A (en) * | 2009-04-08 | 2009-09-09 | 西安理工大学 | Computer assisted hand language communication method under special session context |
| CN104125548A (en) * | 2013-04-27 | 2014-10-29 | 中国移动通信集团公司 | Method of translating conversation language, device and system |
| KR20150086902A (en) * | 2014-01-21 | 2015-07-29 | 박삼기 | Finger-language translation providing system for deaf person |
| CN105100482A (en) * | 2015-07-30 | 2015-11-25 | 努比亚技术有限公司 | Mobile terminal and system for realizing sign language identification, and conversation realization method of the mobile terminal |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018001088A1 (en) * | 2016-06-30 | 2018-01-04 | 中兴通讯股份有限公司 | Method and apparatus for presenting communication information, device and set-top box |
| CN106943740A (en) * | 2017-04-25 | 2017-07-14 | 合肥充盈信息科技有限公司 | A kind of gesture language-voice game interactive system |
| CN111144367A (en) * | 2019-12-31 | 2020-05-12 | 重庆百事得大牛机器人有限公司 | Auxiliary semantic recognition method based on gesture recognition |
| WO2023066023A1 (en) * | 2021-10-20 | 2023-04-27 | 中兴通讯股份有限公司 | Gesture-based communication method and apparatus, storage medium, and electronic apparatus |
| CN113923471A (en) * | 2021-12-10 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Interaction method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106254960A (en) | A kind of video call method for communication disorders and system | |
| CN107728780A (en) | A kind of man-machine interaction method and device based on virtual robot | |
| CN101433072B (en) | Call centers with image or video based priority | |
| CN106550156A (en) | A kind of artificial intelligence's customer service system and its implementation based on speech recognition | |
| CN109819127B (en) | Method and system for managing crank calls | |
| CN109413286A (en) | A kind of intelligent customer service voice response system and method | |
| CN107133349A (en) | One kind dialogue robot system | |
| CN110072075A (en) | Conference management method, system and readable storage medium based on face recognition | |
| CN101888452A (en) | Multi-access customer service system and method thereof | |
| CN101923853A (en) | Speaker recognition method, equipment and system | |
| US8855280B1 (en) | Communication detail records (CDRs) containing media for communications in controlled-environment facilities | |
| CN108446395A (en) | A kind of police service information processing method and system based on big data | |
| CN107734160A (en) | A kind of language mutual aid method based on smart mobile phone | |
| CN111080926A (en) | Auxiliary interaction method and device for self-service equipment | |
| CN110321415A (en) | A kind of phone socket joint type phone robot system | |
| CN106506883A (en) | The calling-out method of call center and system | |
| CN106230985A (en) | A kind of based on the big data processing method of Internet of Things, system and service processing end | |
| CN114785842B (en) | Robot scheduling method, device, equipment and medium based on voice exchange system | |
| CN105046567A (en) | Community service system based on socialization | |
| CN109214326A (en) | A kind of information processing method, device and system | |
| CN208188895U (en) | emergency management system | |
| CN103680502A (en) | Application and realization method of internet-of-vehicle-oriented intelligent voice network | |
| CN102638778A (en) | System and method for monitoring internetwork junk short messages | |
| CN107783650A (en) | A kind of man-machine interaction method and device based on virtual robot | |
| CN110321397A (en) | Traffic guidance method and system based on GIS map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161221 |
|
| RJ01 | Rejection of invention patent application after publication |