CN104038617A - Calling method and intelligent mobile terminal - Google Patents
Calling method and intelligent mobile terminal Download PDFInfo
- Publication number
- CN104038617A CN104038617A CN201310069841.3A CN201310069841A CN104038617A CN 104038617 A CN104038617 A CN 104038617A CN 201310069841 A CN201310069841 A CN 201310069841A CN 104038617 A CN104038617 A CN 104038617A
- Authority
- CN
- China
- Prior art keywords
- sign language
- border
- mobile terminal
- language action
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Telephone Function (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the invention provides a calling method and an intelligent mobile terminal. The method is applied to the intelligent mobile terminal. The intelligent mobile terminal has a shooting function. The method comprises the following steps: hand behaviors are shot to form an initial video after a sign language calling function is started; at least one sign language image is identified from the initial video, wherein sign language actions in adjacent sign language images are different; feature points of the sign language actions are acquired in the sign language images in order, and the meanings of the sign language actions are determined according to the feature points; and speeches corresponding to the meanings are selected and sent. For people with functional disabilities, hand behaviors are shot, and the specific meanings of the hand behaviors are analyzed through image processing. Thus, the meaning that people with functional disabilities want to express can be understood according to their hand behaviors and can be transformed into speech so that the people can make a call to others.
Description
Technical field
The present invention relates to mobile terminal technology, refer to especially a kind of method of conversing and intelligent mobile terminal.
Background technology
Intelligent mobile terminal is starting useful exploration aspect miscellaneous function impedient people, for example, for example, for the deaf-mute of non-blind person colony-, note does not have the obstacle in use procedure, the function of deaf-mute's accesses network that equally also clog-free use intelligent mobile terminal provides in the least.
There are the following problems for prior art: for deaf-mute colony, still cannot converse normally by voice and the other end.
Summary of the invention
The technical problem to be solved in the present invention is to provide a kind of method of conversing and intelligent mobile terminal, solves in prior art, and deaf-mute still cannot carry out by voice and the other end defect of normal call.
For solving the problems of the technologies described above, embodiments of the invention provide a kind of method of conversing, and are applied to intelligent mobile terminal, and described intelligent mobile terminal has shoot function, method comprises: after sign language call function starts, the behavior of taking hand forms initial video; From initial video, identify at least one sign language image, the sign language action in adjacent sign language image is different; According to order, in each sign language image, gather the characteristic point of sign language action, according to described characteristic point, determine the implication of described sign language action; Select speech concurrent corresponding to described implication to send described voice.
In described method, from initial video, identify at least one sign language image, specifically comprise: with very first time interval, initial video is divided into a plurality of sections, in each section, comprise a plurality of frames, in a section, select at least one frame clearly, the border of attempting obtaining described sign language action in described frame clearly, is defined as described sign language image by the frame that has successfully obtained described border.
In described method, sign language action in adjacent sign language image is different, specifically comprise: each sign language action has a border, border has several boundary points, the distance at the edge of each boundary point sonagram picture in one's hands in first-hand sonagram picture is formed to the first distance set, the distance at the edge of each boundary point sonagram picture in one's hands in second-hand's sonagram picture is formed to second distance set, if judge, the first distance set is different from second distance set, assert that sign language action is different.
In described method, according to order, in each sign language image, gather the characteristic point of sign language action, according to described characteristic point, determine the implication of described sign language action, specifically comprise: on the border of sign language action or within border, determine several characteristic points, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
In described method, select speech concurrent corresponding to described implication to send described voice, specifically comprise: after obtaining the corresponding polygonal implication of described standard, according to described implication, obtain voice signal, described voice signal is formed to packet, to receiving terminal, send packet.
In described method, also comprise: after receiving the packet from another terminal, described in demodulation, packet obtains corresponding voice signal, according to described voice signal, obtains corresponding standard polygon, according to described standard polygon, generates corresponding sign language action; On described intelligent mobile terminal, show described sign language action.
An intelligent mobile terminal, comprising: image unit, and for starting after sign language call function, the behavior of taking hand forms initial video; Graphics processing unit, for identify at least one sign language image from initial video, the sign language action in adjacent sign language image is different; Sign language processing unit, for gather the characteristic point of sign language action at each sign language image according to order, determines the implication of described sign language action according to described characteristic point; Voice unit, for selecting speech concurrent corresponding to described implication to send described voice.
In described intelligent mobile terminal, graphics processing unit comprises: frame identification module, for very first time interval, initial video being divided into a plurality of sections, in each section, comprise a plurality of frames, and in a section, select at least one frame clearly; Frame processing module, for attempt obtaining the border of described sign language action at described frame clearly, is defined as described sign language image by the frame that has successfully obtained described border.
In described intelligent mobile terminal, graphics processing unit also comprises: sign language action recognition module, for thering is a border based on each sign language action, border has several boundary points, the distance at the edge of each boundary point sonagram picture in one's hands in first-hand sonagram picture is formed to the first distance set, the distance at the edge of each boundary point sonagram picture in one's hands in second-hand's sonagram picture is formed to second distance set, if judge, the first distance set is different from second distance set, assert that sign language action is different.
In described intelligent mobile terminal, sign language processing unit comprises: characteristic matching module, for determining several characteristic points on the border in sign language action or within border, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
The beneficial effect of technique scheme of the present invention is as follows: for example, for dysfunction personage-deaf-mute, can take the behavior of its hand, and the concrete meaning of the behavior by its hand of image processing and analyzing, so, can understand it according to the behavior of its hand and want the meaning of expressing, and the meaning that this wants to express is converted to voice and the other side converses.
Accompanying drawing explanation
Fig. 1 represents a kind of schematic flow sheet of the method for conversing;
Fig. 2 represents a kind of structural representation of intelligent mobile terminal.
Embodiment
For making the technical problem to be solved in the present invention, technical scheme and advantage clearer, be described in detail below in conjunction with the accompanying drawings and the specific embodiments.
The embodiment of the present invention provides a kind of method of conversing, and as shown in Figure 1, is applied to intelligent mobile terminal, and described intelligent mobile terminal has shoot function,
Method comprises:
Step 101, after sign language call function starts, the behavior of taking hand forms initial video;
Step 102 identifies at least one sign language image from initial video, and the sign language action in adjacent sign language image is different;
Step 103 gathers according to order the characteristic point that sign language is moved in each sign language image, determines the implication of described sign language action according to described characteristic point;
Step 104, selects speech concurrent corresponding to described implication to send described voice.
The technology that application provides, for example, for dysfunction personage-deaf-mute, can take the behavior of its hand, and the concrete meaning of the behavior by its hand of image processing and analyzing, so, can understand it according to the behavior of its hand and want the meaning of expressing, and the meaning that this wants to express is converted to voice and the other side converses.
Intelligent mobile terminal includes camera, thereby start after sign language call function, if it is to serve as dysfunction personage that intelligent mobile terminal is judged current call function, adopt camera to take and form initial video the behavior of hand, in fact, initial video is comprised of several frames.
In a preferred embodiment, from initial video, identify at least one sign language image, specifically comprise:
With very first time interval, initial video is divided into several sections, in each section, comprises several frames,
In a section, select at least one frame clearly,
The border that obtains sign language action in described frame clearly, the region that this border surrounds is described sign language image.
Very first time interval can be an empirical value, to determine according to dysfunction personage's behavioural habits, for example, if dysfunction personage represents a hand motion an average second, very first time interval is 1 second, if average half second represents a hand motion, very first time interval is 0.5 second.
Wherein, frame refers to clearly, in this two field picture, between the formed sign language action of the behavior of hand and the background of two field picture, has obvious difference.
In a preferred embodiment, the sign language action in adjacent sign language image is different, specifically comprises:
Each sign language action has a border, and border has several boundary points,
In first-hand sonagram picture, the distance at the edge of each boundary point sonagram picture in one's hands forms the first distance set,
In second-hand's sonagram picture, the distance at the edge of each boundary point sonagram picture in one's hands forms second distance set,
If judge, the first distance set is different from second distance set, assert that sign language action is different.
In a preferred embodiment, according to order, in each sign language image, gather the characteristic point of sign language action, according to described characteristic point, determine the implication of described sign language action, specifically comprise:
On the border of sign language image or within border, determine several characteristic points, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
Described characteristic point should comprise the characteristic point at palm or the back of the hand position, the characteristic point at thumb position, the characteristic point of other positions, four fingers, the characteristic point of volar edge, the characteristic point of palm and wrist decomposition place.
In a preferred embodiment, select speech concurrent corresponding to described implication to send, specifically comprise:
After obtaining the corresponding polygonal implication of described standard, according to described implication, obtain voice signal, described voice signal is formed to packet, to receiving terminal, send packet.
In a preferred embodiment, after receiving the packet from receiving terminal, packet obtains corresponding voice signal described in demodulation,
According to described voice signal, obtain corresponding standard polygon,
According to described standard polygon, generate corresponding sign language action;
Show described sign language action.
Deaf-mute's camera by intelligent mobile terminal in the process of making a phone call photographs sign language and moves, again sign language action is converted into voice, the sign language that deaf-mute is used changes into voice output, so just can give the normal person who is positioned at receiving terminal by transfer voice, in like manner, normal person's voice arrive after intelligent mobile terminal, be converted into sign language and move and be shown to deaf-mute and see on intelligent mobile terminal.
The embodiment of the present invention provides a kind of intelligent mobile terminal, as shown in Figure 2, comprising:
Image unit 201, for starting after sign language call function, the behavior of taking hand forms initial video;
Graphics processing unit 202, for identify at least one sign language image from initial video, the sign language action in adjacent sign language image is different;
Sign language processing unit 203, for gather the characteristic point of sign language action at each sign language image according to order, determines the implication of described sign language action according to described characteristic point;
Voice unit 204, for selecting speech concurrent corresponding to described implication to send described voice.
In a preferred embodiment, graphics processing unit 202 comprises:
Frame identification module, for very first time interval, initial video being divided into a plurality of sections, comprises a plurality of frames in each section, select at least one frame clearly in a section;
Frame processing module, for attempt obtaining the border of described sign language action at described frame clearly, is defined as described sign language image by the frame that has successfully obtained described border.
In a preferred embodiment, graphics processing unit 202 also comprises:
Sign language action recognition module, for having a border based on each sign language action, border has several boundary points,
The distance at the edge of each boundary point sonagram picture in one's hands in first-hand sonagram picture is formed to the first distance set,
The distance at the edge of each boundary point sonagram picture in one's hands in second-hand's sonagram picture is formed to second distance set,
If judge, the first distance set is different from second distance set, assert that sign language action is different.
In a preferred embodiment, sign language processing unit 203 comprises:
Characteristic matching module, for determining several characteristic points on the border in sign language action or within border, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
Intelligent mobile terminal not only can send voice signal to receiving terminal by miscellaneous function impedient people, and can be used as receiving terminal, after the voice signal of reception from normal person, be converted into sign language action, this process comprises: after receiving the packet from receiving terminal, described in demodulation, packet obtains corresponding voice signal
According to described voice signal, obtain corresponding standard polygon,
According to described standard polygon, generate corresponding sign language action;
On described intelligent mobile terminal, show described sign language action.
Adopt the advantage after this programme to be: deaf-mute's camera function by intelligent mobile terminal in the process of making a phone call photographs sign language and moves, sign language action is converted into voice output, so just can give the normal person who is positioned at receiving terminal by transfer voice, in like manner, normal person's voice arrive after intelligent mobile terminal, be converted into sign language and move and be shown to deaf-mute and see on intelligent mobile terminal.
The above is the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, do not departing under the prerequisite of principle of the present invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (10)
1. a method of conversing, is characterized in that, is applied to intelligent mobile terminal, and described intelligent mobile terminal has shoot function,
Method comprises:
After sign language call function starts, the behavior of taking hand forms initial video;
From initial video, identify at least one sign language image, the sign language action in adjacent sign language image is different;
According to order, in each sign language image, gather the characteristic point of sign language action, according to described characteristic point, determine the implication of described sign language action;
Select speech concurrent corresponding to described implication to send described voice.
2. method according to claim 1, is characterized in that, identifies at least one sign language image from initial video, specifically comprises:
With very first time interval, initial video is divided into a plurality of sections, in each section, comprises a plurality of frames,
In a section, select at least one frame clearly,
The border of attempting obtaining described sign language action in described frame clearly, is defined as described sign language image by the frame that has successfully obtained described border.
3. method according to claim 1, is characterized in that, the sign language action in adjacent sign language image is different, specifically comprises:
Each sign language action has a border, and border has several boundary points,
The distance at the edge of each boundary point sonagram picture in one's hands in first-hand sonagram picture is formed to the first distance set,
The distance at the edge of each boundary point sonagram picture in one's hands in second-hand's sonagram picture is formed to second distance set,
If judge, the first distance set is different from second distance set, assert that sign language action is different.
4. method according to claim 1, is characterized in that, gathers the characteristic point of sign language action according to order in each sign language image, determines the implication of described sign language action according to described characteristic point, specifically comprises:
On the border of sign language action or within border, determine several characteristic points, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
5. method according to claim 4, is characterized in that, selects speech concurrent corresponding to described implication to send described voice, specifically comprises:
After obtaining the corresponding polygonal implication of described standard, according to described implication, obtain voice signal, described voice signal is formed to packet, to receiving terminal, send packet.
6. method according to claim 1, is characterized in that, also comprises:
After receiving the packet from another terminal, packet obtains corresponding voice signal described in demodulation,
According to described voice signal, obtain corresponding standard polygon,
According to described standard polygon, generate corresponding sign language action;
On described intelligent mobile terminal, show described sign language action.
7. an intelligent mobile terminal, is characterized in that, comprising:
Image unit, for starting after sign language call function, the behavior of taking hand forms initial video;
Graphics processing unit, for identify at least one sign language image from initial video, the sign language action in adjacent sign language image is different;
Sign language processing unit, for gather the characteristic point of sign language action at each sign language image according to order, determines the implication of described sign language action according to described characteristic point;
Voice unit, for selecting speech concurrent corresponding to described implication to send described voice.
8. intelligent mobile terminal according to claim 7, is characterized in that, graphics processing unit comprises:
Frame identification module, for very first time interval, initial video being divided into a plurality of sections, comprises a plurality of frames in each section, select at least one frame clearly in a section;
Frame processing module, for attempt obtaining the border of described sign language action at described frame clearly, is defined as described sign language image by the frame that has successfully obtained described border.
9. intelligent mobile terminal according to claim 7, is characterized in that, graphics processing unit also comprises:
Sign language action recognition module, for having a border based on each sign language action, border has several boundary points,
The distance at the edge of each boundary point sonagram picture in one's hands in first-hand sonagram picture is formed to the first distance set,
The distance at the edge of each boundary point sonagram picture in one's hands in second-hand's sonagram picture is formed to second distance set,
If judge, the first distance set is different from second distance set, assert that sign language action is different.
10. intelligent mobile terminal according to claim 7, is characterized in that, sign language processing unit comprises:
Characteristic matching module, for determining several characteristic points on the border in sign language action or within border, connect each characteristic point and form polygon to be measured, described polygon to be measured is mated with the standard polygon of depositing in advance, after the match is successful, obtain the corresponding polygonal implication of described standard.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310069841.3A CN104038617B (en) | 2013-03-04 | 2013-03-04 | A kind of method carrying out conversing and intelligent mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310069841.3A CN104038617B (en) | 2013-03-04 | 2013-03-04 | A kind of method carrying out conversing and intelligent mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104038617A true CN104038617A (en) | 2014-09-10 |
| CN104038617B CN104038617B (en) | 2016-08-03 |
Family
ID=51469202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310069841.3A Expired - Fee Related CN104038617B (en) | 2013-03-04 | 2013-03-04 | A kind of method carrying out conversing and intelligent mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104038617B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110874554A (en) * | 2018-09-03 | 2020-03-10 | 阿里巴巴集团控股有限公司 | Action recognition method, terminal device, server, system and storage medium |
| CN113923471A (en) * | 2021-12-10 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Interaction method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101452705A (en) * | 2007-12-07 | 2009-06-10 | 希姆通信息技术（上海）有限公司 | Voice character conversion nd cued speech character conversion method and device |
| CN101594434A (en) * | 2009-06-16 | 2009-12-02 | 中兴通讯股份有限公司 | The sign language processing method and the sign language processing mobile terminal of portable terminal |
| CN101605399A (en) * | 2008-06-13 | 2009-12-16 | 英华达(上海)电子有限公司 | A kind of portable terminal and method that realizes Sign Language Recognition |
| CN101605158A (en) * | 2008-06-13 | 2009-12-16 | 鸿富锦精密工业（深圳）有限公司 | Mobile phone dedicated for deaf-mutes |
| CN101661330A (en) * | 2009-09-22 | 2010-03-03 | 深圳华为通信技术有限公司 | Method for converting sign language and terminal thereof |
| CN101661374A (en) * | 2009-09-22 | 2010-03-03 | 深圳华为通信技术有限公司 | Method for outputting sign language and terminal thereof |
-
2013
- 2013-03-04 CN CN201310069841.3A patent/CN104038617B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101452705A (en) * | 2007-12-07 | 2009-06-10 | 希姆通信息技术（上海）有限公司 | Voice character conversion nd cued speech character conversion method and device |
| CN101605399A (en) * | 2008-06-13 | 2009-12-16 | 英华达(上海)电子有限公司 | A kind of portable terminal and method that realizes Sign Language Recognition |
| CN101605158A (en) * | 2008-06-13 | 2009-12-16 | 鸿富锦精密工业（深圳）有限公司 | Mobile phone dedicated for deaf-mutes |
| CN101594434A (en) * | 2009-06-16 | 2009-12-02 | 中兴通讯股份有限公司 | The sign language processing method and the sign language processing mobile terminal of portable terminal |
| CN101661330A (en) * | 2009-09-22 | 2010-03-03 | 深圳华为通信技术有限公司 | Method for converting sign language and terminal thereof |
| CN101661374A (en) * | 2009-09-22 | 2010-03-03 | 深圳华为通信技术有限公司 | Method for outputting sign language and terminal thereof |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110874554A (en) * | 2018-09-03 | 2020-03-10 | 阿里巴巴集团控股有限公司 | Action recognition method, terminal device, server, system and storage medium |
| CN110874554B (en) * | 2018-09-03 | 2023-05-26 | 阿里巴巴集团控股有限公司 | Action recognition method, terminal device, server, system and storage medium |
| CN113923471A (en) * | 2021-12-10 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Interaction method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104038617B (en) | 2016-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110865705B (en) | Multi-mode fusion communication method and device, head-mounted equipment and storage medium | |
| CN108198569B (en) | Audio processing method, device and equipment and readable storage medium | |
| CN106454481B (en) | A kind of method and device of live broadcast of mobile terminal interaction | |
| US11445145B2 (en) | Method and device for controlling camera shooting, smart device and computer storage medium | |
| CN107340865A (en) | Multi-modal virtual robot exchange method and system | |
| CN106127828A (en) | The processing method of a kind of augmented reality, device and mobile terminal | |
| CN108198130B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US20170161553A1 (en) | Method and electronic device for capturing photo | |
| EP2526507A1 (en) | Meeting room participant recogniser | |
| CN105447164A (en) | Method and apparatus for automatically pushing chat expressions | |
| CN104506946B (en) | A kind of TV programme recognition methods and system based on image recognition | |
| CN103886632A (en) | Method for generating user expression head portrait and communication terminal | |
| CN110910887B (en) | Voice wake-up method and device | |
| CN108898592A (en) | Prompt method and device, the electronic equipment of camera lens degree of fouling | |
| CN103108124B (en) | Image acquiring method, device and mobile terminal | |
| CN106648760A (en) | Terminal and method thereof for cleaning background application programs based on face recognition | |
| CN110536095A (en) | Call method, device, terminal and storage medium | |
| CN106157262B (en) | Augmented reality processing method and device and mobile terminal | |
| CN105898137A (en) | Image collection and information push methods, image collection and information push devices and mobile phone | |
| JP2010239499A (en) | Communication terminal unit, communication control unit, method of controlling communication of communication terminal unit, and communication control program | |
| CN110956059A (en) | Dynamic gesture recognition method and device and electronic equipment | |
| CN108399009A (en) | The method and device of smart machine is waken up using human-computer interaction gesture | |
| CN110111795B (en) | Voice processing method and terminal equipment | |
| CN110139021A (en) | Auxiliary shooting method and terminal device | |
| CN104038617A (en) | Calling method and intelligent mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160803 Termination date: 20210304 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |