CN109409276A - A kind of stalwartness sign language feature extracting method - Google Patents
A kind of stalwartness sign language feature extracting method Download PDFInfo
- Publication number
- CN109409276A CN109409276A CN201811218298.8A CN201811218298A CN109409276A CN 109409276 A CN109409276 A CN 109409276A CN 201811218298 A CN201811218298 A CN 201811218298A CN 109409276 A CN109409276 A CN 109409276A
- Authority
- CN
- China
- Prior art keywords
- sign language
- network
- feature
- network based
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of healthy and strong sign language feature extracting method, and sign language feature healthy and strong and with significant separating capacity can be extracted from the sign language of multiple scenes movement, belongs to medical treatment & health and information technology field.The sign language feature extracting method fights the network architecture using depth, including three sub-networks: the feature extraction network based on depth convolutional coding structure, the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure.By minimizing sign language action classification evaluated error simultaneously and maximizing scene information evaluated error, this method guarantees that ga s safety degree of the extracted sign language feature between different sign languages movement and the consistency under different scenes, so that the sign Language Recognition using this method can across scene work.The present invention will promote performance of the sign Language Recognition in practical more scene work, exchange the condition of offer with the effective of ordinary people for dysaudia personage.
Description
Technical field
The invention belongs to medical treatment & healths and information technology field, are related to a kind of healthy and strong sign language feature extracting method, Ke Yicong
Sign language feature healthy and strong and with significant separating capacity is extracted in the sign language movement of multiple scenes.This method is by depth pair
Anti- network extracts sign language feature, it is ensured that ga s safety degree of this feature between different sign languages movement and under the different scenes
Consistency so that the sign Language Recognition using this method can across scene work.
Background technique
Sign language is the language of the daily exchange of dysaudia personage, for promoting the exchange between dysaudia personage to play weight
It acts on.However, sign language movement is complex and is difficult to remember, sign language is caused to be difficult to be grasped.Sign Language Recognition can be automatic
Sign language movement is identified, to effectively promote exchanging for dysaudia personage and ordinary people.
Currently, researcher devises a variety of methods to realize Sign Language Recognition, and has carried out beneficial exploration:
(bibliography: Huang Aifa, Xu Xiangmin, Xing Xiaofen, Li Zhaohai, Ni Haomiao one kind are based on Leap to Huang Aifa et al.
Manual alphabet recognition methods [P] Chinese invention patent of Motion, the number of applying for a patent: CN201510254098.8,2015.) it mentions
Sign Language Recognition is realized using body-sensing sensor out, sign language information is acquired by somatosensory device, extracts sign language feature, and then be based on
Template matching method realizes Sign Language Recognition.
Hu Zhangfang et al. (bibliography: static state of Hu Zhangfang, Luo Yuan, Zhang Yi, Yang Lin, the Xi Bing based on Kinect sensor
Manual alphabet identifying system and method [P] Chinese invention patent, the number of applying for a patent: CN201410191394.3,2014.) it proposes
Human body sign language action video is captured by Kinect camera, realizes that sign language is dynamic based on image procossing and mode identification method later
It identifies.
The above method mostly uses mode identification method to design sign Language Recognition, and the core of pattern-recognition is from acquisition
The feature with separating capacity is extracted in signal.However, often failing in across scene domain pattern recognition methods.Such as:
The sign Language Recognition that office's training study finishes, discrimination can significantly reduce when using at home.This be substantially due to
Caused by not accounting for across scene ability when sign language feature extraction.
In view of this, the present invention fights network by depth to extract sign language feature, it is ensured that this feature is dynamic in different sign languages
Ga s safety degree between work and the consistency under different scenes, so that the sign Language Recognition using this method can be with
Across scene work.The present invention will promote the performance of sign Language Recognition in actual operation, to effectively promote dysaudia
Personage exchanges with ordinary people's.
Summary of the invention
The purpose of the present invention is overcoming the deficiencies of existing technologies, providing one kind can mention from the sign language of multiple scenes movement
Take out healthy and strong and the sign language feature with significant separating capacity method.Compared with prior art, method of the invention will make
The trained sign Language Recognition under a scene is obtained, good sign language still can be obtained when working under new scene and is known
Other performance.
Technical solution of the present invention:
A kind of stalwartness sign language feature extracting method, this method mainly by based on depth convolutional coding structure feature extraction network,
Scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure, which cooperate, to be completed；Its job step
It suddenly include the calculating of offline network parameter and two stages of online sign language feature extraction, offline network parameter calculation phase is by known
Sign language act true value, realized by minimizing cost function and the solution of all-network parameter calculated, online sign language feature
The information that the extraction stage acquires according to sensor currently entered extracts healthy and strong sign language feature, specific as follows:
1) offline network parameter calculation phase
(1.1) human body executes a certain known sign language movement under a certain known scene, and Sign Language Recognition sensor is by acquisition
Sign language action message is input in the feature extraction network based on depth convolutional coding structure, the sign language that the output of feature extraction network is extracted
Feature is simultaneously transmitted to the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure；
(1.2) depth is carried out to the sign language feature that feature extraction network extracts based on the scenery identification network of full connection structure
Analysis, identifies and outputting sign language acts corresponding scene information；
(1.3) sorter network based on rarefaction representation structure carries out identification point to the sign language feature that feature extraction network extracts
Class exports the sign language action classification identified；
(1.4) according to the sign language action scene information and sign language action classification that identify and known true sign language
Action scene information and true sign language action classification calculate cost function；
(1.5) it is based on error backpropagation algorithm, is realized by minimizing cost function to based on depth convolutional coding structure
The net of feature extraction network, the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure
The solution of network parameter calculates；
(1.6) step (1.1) to (1.5) are repeated, until all-network parameter remains unchanged, offline network parameter calculates rank
Section finishes.
2) online sign language feature extraction phases, human body execute a certain unknown sign language movement, sign language under a certain unknown scene
For identification sensor by the information input of acquisition to the feature extraction network based on depth convolutional coding structure, the sign language for exporting extraction is special
Sign.
The sensing data is the frequency time two of the signal amplitude acquired by wireless receiver, phase information composition
Tie up matrix；
The feature extraction network based on depth convolutional coding structure includes 3-5 layers, and every layer is performed both by convolution, Chi Hua, non-thread
Property three kinds of activation primitive operation；
The scenery identification network based on full connection structure includes 3 layers, and every layer uses full connection structure with next layer,
And execute nonlinear activation operation；
The sorter network based on rarefaction representation structure includes input layer and output layer, and input layer and output layer are using complete
Connection structure, and each output unit numberical range is limited between 0 to 1, meanwhile, sparse constraint is increased to output layer, really
An only output unit is protected to be active；
The cost function is equal to sign language action classification evaluated error and subtracts scene information evaluated error, by using Adam
Algorithm carries out minimum operation to cost function, solves by error backpropagation algorithm and calculates whole network parameter.
Beneficial effects of the present invention: can provide a kind of healthy and strong sign language feature extracting method, and this method fights net by depth
Network extracts sign language feature, it is ensured that ga s safety degree of this feature between different sign languages movement and one under the different scenes
Cause property, so that the sign Language Recognition using this method can across scene work.The present invention will promote sign Language Recognition
Performance in practical more scene work, exchanges the condition of offer with the effective of ordinary people for dysaudia personage.
Detailed description of the invention
Fig. 1 is the system structure functional block diagram of the method for the present invention.
Specific embodiment
Specific implementation of the invention is specifically elaborated below with reference to technical solution and attached drawing.
Embodiment uses system structure shown in FIG. 1.System constitutes as follows: the feature extraction net based on depth convolutional coding structure
Network is formed by 3 layers, and every layer is performed both by 3 × 3 convolution operation, the operation of 2 × 2 pondization and the activation primitive based on RELU；Base
It is formed in the scenery identification network of full connection structure by 3 layers, every layer is used full connection structure with next layer, and swashed using RELU
Function living carries out nonlinear operation；Sorter network based on rarefaction representation structure is formed by 2 layers, and input layer and output layer are using complete
Connection structure, and each output unit numberical range is limited between 0 to 1, meanwhile, sparse constraint is increased to output layer, really
An only output unit is protected to be active.Sign Language Recognition otherwise, is shared using radio identification under 5 kinds of scenes
30 kinds of sign languages movement, sensing data be signal amplitude acquire by wireless receiver, phase information composition 60 × 200
Frequency time two-dimensional matrix, the feature extraction network inputs frequency time two-dimensional matrix based on depth convolutional coding structure, output 64 × 1
Sign language feature；The sign language feature vector is input to the scenery identification network based on full connection structure, and the sign language of output 5 × 1 is dynamic
Make scene vector；The sign language feature vector is input to the sorter network based on rarefaction representation structure, the sign language movement of output 30 × 1
Categorization vector.Offline network parameter calculation phase is solved using Adam algorithmic minimizing cost function and calculates network parameter；Online
Sign language feature extraction phases, the frequency directly obtained using the feature extraction network based on depth convolutional coding structure from current sign language movement
Sign language feature is extracted in rate time 2-D matrix.
Test shows in applying across scene, after healthy and strong sign language feature extracting method of the invention, Sign Language Recognition system
The accuracy rate of system can be obviously improved.
Claims (1)
1. a kind of stalwartness sign language feature extracting method, which is characterized in that this method is mainly by the feature based on depth convolutional coding structure
Network, the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure is extracted to have cooperated
At；Its work step includes that offline network parameter calculates and two stages of online sign language feature extraction, the calculating of offline network parameter
Stage acts true value by known sign language, realizes that the solution to all-network parameter calculates by minimizing cost function,
The information that online sign language feature extraction phases are acquired according to sensor currently entered, extracts healthy and strong sign language feature, specifically such as
Under:
1) offline network parameter calculation phase
(1.1) human body executes a certain known sign language movement under a certain known scene, and Sign Language Recognition sensor is by the sign language of acquisition
Action message is input in the feature extraction network based on depth convolutional coding structure, the sign language feature that the output of feature extraction network is extracted
And it is transmitted to the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure；
(1.2) depth point is carried out to the sign language feature that feature extraction network extracts based on the scenery identification network of full connection structure
Analysis, identifies and outputting sign language acts corresponding scene information；
(1.3) sorter network based on rarefaction representation structure carries out identification classification to the sign language feature that feature extraction network extracts,
Export the sign language action classification identified；
(1.4) according to the sign language action scene information and sign language action classification and known true sign language movement identified
Scene information and true sign language action classification calculate cost function；
(1.5) it is based on error backpropagation algorithm, is realized by minimizing cost function to the feature based on depth convolutional coding structure
Extract the network ginseng of network, the scenery identification network based on full connection structure and the sorter network based on rarefaction representation structure
Several solutions calculates；
(1.6) step (1.1) to (1.5) are repeated, until all-network parameter remains unchanged, offline network parameter calculation phase is complete
Finish；
2) online sign language feature extraction phases, human body execute a certain unknown sign language movement, Sign Language Recognition under a certain unknown scene
The information input of acquisition to the feature extraction network based on depth convolutional coding structure, is exported the sign language feature of extraction by sensor；
The sensing data is the frequency time Two-Dimensional Moment of the signal amplitude acquired by wireless receiver, phase information composition
Battle array；
The feature extraction network based on depth convolutional coding structure includes 3-5 layers, and every layer is performed both by convolution, Chi Hua, non-linear swashs
Three kinds of operations of function living；
The scenery identification network based on full connection structure includes 3 layers, and every layer uses full connection structure with next layer, and holds
The operation of row nonlinear activation；
The sorter network based on rarefaction representation structure includes input layer and output layer, and input layer is used with output layer and connect entirely
Structure, and each output unit numberical range is limited between 0 to 1, meanwhile, sparse constraint is increased to output layer, it is ensured that only
There is an output unit to be active；
The cost function is equal to sign language action classification evaluated error and subtracts scene information evaluated error, by using Adam algorithm
Minimum operation is carried out to cost function, is solved by error backpropagation algorithm and calculates whole network parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811218298.8A CN109409276A (en) | 2018-10-19 | 2018-10-19 | A kind of stalwartness sign language feature extracting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811218298.8A CN109409276A (en) | 2018-10-19 | 2018-10-19 | A kind of stalwartness sign language feature extracting method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109409276A true CN109409276A (en) | 2019-03-01 |
Family
ID=65467747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811218298.8A Withdrawn CN109409276A (en) | 2018-10-19 | 2018-10-19 | A kind of stalwartness sign language feature extracting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109409276A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175551A (en) * | 2019-05-21 | 2019-08-27 | 青岛科技大学 | A kind of sign Language Recognition Method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1204826A (en) * | 1998-04-18 | 1999-01-13 | 茹家佑 | Method for deaf-dumb voice learning dialogue and pronunciation synchronous feedback device |
| CN105205475A (en) * | 2015-10-20 | 2015-12-30 | 北京工业大学 | Dynamic gesture recognition method |
| CN105989336A (en) * | 2015-02-13 | 2016-10-05 | 中国科学院西安光学精密机械研究所 | Scene identification method based on deconvolution deep network learning with weight |
| CN106685546A (en) * | 2016-12-29 | 2017-05-17 | 深圳天珑无线科技有限公司 | Wireless human body sensing method and server |
| US20170220923A1 (en) * | 2016-02-02 | 2017-08-03 | Samsung Electronics Co., Ltd. | Gesture classification apparatus and method using emg signal |
| CN107679491A (en) * | 2017-09-29 | 2018-02-09 | 华中师范大学 | A kind of 3D convolutional neural networks sign Language Recognition Methods for merging multi-modal data |
| CN107871136A (en) * | 2017-03-22 | 2018-04-03 | 中山大学 | The image-recognizing method of convolutional neural networks based on openness random pool |
| CN107886064A (en) * | 2017-11-06 | 2018-04-06 | 安徽大学 | A kind of method that recognition of face scene based on convolutional neural networks adapts to |
| CN108491077A (en) * | 2018-03-19 | 2018-09-04 | 浙江大学 | A kind of surface electromyogram signal gesture identification method for convolutional neural networks of being divided and ruled based on multithread |
-
2018
- 2018-10-19 CN CN201811218298.8A patent/CN109409276A/en not_active Withdrawn
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1204826A (en) * | 1998-04-18 | 1999-01-13 | 茹家佑 | Method for deaf-dumb voice learning dialogue and pronunciation synchronous feedback device |
| CN105989336A (en) * | 2015-02-13 | 2016-10-05 | 中国科学院西安光学精密机械研究所 | Scene identification method based on deconvolution deep network learning with weight |
| CN105205475A (en) * | 2015-10-20 | 2015-12-30 | 北京工业大学 | Dynamic gesture recognition method |
| US20170220923A1 (en) * | 2016-02-02 | 2017-08-03 | Samsung Electronics Co., Ltd. | Gesture classification apparatus and method using emg signal |
| CN106685546A (en) * | 2016-12-29 | 2017-05-17 | 深圳天珑无线科技有限公司 | Wireless human body sensing method and server |
| CN107871136A (en) * | 2017-03-22 | 2018-04-03 | 中山大学 | The image-recognizing method of convolutional neural networks based on openness random pool |
| CN107679491A (en) * | 2017-09-29 | 2018-02-09 | 华中师范大学 | A kind of 3D convolutional neural networks sign Language Recognition Methods for merging multi-modal data |
| CN107886064A (en) * | 2017-11-06 | 2018-04-06 | 安徽大学 | A kind of method that recognition of face scene based on convolutional neural networks adapts to |
| CN108491077A (en) * | 2018-03-19 | 2018-09-04 | 浙江大学 | A kind of surface electromyogram signal gesture identification method for convolutional neural networks of being divided and ruled based on multithread |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175551A (en) * | 2019-05-21 | 2019-08-27 | 青岛科技大学 | A kind of sign Language Recognition Method |
| CN110175551B (en) * | 2019-05-21 | 2023-01-10 | 青岛科技大学 | Sign language recognition method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292813B (en) | A kind of multi-pose Face generation method based on generation confrontation network | |
| CN110531861B (en) | Method and device for processing motor imagery electroencephalogram signal and storage medium | |
| Ale et al. | Deep learning based plant disease detection for smart agriculture | |
| CN103942577B (en) | Based on the personal identification method for establishing sample database and composite character certainly in video monitoring | |
| Li et al. | Intelligent sports training system based on artificial intelligence and big data | |
| CN108433722A (en) | Portable brain electric collecting device and its application in SSVEP and Mental imagery | |
| Wang et al. | Csi-net: Unified human body characterization and pose recognition | |
| CN109171707A (en) | A kind of intelligent cardiac figure classification method | |
| CN110353702A (en) | A kind of emotion identification method and system based on shallow-layer convolutional neural networks | |
| CN104771163A (en) | Electroencephalogram feature extraction method based on CSP and R-CSP algorithms | |
| CN106846380B (en) | A kind of medical image registration method and equipment | |
| CN107122050B (en) | Stable state of motion visual evoked potential brain-computer interface method based on CSFL-GDBN | |
| CN110059564B (en) | Feature extraction method based on power spectral density and cross-correlation entropy spectral density fusion | |
| CN111582200B (en) | Human body posture estimation method, device, electronic equipment and medium | |
| Zhang et al. | Gait-based age estimation with deep convolutional neural network | |
| CN103327359B (en) | Video significance region searching method applied to video quality evaluation | |
| CN110074779A (en) | A kind of EEG signal identification method and device | |
| CN107480716A (en) | A kind of combination EOG and video pan signal recognition method and system | |
| CN108596269A (en) | A kind of recognizer of the plantar pressure image based on SVM+CNN | |
| CN110298295A (en) | Mobile terminal on-line study measure of supervision based on recognition of face | |
| Chen et al. | Feature map pooling for cross-view gait recognition based on silhouette sequence images | |
| CN110188610A (en) | A kind of emotional intensity estimation method and system based on deep learning | |
| Kwon et al. | Hands-free human activity recognition using millimeter-wave sensors | |
| Zheng et al. | An efficient mobile model for insect image classification in the field pest management | |
| Tang et al. | Research on 3D human pose estimation using RGBD camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190301 |
|
| WW01 | Invention patent application withdrawn after publication |