CN107122042A - The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined - Google Patents
The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined Download PDFInfo
- Publication number
- CN107122042A CN107122042A CN201710198920.2A CN201710198920A CN107122042A CN 107122042 A CN107122042 A CN 107122042A CN 201710198920 A CN201710198920 A CN 201710198920A CN 107122042 A CN107122042 A CN 107122042A
- Authority
- CN
- China
- Prior art keywords
- chinese
- writing
- finger tip
- character
- dynamic gesture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/32—Digital ink
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Document Processing Apparatus (AREA)
Abstract
The invention discloses the Chinese-character writing method and system that a kind of quiet dynamic gesture is combined, this method is mainly shown using Kinect device, with reference to quiet dynamic gesture, at Unity interfaces, to simulate Chinese-character writing.Aim at " letter>Phonetic>The whole writing process of Chinese character ".The present invention focuses on alphabetical completion dynamic gesture writing, multiple writing letter composition phonetics, the expected Chinese character of static gesture selection.Process is easy-to-understand, is easy to be realized and studied in laboratory.Application prospect for quiet dynamic gesture backward specifies direction, has paved road.
Description
Technical field
The present invention relates to a kind of Chinese-character writing method and system, more particularly to one kind are real based on Unity interfaces on Kinect
The naked hand Chinese-character writing method and system that existing quiet dynamic gesture is combined.
Background technology
Virtual reality (Virtual RealityVR) is using computer technology as core, with reference to related science technology, generation
The digitized environment highly approximate in terms of vision, hearing, touch sense with certain limit true environment, user by it is necessary equipment with
Object in digitized environment interacts, influenced each other, and can produce the impression for coming to correspondence true environment personally and experience
(Zhao Qinping《(F volumes of Chinese science:Information science)》)
Human-computer interaction technology include machine by output or display device give people to provide it is a large amount of for information about and prompting is asked for instructions
Inputted for information about to machine by input equipment Deng, people, answer a question and point out to ask for instructions.As revolutionary product,
Kinect somatosensory device can obtain coloured image, depth image and skeleton image, and there is provided a kind of brand-new people
The mode of machine interaction.It can catch, track action, gesture and the sound of human body.(Wang Songlin《Gesture based on Kinect is known
Not with Robot Control Technology research》)
Gesture identification is a very challenging research topic.There is the spies such as diversity, ambiguity in natural gesture
Point, it is many that complicated dynamic gesture must account for change, the movement locus of hand, the resemblance of hand in gesture time sequence etc.
Individual aspect, this causes Gesture Recognition to face many key issues.And under Kinect effects the gesture identification that carries out with
There is the phenomenon that precision is not high, stability is not high and shakes in track following, cause unstable result, do not reach real-time man-machine friendship
Mutual effect.
Gesture Recognition based on data glove is the data transmitted according to data glove to parse corresponding gesture.Together
The Gesture Recognition of view-based access control model information is compared, and the gesture identification method based on data glove has little interference by environment, is known
Rate is not high, the advantage such as real-time.With the development of man-machine interaction mode diversification, the Gesture Recognition based on data glove
Attract attention and study further.(Zang Kun《Data glove Research of Gesture Recognition》)
The content of the invention
In view of the above-mentioned deficiencies in the prior art, it is an object of the present invention to provide it is a kind of based on Kinect Unity interfaces it is quiet
Dynamic gesture combine Chinese-character writing method and system, this method mainly using Kinect device, with reference to quiet dynamic gesture,
Unity interfaces are shown, naked hand simulates Chinese-character writing.Aim at " letter->Phonetic->The whole of Chinese character " was write
Journey.
The technical scheme that the present invention solves above-mentioned technical problem use is as follows：
A kind of Chinese-character writing method that quiet dynamic gesture is combined, this method comprises the following steps：
Face datection is carried out using Kinect, operator is determined；
Operator's both hands profile and respectively finger tip are recognized, and determines to write finger tip；
Real-time tracking writing finger tip simultaneously carries out track drafting, detects when completing order, will leave handwriting trace
Picture carries out OCR identifications；Corresponding letters are obtained, when the letter of acquisition is not expected letter, the letter is deleted in triggering, again
Draw；The step is repeated until completing whole phonetic；
The corresponding candidate Chinese character of the phonetic is pushed according to whole phonetic, Chinese character needed for being selected using static gesture.
A kind of Kanji writing systems that quiet dynamic gesture is combined, the system includes realizing client and the realization of writing letter
The server at Unity chinese displies interface, client is based on logical by Socket between Kinect device, and client and server
Letter is connected；The Face datection to operator and the detection to both hands profile and finger tip are completed in client；And by real time with
The movement of track finger tip carries out the drafting of track；Detect after order of completing, OCR identifications are carried out to track picture, obtain
Corresponding letter；Until completing phonetic, client sends whole phonetic to server, and server shows the pinyin candidate Chinese
Word, expected Chinese character is selected by static gesture.
Generally, tracking finger tip it is mobile when the phenomenon of " shake " occurs, the noise being primarily due in system and disturb
Influence.Tracking finger tip process carries out smooth operation using Kalman filter in the present invention.
When carrying out track drafting, breakpoint is set：Fist, which will be held, causes system detectio less than writing finger tip, to form breakpoint,
Hand is moved to next stroke starting point by holding state of clenching fist from upper stroke terminal, is stretched out writing finger tip and is continued to write.Using this
Method can effectively improve the normalization and identifiability of track.
In above-mentioned technical proposal, the specific method of detection both hands profile and finger tip may be referred to《Three-dimensional based on Kinect
Multifinger tracking algorithm and application》(Yan Hao), using the finger tip detection algorithm based on pixel classifications, but is necessarily improved：Root
The characteristics of according to Kinect data, four shape facilities are defined, the classification based on pixel is then carried out by GPU opponent region,
After obtaining classification results, directly fingertip area do not cluster obtain fingertip location, but by fingertip area and finger area
Domain combines, and carries out ellipse fitting to these regions, the algorithm not only increases the speed of finger tip detection, and greatly carries
The high accuracy of identification.
The present invention focuses on completion dynamic gesture and writes alphabetical, multiple writing letter composition phonetics, static gesture selection expection
Chinese character.Process is easy-to-understand, is easy to be realized and studied in laboratory.Application prospect for quiet dynamic gesture backward is indicated
Direction, road is paved.
Brief description of the drawings
Fig. 1 is system operational flow diagram；
Fig. 2 is Socket correspondence figures；
Fig. 3 writes the Operation interface diagram of letter to realize；
Fig. 4 is that finger tip motion track draws exemplary plot；
Fig. 5 is uppercase regular writing exemplary plot；
Fig. 6 is that Unity display interfaces Chinese character selects exemplary plot.
Embodiment
It is main below that the application example that a quiet dynamic gesture of the invention is combined is provided.Basic procedure is referring to Fig. 1.
Firstly the need of connected using Socket communications realize writing letter client and realize Unity chinese displies circle
The server in face.Guarantee regular communication messages.Such as Fig. 2；
Open Kinect and carry out Face datection.Give tacit consent to first detected face when multiple faces are detected simultaneously
For this operating personnel；
The detection of finger and finger tip.The position of two hand general profiles of real-time synchronization and finger tip.Naked hand is used herein as to carry out
Operation：A finger is stretched out, a green point occurs in interface；Stretch out two fingers, interface occur two green points, the like completion
Detection.Such as Fig. 3；Usual Kinect to hand region segmentation is out detected and recognized according to depth value in itself, due to outer
Boundary's noise jamming and single depth value cause bigger error, unsuitable to be tested and extended.In addition regional edge of going smoothly can also be made
The boundary of edge is relatively obscured, and is once swung, and effect is more bad.Yan Hao is employed herein《Based on Kinect's
Three-dimensional multifinger tracking algorithm and application》In the method mentioned.
Finger tip is tracked and track is drawn.Water-color paintbrush is simulated using finger tip and carries out alphabetical writing：When the right hand one finger of detection,
Finger type is not limited, and left hand clenches one's fists all the time, it is detected that finger tip number is 0.It can be carried out depicting for track.Such as
Fig. 4.Note avoiding the occurrence of the phenomenon of " vertical ", result in the illusion of " right hand is clenched fist ".The problem of reason comes from crevice projection angle：
When our right hand be exactly in it is vertical with Kinect generations, then finger tip can be blocked by finger, cause to can't detect finger tip
Illusion；Because the phenomenon (influence of noise and interference in system) of shake occurs in general finger tip movement, this
In invention smooth operation is carried out using Kalman filter；
Letter writing standardization.The regular example of capitalization that reference picture 5 is embodied is, it is necessary to add a group state machine system：It is left
Hand is stretched out the five fingers and remained stationary as, and right hand holding of clenching fist can't detect a finger.Now the movement of the right hand will not produce track, shape
Into breakpoint, when the right hand stretches out forefinger, the now movement of the right hand may proceed to produce track again.Thus process can simulate capitalization
Regular writing；
Track identification OCR.It is identified using GoogleTesseract Open Source OCR, herein only to big
The mother that writes draws and handled.The five fingers are opened with left hand, the status mechanism that the right hand clenches one's fists carrys out trigger event；
Rule of judgment.When recognized be not expected letter when, it is necessary to the letter of deletion error：The right hand is held with a firm grip all the time
Fist, left hand becomes the state of a finger, and finger type is not limited.And the corresponding fingertip locations of Kinect, phase in observing interface
The green point answered.And it is moved to the button on interface with " cancellation ", the event of triggering deletion error letter.As Fig. 4 is corresponding
" Cancel " button.Then repeat the above steps；
Rule of judgment.Whether the letter that writing frame is present reaches expected number.If " bai " is, it is necessary to " b ", " a " and " i "
Three alphabetical correct writings and it is shown in dialog box, now writing frame contains 3 letters.If it is above-mentioned to be not reaching to repetition
Step；If now reaching expected requirement, it is necessary to this character string, that is, complete phonetic, issue and realize that unity shows
Show in the server at interface：The right hand is clenched fist all the time, and left hand becomes the state of a finger, and finger type is not limited.And observing interface
The corresponding fingertip location of upper Kinect, corresponding green point.And be moved on interface with " it is determined that " button carry out trigger event.Such as
Corresponding " Sure " buttons of Fig. 4.Now writing frame is cleared, and both hands keep clench fist state, the response of waiting for server；
It is sent to unity interfaces, the desired Chinese character of selection." pinyin-Hanzi " database is retrieved, Unity interfaces occur
Corresponding five Chinese characters (such as " bai ", there is " white ", " hundred ", " cypress ", " pendulum " and " visiing "), followed by the selection of Chinese character：The right hand
Fist is held, makes its detection should not finger tip；Which Chinese character at the finger tip number correspondence Unity interfaces that left hand is detected.Such as Fig. 6
It is shown；
Above-mentioned steps are repeated if necessary to writing Chinese characters again.
Claims (4)
1. a kind of Chinese-character writing method that quiet dynamic gesture is combined, it is characterised in that this method comprises the following steps：
Face datection is carried out using Kinect, operator is determined；
Operator's both hands profile and respectively finger tip are recognized, and determines to write finger tip；
Real-time tracking writing finger tip simultaneously carries out track drafting, detects when completing order, will leave the picture of handwriting trace
Carry out OCR identifications；Corresponding letters are obtained, when the letter of acquisition is not expected letter, the letter is deleted in triggering, is painted again
System；The step is repeated until completing whole phonetic；
The corresponding candidate Chinese character of the phonetic is pushed according to whole phonetic, Chinese character needed for being selected using static gesture.
2. the Chinese-character writing method that quiet dynamic gesture according to claim 1 is combined, it is characterised in that tracking finger tip process
Carry out carrying out smooth operation using Kalman filter during the drafting of track.
3. the Chinese-character writing method that quiet dynamic gesture according to claim 1 is combined, it is characterised in that painted carrying out track
When processed, breakpoint is set：Fist will be held and cause system detectio less than writing finger tip, breakpoint is formed, holding clench fist state by hand from upper
One stroke terminal is moved to next stroke starting point, stretches out writing finger tip and continues to write.
4. the Kanji writing systems that a kind of quiet dynamic gesture is combined, it is characterised in that the system includes realizing the visitor of writing letter
Family end and the server for realizing Unity chinese displies interface, client are based on passing through Socket between Kinect device, with server
Communicate with connecting；The Face datection to operator and the detection to both hands profile and finger tip are completed in client；And by real-time
The movement of finger tip is tracked to carry out the drafting of track；Detect after order of completing, OCR identifications are carried out to track picture, obtained
Obtain alphabetical accordingly；Until completing phonetic, client sends whole phonetic to server, and server shows the pinyin candidate Chinese
Word, expected Chinese character is selected by static gesture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710198920.2A CN107122042A (en) | 2017-03-29 | 2017-03-29 | The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710198920.2A CN107122042A (en) | 2017-03-29 | 2017-03-29 | The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107122042A true CN107122042A (en) | 2017-09-01 |
Family
ID=59718277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710198920.2A Pending CN107122042A (en) | 2017-03-29 | 2017-03-29 | The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107122042A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107680104A (en) * | 2017-09-23 | 2018-02-09 | 天津大学 | Hand Gesture Segmentation method based on kalman filtering |
| CN113963355A (en) * | 2021-12-22 | 2022-01-21 | 北京亮亮视野科技有限公司 | OCR character recognition method, device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1144353A (en) * | 1995-10-11 | 1997-03-05 | 柳德钟 | Regular Chinese phonetic alphabet Chinese character input method |
| CN103226388A (en) * | 2013-04-07 | 2013-07-31 | 华南理工大学 | Kinect-based handwriting method |
-
2017
- 2017-03-29 CN CN201710198920.2A patent/CN107122042A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1144353A (en) * | 1995-10-11 | 1997-03-05 | 柳德钟 | Regular Chinese phonetic alphabet Chinese character input method |
| CN103226388A (en) * | 2013-04-07 | 2013-07-31 | 华南理工大学 | Kinect-based handwriting method |
Non-Patent Citations (1)
| Title |
|---|
| 晏浩: "基于Kinect的实时稳定的三维多手指跟踪算法", 《计算机辅助设计与图形学学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107680104A (en) * | 2017-09-23 | 2018-02-09 | 天津大学 | Hand Gesture Segmentation method based on kalman filtering |
| CN107680104B (en) * | 2017-09-23 | 2020-10-23 | 天津大学 | Gesture segmentation method based on Kalman filtering |
| CN113963355A (en) * | 2021-12-22 | 2022-01-21 | 北京亮亮视野科技有限公司 | OCR character recognition method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Air-writing recognition—Part I: Modeling and recognition of characters, words, and connecting motions | |
| KR101825154B1 (en) | Overlapped handwriting input method | |
| US20150084859A1 (en) | System and Method for Recognition and Response to Gesture Based Input | |
| CN103226387B (en) | Video fingertip localization method based on Kinect | |
| Kin et al. | Proton++ a customizable declarative multitouch framework | |
| Yang et al. | Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine | |
| CN103093196B (en) | Character interactive input and recognition method based on gestures | |
| Kumar et al. | A lexicon-free approach for 3D handwriting recognition using classifier combination | |
| CN109344793A (en) | Aerial hand-written method, apparatus, equipment and computer readable storage medium for identification | |
| CN103226388A (en) | Kinect-based handwriting method | |
| CN106502390B (en) | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognition | |
| CN108628455B (en) | Virtual sand painting drawing method based on touch screen gesture recognition | |
| CN104714650B (en) | A kind of data inputting method and device | |
| CN107346207B (en) | Dynamic gesture segmentation recognition method based on hidden Markov model | |
| CN104915009B (en) | The method and system of gesture anticipation | |
| CN102662465A (en) | Method and system for inputting visual character based on dynamic track | |
| CN107122042A (en) | The Chinese-character writing method and system that a kind of quiet dynamic gesture is combined | |
| Shen et al. | Fast and robust mid-air gesture typing for ar headsets using 3d trajectory decoding | |
| KR20190027287A (en) | The method of mimesis for keyboard and mouse function using finger movement and mouth shape | |
| Spanogianopoulos et al. | Human computer interaction using gestures for mobile devices and serious games: A review | |
| JPWO2014054717A1 (en) | User interface device, user interface method, program, and computer-readable information storage medium | |
| Raj et al. | Human computer interaction using virtual user computer interaction system | |
| CN111240485A (en) | Basketball referee gesture recognition method based on Kinect skeleton information | |
| Amma et al. | Airwriting: Bringing text entry to wearable computers | |
| US9870063B2 (en) | Multimodal interaction using a state machine and hand gestures discrete values |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170901 |
|
| RJ01 | Rejection of invention patent application after publication |