CN106774995A - A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic - Google Patents
A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic Download PDFInfo
- Publication number
- CN106774995A CN106774995A CN201611157261.XA CN201611157261A CN106774995A CN 106774995 A CN106774995 A CN 106774995A CN 201611157261 A CN201611157261 A CN 201611157261A CN 106774995 A CN106774995 A CN 106774995A
- Authority
- CN
- China
- Prior art keywords
- brushwork
- dimensional
- style
- dimensional style
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to a kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic, belong to field of human-computer interaction.User is gathered by ultrasound positioning device and streaks the three-dimensional coordinate of track in space in the three-dimensional style of brushwork of input；Calculate space and streak the three-dimensional coordinate of track in tri- projection coordinates of plane of xOy, zOx and yOz, and be smoothed, remove the little deviation caused by the shake of user's hand or ambient noise；Three-dimensional style of brushwork projection coordinate after by smoothing processing is matched with the projection coordinate in system three-dimensional style of brushwork storehouse, the match is successful then performs corresponding instruction, it fails to match then carries out the adaptive updates in system style of brushwork storehouse automatically, to the three-dimensional style of brushwork for adding user to be accustomed in system, to adapt to the input habit of different user.The style of brushwork is introduced three dimensions by the present invention, improves the interactive sense of reality, proposes three-dimensional style of brushwork recognition methods with recognition speed and accuracy higher, and Consumer's Experience is good.
Description
Technical field
The invention belongs to field of human-computer interaction, and in particular to a kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic.
Background technology
The high speed development of information technology is human being's production, life brings extensively and deep effect, is also promoted man-machine
The development of interaction technique, it is traditional based on the man-machine of windows, icons, menu, point device patterns (abbreviation WIMP)
Interactive mode needs continually to switch mouse, keyboard, interface tool button, interaction poor continuity, has been difficult to meet the need of people
Will.And pen-based interaction meets mankind's natural interaction characteristic, with preferable continuity and naturality, but based on biography more than pen-based interaction
The paper pen metaphor of system, can only realize interacting in two dimensional surface, limitation be shown in space application, therefore the two-dimentional style of brushwork is handed over
The three-dimensional style of brushwork interaction technique for being mutually combined generation with three-dimensional real scene interaction has important researching value.The three-dimensional style of brushwork is handed over
Physical distance is introduced style of brushwork interaction by mutual technology, interacting for pen and display terminal can be realized in the range of certain distance, with more
The mode of natural harmony carries out terminal operation, is not limited by use environment and user behavior, has broad application prospects.
In terms of pen-based interaction, domestic Patents are mostly ground in terms of two-dimentional pen-based interaction and its correlation technique
Study carefully.Chinese patent " plane three-freedom-degree pen type force sense interactive " (application number CN201110221211.4) discloses a kind of flat
Face Three Degree Of Freedom degree pen type force sense interactive, may apply to play, imparts knowledge to students, in the field such as painting practice, making user pass through to hold
Hold interaction pen and directly experience corresponding feedback force and moment of torsion while operated on device screen, realize plane visual and
The fusion of force display.Chinese patent " judges the pen-based interaction method and system for the treatment of based on handwriting input state self-adaption "
(application number CN201510018914.5) discloses a kind of pen-based interaction side that treatment is judged based on handwriting input state self-adaption
Method, during user carries out hand-drawing graphics input and handwriting is input into, automatic decision user's current input state is gone forward side by side
Row real-time processing, simplifies user's operating process, makes user's nature, efficiently completes pen-based interaction.Chinese patent is " based on projection
The pen type interactive system of tracking " (application number CN201010199301.3) discloses a kind of pen type interactive system based on projection,
Completion pen-based interaction is cooperated to by infrared projection scan module, wireless communication receiver module, projecting apparatus and interaction pen,
Realize on a large scale, accurate interaction while realize brush writing interaction effect true to nature.Tian Feng et al. is in paper《UPIL：With
The pen type interactive system design of user-center》In propose a kind of design side of the pen type interactive system of customer-centric
Method, improves the availability of pen type interactive system.But these methods for designing only consider two dimensional surface situation, by pen-based interaction skill
Art is limited in plane operations, does not propose to be applied to the methods such as the pen-based interaction identification of three dimensions, person's handwriting tracking, limits use
The naturality of family operation and flexibility.
Gesture is a kind of naturally direct man-machine interaction mode of three dimensions, in terms of gesture interaction, the existing phase in the country
Patent is closed to be studied.Chinese patent " gesture identifying device, gesture identification method and gesture recognition system " (application number
CN201510587921.7 a kind of gesture identifying device, method and system) are disclosed, the system recognizes motion hand by sensor
Refer to the input mode needed relative to the movable information and device of other fingers, and by input mode generation unit correspondence generation key
Disk input position, function key, mouse movement etc., realize fast and accurately simulation input content.Chinese patent " know by a kind of gesture
Other method and device " (application number CN201610195737.2) discloses a kind of gesture identification method, is obtained by hand video
The Density Distribution Feature vector of gesture to be identified, by the Euclidean distance for calculating the vector and preset standard gesture library characteristic vector
To determine target gesture, the accuracy of gesture identification is effectively improved.Chinese patent " control method of infrared gesture identifying device,
Device and equipment " (application number CN201610286718.0) discloses a kind of control method of infrared gesture identifying device, infrared
Gesture identifying device and equipment, improve the accuracy rate and reliability of gesture identification, lift user experience.These design sides
Although method is all the staff activity of three dimensions situation, gesture interaction activity is all concentrated on, do not consider style of brushwork interaction three
Application in dimension space, and there is the inherent shortcomings such as the otherness on ambiguity, diversity and space-time in gesture interaction, cause gesture
Identifying system discrimination is low, and real-time is poor.
The interaction of the three-dimensional style of brushwork is one kind of gesture activity, many relative to finger-joint, the features such as flexibility is strong, three-dimensional pen type
Interaction only needs to change the parameters such as moving displacement, the anglec of rotation of pen, can overcome ambiguity, diversity etc. by the control to pen
Defect, precision is higher.It is domestic at present still at an early stage to the research of three-dimensional style of brushwork interaction technique, three-dimensional style of brushwork interaction dress
Put, localization method and style of brushwork recognition methods etc. are not studied fully.
The content of the invention
The present invention provides a kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic, it is therefore an objective to which design is adapted to the three-dimensional style of brushwork and hands over
Localization method and style of brushwork recognition methods that mutual device is used, the positioning accurate of three-dimensional style of brushwork device is improved using ultrasonic positioning techniques
Degree；By by the three-dimensional coordinate of three-dimensional style of brushwork space tracking be converted into two-dimensional projection's coordinates matrix and with system three-dimensional style of brushwork storehouse in
Corresponding two-dimensional projection's coordinates matrix is matched, and realizes identification of the system to the three-dimensional style of brushwork；Using system three-dimensional style of brushwork storehouse
Adaptive learning mechanism adapts to the input habit of user, reduces influence of user's custom to three-dimensional style of brushwork recognition effect.
The technical solution adopted by the present invention is：Comprise the following steps：
Step (1) three-dimensional style of brushwork data acquisition：User is gathered in the three-dimensional style of brushwork of input in sky using ultrasonic positioning techniques
Between streak the three-dimensional coordinate of track；
Step (2) style of brushwork data for projection is calculated：The three-dimensional coordinate for calculating the three-dimensional style of brushwork space tracking of user input exists
Projection coordinate in tri- planes of xOy, zOx and yOz, and be smoothed, remove the shake because of user's hand or ambient noise
The little deviation for causing；
Step (3) three-dimensional style of brushwork matching：Three-dimensional pen after respectively in tri- planes of xOy, zOx and yOz to smoothing processing
Gesture projection coordinate is matched with corresponding projection coordinate in system three-dimensional style of brushwork storehouse, is recognized if both matching degrees are more than 90%
For the match is successful, the corresponding instruction of the three-dimensional style of brushwork is performed；
Step (4) three-dimensional style of brushwork storehouse adaptive learning：System three-dimensional style of brushwork storehouse is carried out according to user's three-dimensional style of brushwork custom
Adaptive updates, make system progressively adapt to the input habit of user.
Further, the three-dimensional style of brushwork data acquisition in the step (1) to implement process as follows：It is fixed using ultrasound
Position technology, sends synchronous reception signal by infrared synchronous device first, for ultrasonic receiver provides time reference, passes through afterwards
The supersonic generator that style of brushwork interactive device front end is installed sends ultrasonic wave, is connect by 3 ultrasonic waves installed on display terminal
Device synchronization received ultrasonic signal is received, so as to streak the three-dimensional coordinate S of track in space when orienting the user input three-dimensional style of brushwork
(xS,yS,zS)：
zS≥0
Wherein A (xA,0,0)、B(0,yB,0)、C(xC, 0,0) and it is respectively 3 ultrasonic receivers installed on display terminal
Three-dimensional coordinate, l1、l2、l3Respectively the distance between pen tip position and 3 ultrasonic receivers, can be received by ultrasonic wave
Device is obtained.
Because human gesture's activity is general in 10Hz or so, and style of brushwork interaction is more real gesture activity, according to how
Qwest's law understands, as sample frequency fSMore than highest frequency f in signalmax2 times when sampling after data signal can be complete
Site preparation retains the information in primary signal, and original analog signal is recovered without distortion, and sampling frequency is should ensure that in engineer applied
Rate is 5~10 times of signal highest frequency, therefore present invention determine that ultrasonic sensor sending signal frequency is 60Hz, i.e. signal
Sent once every 16ms, ensureing that reception signal is successional while reducing plant energy consumption.
Further, it is as follows that what the style of brushwork data for projection in the step (2) was calculated implements process：User is made first
The three-dimensional coordinate z of the three-dimensional style of brushwork space tracking of inputS=0, the projection for calculating three-dimensional style of brushwork space tracking in xOy planes is sat
Mark, and by the projection coordinate with two-dimensional projection coordinates matrix S1(xS,yS) form storage；Make the three-dimensional style of brushwork of user input empty
Between track three-dimensional coordinate yS=0, calculate projection coordinate of the three-dimensional style of brushwork space tracking in zOx planes, and by the projection coordinate
With two-dimensional projection coordinates matrix S2(xS,zS) form storage；Make the three-dimensional coordinate x of the three-dimensional style of brushwork space tracking of user inputS
=0, calculate projection coordinate of the three-dimensional style of brushwork in yOz planes, and by the projection coordinate with two-dimensional projection coordinates matrix S3(yS,zS)
Form storage：
3 two-dimensional projection's coordinates matrixs are smoothed using Savitzky-Golay smoothing filters afterwards, are obtained
Two-dimensional projection's coordinates matrix after to smoothing processing, removes the trajector deviation caused by the shake of user's hand or ambient noise.For
Ensure system real time, the present invention is using 3 × 3 Savitzky-Golay smoothing filter templates to S1(xS,yS)、S2(xS,
zS)、S3(yS,zS) be respectively processed, by moving window by projection matrix and filter template convolution, using least square
Method carries out best fit, and 3 × 3 convolution masks a (k) that the present invention is used are specially：
Further, it is as follows that what the three-dimensional style of brushwork in the step (3) was matched implements process：By the system three-dimensional style of brushwork
3 two-dimensional projection's coordinates matrixs of each style of brushwork are used as pattern matrix in storehouse；By the two dimension after user input and smoothed treatment
Projection coordinate's matrix is used as objective matrix；Two-dimensional projection by the three-dimensional style of brushwork effective coverage of user input from after smoothing processing sits
Mark matrix in split, and zoom to pattern matrix identical size, as the three-dimensional style of brushwork match sub-goal matrix；It is logical
The Normalized Cross Correlation Function for crossing calculation template matrix and sub-goal matrix is characterized to both correlations, and will normalization
Cross-correlation function value thinks that the match is successful as matching degree between the two if matching degree is more than 90%, performs three-dimensional
The corresponding instruction of the style of brushwork.
System three-dimensional style of brushwork storehouse internal storage system three-dimensional style of brushwork information set in advance, attainable basic function includes：
Amplify, reduce, on turn over, under turn over, cancel, close, opening, advancing, retreating, maximized window and minimize window, each is three-dimensional
The style of brushwork is recorded in 3 two-dimensional projection's coordinates matrix modes.
By the m × m ranks two-dimensional projection coordinates matrix S of each style of brushwork in three-dimensional style of brushwork storehousenAs pattern matrix An；By user
Be input into and by smoothing processing M × N ranks two-dimensional projection's coordinates matrix as objective matrix Bn；The matching of the three-dimensional style of brushwork firstly the need of
M × N ranks two-dimensional projection coordinates matrix B by the three-dimensional style of brushwork effective coverage of user input from after smoothing processingnIn split,
The matrix that will be partitioned into afterwards is scaled and pattern matrix AnIdentical size, i.e. m × m ranks, using the matrix after scaling as son
Objective matrix Cn, represent three-dimensional style of brushwork space tracking in tri- two dimensions of plane of xOy, zOx and yOz when wherein n=1,2,3 respectively
Projection coordinate's matrix；By calculation template matrix AnWith sub-goal Matrix CnNormalized Cross Correlation Function to both correlations
Characterized, and using Normalized Cross Correlation Function value as matching degree R between the twon, realize template matches process, template
Matrix AnWith sub-goal Matrix CnCross-correlation function normalized form it is as follows：
It is 90% that the present invention chooses pattern matrix and the matching degree threshold value of sub-goal matrix, only when the three-dimensional of user input
When 3 two-dimensional projection's coordinates matrixs of the style of brushwork reach 90% with pattern matrix matching degree, system judges the three-dimension gesture
With success, the corresponding instruction of the three-dimensional style of brushwork is performed.Matching degree threshold value 90% is set up on the basis of many experiments, works as initialization system
Matching degree threshold value can ensure system real time when being 90%, and the most three-dimensional style of brushworks inputs of user are intended to be known by system
Not.Pattern matrix size has an impact to system processing speed and style of brushwork accuracy of identification, and template crosses senior general causes amount of calculation to increase,
Influence system real time, the too small recognition effect that will reduce the three-dimensional style of brushwork of template.The present invention is sat using 32 × 32 two-dimensional projection
Mark pattern matrix A of the matrix as system three-dimensional style of brushwork storehousen, can simultaneously ensure system real time and style of brushwork accuracy of identification.
Further, the three-dimensional style of brushwork storehouse adaptive learning in the step (4) to implement process as follows：Work as user
The three-dimensional style of brushwork B being input into for the first time1(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,a2,a3) matching journey
When degree is less than 90%, the system None- identified style of brushwork is not responding to user instruction, then system enters three-dimensional style of brushwork storehouse adaptive learning
In the stage, adaptive updates are carried out to three-dimensional style of brushwork storehouse according to user's three-dimensional style of brushwork custom.BiIt is the three-dimensional pen of user's i ＆ lt input
Gesture information；a1、a2、a3It is projection coordinate square of the three-dimensional style of brushwork in the three-dimensional style of brushwork storehouse of system in tri- planes of xOy, zOx and yOz
Battle array；b1、b2、b3For user for the first time input the three-dimensional style of brushwork tri- planes of xOy, zOx and yOz projection coordinate's matrix.
As the three-dimensional style of brushwork B of user's input for the first time1(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,
a2,a3) matching degree when being less than 90%, the system None- identified style of brushwork, then the style of brushwork information temporary storage enter the interim three-dimensional pen of system
In gesture storehouse, and three-dimensional style of brushwork B is re-entered by user2(b1,b2,b3), the three-dimensional style of brushwork B that system-computed user is input into for second2
(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,a2,a3), the three-dimensional style of brushwork B that is input into of user second2(b1,
b2,b3) with user for the first time input three-dimensional style of brushwork B1(b1,b2,b3) (or certain two-dimensional projection's coordinates matrix of the three-dimensional style of brushwork)
Matching degree, and the three-dimensional style of brushwork certain the two-dimensional projection's coordinates matrix of the style of brushwork (or three-dimensional) by matching degree more than 90% is recorded into
In system three-dimensional style of brushwork storehouse, instead of the three-dimensional style of brushwork in current system three-dimensional style of brushwork storehouse, (or certain two-dimensional projection of the three-dimensional style of brushwork sits
Mark matrix), system is progressively adapted to the input habit of user.
The three-dimensional style of brushwork of the interim three-dimensional style of brushwork storehouse memory storage user input of system, it amplifies, reduces, on turn over, under turn over, remove
Pin, close, open, advancing, retreating, maximized window and minimizing that the three-dimensional style of brushwork storehouse of window function and system is corresponding, each three
The dimension style of brushwork is recorded in 3 two-dimensional projection's coordinates matrix modes.
The present invention has the following advantages that：
1st, three-dimensional style of brushwork interactive device can realize that pen is interacted with the noncontact of display terminal, with more natural, harmonious side
Formula carries out terminal operation；
The three-dimensional coordinate of track is streaked in space when the 2nd, determining the input three-dimensional style of brushwork using ultrasonic positioning techniques, with positioning
High precision, the advantage such as real-time is good, overcomes the ambiguity and diversity defect of gesture activity；
3rd, space style of brushwork information is converted into two-dimensional projection's information to be processed, reduces computation complexity, improve system
The recognition speed of the three-dimensional style of brushwork；
4th, the style of brushwork information of user input is matched with three-dimensional style of brushwork storehouse information using template matching method, identification essence
Degree is high, and processing speed is fast；
5th, using Normalized Cross Correlation Function value as matching degree, the style of brushwork information to user input is described, real
Automatically updating for existing three-dimensional style of brushwork storehouse, is suitable for the input habit of different user, reduces user's custom to gesture recognition effect
Influence, improves the Consumer's Experience of three-dimensional style of brushwork interactive system.
Brief description of the drawings
Fig. 1 is the three-dimensional style of brushwork recognition methods framework based on localization by ultrasonic；
Fig. 2A is the operation chart that the page is closed using three-dimensional style of brushwork interactive device；
Fig. 2 B are ultrasonic positioning techniques schematic diagrams；
Fig. 3 A are that the projection of three-dimensional " revocation " style of brushwork calculates schematic diagram；
Fig. 3 B are projection of three-dimensional " revocation " style of brushwork in xOy planes；
Fig. 3 C are projection of three-dimensional " revocation " style of brushwork in zOx planes；
Fig. 3 D are projection of three-dimensional " revocation " style of brushwork in yOz planes；
Fig. 4 A are that the three-dimensional style of brushwork by taking " revocation " operation as an example matches schematic diagram；
Fig. 4 B are three-dimensional style of brushwork template matches principle schematics.
Specific embodiment
Below in conjunction with the accompanying drawings with three-dimensional " revocation " style of brushwork as a example by embodiment the present invention will be described in detail.
Xu Lishuan, Tian Feng et al. are in paper《Reproduce an intelligent sketching abnormal frame Joints of two-way adaptation》It is middle " to be gone out with stroke
The symbol for coming calls the technology of computer command " it is referred to as " gesture ", but refer to the plane style of brushwork interaction scenario concept more, therefore
Be referred to as three dimensions " technology of computer command is called with stroke symbol out " " the three-dimensional style of brushwork " by the present invention.
Fig. 1 is the three-dimensional style of brushwork recognition methods framework based on localization by ultrasonic, and methods described mainly includes：
Step 102 three-dimensional style of brushwork data acquisition：Using ultrasonic positioning techniques collection user in the three-dimensional style of brushwork 101 of input
Streak the three-dimensional coordinate of track in space；
Step 103 style of brushwork data for projection is calculated：The three-dimensional coordinate for calculating the three-dimensional style of brushwork space tracking of user input 101 exists
Projection coordinate in tri- planes of xOy, zOx and yOz, and be smoothed, remove the shake because of user's hand or ambient noise
The little deviation for causing；
Step 104 three-dimensional style of brushwork matching：Three-dimensional pen after respectively in tri- planes of xOy, zOx and yOz to smoothing processing
Coordinate data in gesture projection coordinate and system the three-dimensional style of brushwork storehouse is matched, and is thought if both matching degrees are more than 90%
With success, the corresponding instruction of the three-dimensional style of brushwork is performed；
Step 105 three-dimensional style of brushwork storehouse adaptive learning：System three-dimensional style of brushwork storehouse is carried out according to user's three-dimensional style of brushwork custom
Adaptive updates, make system progressively adapt to the input habit of user.
The three-dimensional memory storage of style of brushwork storehouse 106 dispatch from the factory before system three-dimensional style of brushwork information set in advance, it is attainable basic
Function includes：Amplify, reduce, on turn over, under turn over, cancel, close, opening, advancing, retreating, maximized window, minimize window,
Each three-dimensional style of brushwork is recorded in 3 two-dimensional projection's coordinates matrix modes.
Fig. 2A is the operation chart that the page is closed using three-dimensional style of brushwork interactive device, and user represents in space input and " closes
Close " the three-dimensional style of brushwork 201 of instruction, in sky when gathering the user input three-dimensional style of brushwork by 3 ultrasonic receivers 202,203,204
Between streak the three-dimensional coordinate of track, calculate space and streak the three-dimensional coordinate of track and sat in the projection of tri- planes of xOy, zOx and yOz
Mark, and be smoothed, remove the little deviation caused by the shake of user's hand or ambient noise；After by smoothing processing three
Dimension style of brushwork projection coordinate is matched with the coordinate data in system three-dimensional style of brushwork storehouse, and the match is successful then performs corresponding instruction, closes
Current page is closed, homepage is returned.Style of brushwork interactive system of the present invention uses electronic whiteboard 205 as display terminal, can
Connection electronic equipment and by projecting apparatus 206 realize image show, the function such as real-time projection.
Fig. 2 B are ultrasonic positioning techniques schematic diagrams, and the present invention uses ultrasonic positioning techniques, is sent out by infrared synchronous device first
Synchronous reception signal is sent, for ultrasonic receiver provides time reference, and is connect with 3 supersonic wave synchronisms installed on display terminal
Space coordinates 208,209,210 are set up on the basis of receipts device 202,203,204, the ultrasound installed by style of brushwork interactive device front end
Wave producer 207 sends ultrasonic wave, is received by the 3 supersonic wave synchronism receivers 202,203,204 installed on display terminal
Ultrasonic signal, so as to streak the three-dimensional coordinate S (x of track in space when orienting the user input three-dimensional style of brushworkS,yS,zS)：
zS≥0
Wherein A (xA,0,0)、B(0,yB,0)、C(xC, 0,0) be respectively on display terminal 205 install 3 ultrasonic waves connect
Receive the three-dimensional coordinate of device, l1、l2、l3Respectively the distance between pen tip position and 3 ultrasonic receivers, can be by ultrasonic wave
Receiver is obtained.
Because human gesture's activity is general in 10Hz or so, and style of brushwork interaction is more real gesture activity, according to how
Qwest's law understands, as sample frequency fSMore than highest frequency f in signalmax2 times when sampling after data signal can be complete
Site preparation retains the information in primary signal, and primary signal is recovered without distortion, should ensure that sample frequency is letter in engineer applied
5~10 times of number highest frequency, therefore present invention determine that ultrasonic sensor sending signal frequency be 60Hz, i.e. signal every
16ms sends once, is ensureing that reception signal is successional while reducing plant energy consumption.
Fig. 3 A are that the projection of three-dimensional " revocation " style of brushwork calculates schematic diagram, and Fig. 3 B are three-dimensional " revocation " style of brushworks in xOy planes
Projection；Fig. 3 C are projection of three-dimensional " revocation " style of brushwork in zOx planes；Fig. 3 D are projection of three-dimensional " revocation " style of brushwork in yOz planes；
The three-dimensional coordinate z of the three-dimensional style of brushwork space tracking of user input is made firstS=0, three-dimensional style of brushwork space tracking is calculated in xOy planes
Projection coordinate 2111, and by the projection coordinate with two-dimensional projection coordinates matrix S1(xS,yS) form storage；Make user input
Three-dimensional style of brushwork space tracking three-dimensional coordinate yS=0, projection coordinate 2112 of the three-dimensional style of brushwork in zOx planes is calculated, and should
Projection coordinate is with two-dimensional projection coordinates matrix S2(xS,zS) form storage；Make the three of the three-dimensional style of brushwork space tracking of user input
Dimension coordinate xS=0, projection coordinate 2113 of the three-dimensional style of brushwork in yOz planes is calculated, and by the projection coordinate with two-dimensional projection's coordinate
Matrix S3(yS,zS) form storage：
3 two-dimensional projection's coordinates matrixs are smoothed using Savitzky-Golay smoothing filters afterwards, are obtained
Two-dimensional projection's coordinates matrix after to smoothing processing, removes the trajector deviation caused by the shake of user's hand or ambient noise.For
Ensure system real time, the present invention is using 3 × 3 Savitzky-Golay smoothing filters template to three-dimensional style of brushwork space tracking
Three-dimensional coordinate tri- planes of xOy, zOx and yOz projection coordinate S1(xS,yS)、S2(xS,zS)、S3(yS,zS) 211 enter respectively
Row treatment, by moving window by projection matrix and filter template convolution, best fit, this hair is carried out using least square method
3 × 3 convolution masks a (k) of bright use are specially：
Fig. 4 A are that the three-dimensional style of brushwork by taking " revocation " operation as an example matches schematic diagram, and what the style of brushwork was matched implements process such as
Under：Using 3 two-dimensional projection's coordinates matrixs of each style of brushwork in system three-dimensional style of brushwork storehouse as pattern matrix；By user input and pass through
Two-dimensional projection's coordinates matrix after smoothing processing is used as objective matrix 402；By the three-dimensional style of brushwork effective coverage of user input from flat
Split 403 in two-dimensional projection's coordinates matrix 401 after sliding treatment, and zoom to pattern matrix identical size, as
The sub-goal matrix 404 of three-dimensional style of brushwork matching；By calculation template matrix 405 and the Normalized Cross Correlation Function of sub-goal matrix
Correlation to both is characterized, and using Normalized Cross Correlation Function value as matching degree between the two, if matching journey
Degree thinks that the match is successful more than 90%, performs " revocation " instruction.
By the m × m ranks two-dimensional projection coordinates matrix S of each style of brushwork in three-dimensional style of brushwork storehousenAs pattern matrix An405；To use
Family be input into and by smoothing processing M × N ranks two-dimensional projection's coordinates matrix as objective matrix Bn402；Three-dimensional style of brushwork matching is first
First need the three-dimensional style of brushwork effective coverage of user input the M × N ranks two-dimensional projection coordinates matrix B from after smoothing processingnIn point
Cut out, the matrix that will be partitioned into afterwards is scaled and pattern matrix AnThe matrix of same size, i.e. m × m ranks, after scaling
Matrix is used as sub-goal Matrix Cn, represent three-dimensional style of brushwork space tracking in xOy, zOx and yOz tri- when wherein n=1,2,3 respectively
Two-dimensional projection's coordinates matrix of plane.By calculation template matrix AnWith sub-goal Matrix CnNormalized Cross Correlation Function to two
The correlation of person is characterized, and using Normalized Cross Correlation Function value as matching degree R between the twon, realize template
With process.Pattern matrix AnWith sub-goal Matrix CnCross-correlation function normalized form it is as follows：
It is 90% that the present invention chooses pattern matrix and the matching degree threshold value of sub-goal matrix, user input that and if only if
When 3 two-dimensional projection's coordinates matrixs of the three-dimensional style of brushwork reach 90% with pattern matrix matching degree, system judges the three-dimensional hand
The match is successful for gesture, performs the corresponding instruction of the three-dimensional style of brushwork.Matching degree threshold value 90% is set up on the basis of many experiments, works as setting
The matching degree threshold value of system can ensure system real time when being 90%, and the most three-dimensional style of brushwork input intentions of user are
System identification.The size of pattern matrix has an impact to system processing speed and style of brushwork accuracy of identification, and template crosses senior general to be caused to calculate
Amount increase, influences system real time, the too small recognition effect that will reduce the three-dimensional style of brushwork of template.The present invention is using 32 × 32 sizes
Two-dimensional projection coordinates matrix SnAs the pattern matrix A in system three-dimensional style of brushwork storehousen401, can simultaneously ensure system real-time processing speed
With style of brushwork accuracy of identification.
As the three-dimensional style of brushwork B of user's input for the first time1(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,
a2,a3) matching degree when being less than 90%, the system None- identified style of brushwork is not responding to user instruction, then system enters three-dimensional pen
In the gesture storehouse adaptive learning stage, adaptive updates are carried out to three-dimensional style of brushwork storehouse according to user's three-dimensional style of brushwork custom.BiIt is user i-th
The three-dimensional style of brushwork information of secondary input；a1、a2、a3For the three-dimensional style of brushwork in system three-dimensional style of brushwork storehouse is put down for tri- in xOy, zOx and yOz
Projection coordinate's matrix in face；b1、b2、b3Be user for the first time input the three-dimensional style of brushwork in tri- throwings of plane of xOy, zOx and yOz
Shadow coordinates matrix.
As the three-dimensional style of brushwork B of user's input for the first time1(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,
a2,a3) matching degree when being less than 90%, the system None- identified style of brushwork, then the style of brushwork information temporary storage enter the interim three-dimensional pen of system
In gesture storehouse, and three-dimensional style of brushwork B is re-entered by user2(b1,b2,b3), the three-dimensional style of brushwork B that system-computed user is input into for second2
(b1,b2,b3) with system three-dimensional style of brushwork storehouse in three-dimensional style of brushwork A (a1,a2,a3), the three-dimensional style of brushwork B that is input into of user second2(b1,
b2,b3) with user for the first time input three-dimensional style of brushwork B1(b1,b2,b3) (or certain two-dimensional projection's coordinates matrix of the three-dimensional style of brushwork)
Matching degree, and the three-dimensional style of brushwork certain the two-dimensional projection's coordinates matrix of the style of brushwork (or three-dimensional) by matching degree more than 90% is recorded into
In system three-dimensional style of brushwork storehouse, instead of the three-dimensional style of brushwork in current system three-dimensional style of brushwork storehouse, (or certain two-dimensional projection of the three-dimensional style of brushwork sits
Mark matrix), system is progressively adapted to the input habit of user.
The three-dimensional style of brushwork of the interim three-dimensional style of brushwork storehouse memory storage user input of system, it amplifies, reduces, on turn over, under turn over, remove
Pin, closing, opening, advance, retrogressing, maximized window, minimum window function are corresponding with system three-dimensional style of brushwork storehouse, and each is three-dimensional
The style of brushwork is recorded in 3 two-dimensional projection's coordinates matrix modes.
Claims (8)
1. a kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic, it is characterised in that comprise the following steps：
Step (1) three-dimensional style of brushwork data acquisition：Drawn in space in the three-dimensional style of brushwork of input using ultrasonic positioning techniques collection user
Cross the three-dimensional coordinate of track；
Step (2) style of brushwork data for projection is calculated：The three-dimensional coordinate of three-dimensional style of brushwork space tracking of user input is calculated in xOy, zOx
With the projection coordinate in tri- planes of yOz, and it is smoothed, removes micro- caused by the shake of user's hand or ambient noise
Little deviation；
Step (3) three-dimensional style of brushwork matching：The three-dimensional style of brushwork after respectively in tri- planes of xOy, zOx and yOz to smoothing processing is thrown
Shadow coordinate is matched with the coordinate data in system three-dimensional style of brushwork storehouse, thinks to match into if both matching degrees are more than 90%
Work(, performs the corresponding instruction of the three-dimensional style of brushwork；
Step (4) three-dimensional style of brushwork storehouse adaptive learning：System three-dimensional style of brushwork storehouse is carried out according to user's three-dimensional style of brushwork custom adaptive
Should update, system is progressively adapted to the input habit of user.
2. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：The step
(1) three-dimensional style of brushwork data acquisition in implement process as follows：Using ultrasonic positioning techniques, first by infrared synchronous device
Synchronous reception signal is sent, for ultrasonic receiver provides time reference, afterwards by surpassing that style of brushwork interactive device front end is installed
Sonic generator sends ultrasonic wave, and ultrasound information is synchronously received by 3 ultrasonic receivers installed on display terminal, from
And the three-dimensional coordinate of track is streaked in space when orienting the user input three-dimensional style of brushwork, the ultrasonic sensor sending signal for using
Frequency is 60Hz, i.e., signal sends once every 16ms, is ensureing that reception signal is successional while reducing plant energy consumption.
3. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：The step
(2) it is as follows that what the style of brushwork data for projection in was calculated implements process：The three-dimensional style of brushwork space tracking of user input is calculated first
Projection coordinate of the three-dimensional coordinate in tri- planes of xOy, zOx and yOz, then deposited in the form of two-dimensional projection's coordinates matrix
3 two-dimensional projection's coordinates matrixs are finally smoothed by storage using Savitzky-Golay smoothing filters, are smoothed
Two-dimensional projection's coordinates matrix after treatment, removes the trajector deviation caused by the shake of user's hand or ambient noise；To ensure to be
System real-time, using 3 × 3 Savitzky-Golay smoothing filter templates.
4. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：The step
(3) it is as follows that what the three-dimensional style of brushwork in was matched implements process：By 3 two-dimensional projections of each style of brushwork in system three-dimensional style of brushwork storehouse
Coordinates matrix is used as pattern matrix；Using the two-dimensional projection's coordinates matrix after user input and smoothed treatment as objective matrix；
The three-dimensional style of brushwork effective coverage of user input is split from the two-dimensional projection's coordinates matrix after smoothing processing, and is zoomed to
With pattern matrix identical size, as the three-dimensional style of brushwork match sub-goal matrix；By calculation template matrix and sub-goal square
The Normalized Cross Correlation Function of battle array is characterized to both correlations, and using Normalized Cross Correlation Function value as between the two
Matching degree, think that the match is successful if matching degree is more than 90%, perform the corresponding instruction of the three-dimensional style of brushwork.
5. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：The system three
Dimension style of brushwork storehouse internal storage system three-dimensional style of brushwork information set in advance, the basic function of realization includes：Amplify, reduce, on turn over, under
Turn over, cancel, closing, opening, advancing, retreating, maximized window, minimize window, each three-dimensional style of brushwork is with 3 two-dimensional projections
Coordinates matrix mode is recorded.
6. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：System three-dimensional pen
The pattern matrix in gesture storehouse is used to ensure system real-time processing speed and the style of brushwork using two-dimensional projection's coordinates matrix of 32 × 32 sizes
Accuracy of identification；The matching degree threshold value for choosing pattern matrix and sub-goal matrix is 90%, when the matching degree threshold of initialization system
Be worth for 90% when can ensure system real time, and the most three-dimensional style of brushworks input intentions of user are recognized by the system.
7. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 1, it is characterised in that：The step
(4) three-dimensional style of brushwork storehouse adaptive learning in implement process as follows：When the three-dimensional style of brushwork of user input is three-dimensional with system
When the matching degree of the three-dimensional style of brushwork in style of brushwork storehouse is less than 90%, the system None- identified style of brushwork is not responding to user instruction, then be
System enters the three-dimensional style of brushwork storehouse adaptive learning stage, and the style of brushwork information out to be temporarily stored in the interim three-dimensional style of brushwork storehouse of system, and by with
Style of brushwork information is re-entered at family, the three-dimensional pen in the three-dimensional style of brushwork and system three-dimensional style of brushwork storehouse that system-computed user is input into for second
The three-dimensional style of brushwork or certain two-dimensional projection of the three-dimensional style of brushwork that the three-dimensional style of brushwork and user that gesture, user are input into for second are input into for the first time are sat
Mark the matching degree of matrix, and the three-dimensional style of brushwork or certain two-dimensional projection's coordinates matrix of the three-dimensional style of brushwork by matching degree more than 90%
It is recorded into system three-dimensional style of brushwork storehouse, is thrown instead of the three-dimensional style of brushwork or certain two dimension of the three-dimensional style of brushwork in current system three-dimensional style of brushwork storehouse
Shadow coordinates matrix, makes system progressively adapt to the input habit of user.
8. the three-dimensional style of brushwork recognition methods based on localization by ultrasonic according to claim 7, it is characterised in that：Interim three-dimensional pen
The three-dimensional style of brushwork of gesture storehouse memory storage user input, it amplifies, reduces, on turn over, under turn over, cancel, close, opening, advancing, retreating,
Maximized window, minimum window function are corresponding with system three-dimensional style of brushwork storehouse, and each three-dimensional style of brushwork is with 3 two-dimensional projection's coordinates
Matrix-style is recorded.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611157261.XA CN106774995B (en) | 2016-12-14 | 2016-12-14 | A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611157261.XA CN106774995B (en) | 2016-12-14 | 2016-12-14 | A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106774995A true CN106774995A (en) | 2017-05-31 |
| CN106774995B CN106774995B (en) | 2019-05-03 |
Family
ID=58888982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611157261.XA Active CN106774995B (en) | 2016-12-14 | 2016-12-14 | A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106774995B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
| CN111862207A (en) * | 2020-06-05 | 2020-10-30 | 深圳市欣智旺电子有限公司 | Gesture judgment method and device for handwriting equipment, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200519764A (en) * | 2002-12-26 | 2005-06-16 | Koninkl Philips Electronics Nv | Method and system for three-dimensional handwriting recognition |
| CN102810008A (en) * | 2012-05-16 | 2012-12-05 | 北京捷通华声语音技术有限公司 | Air input system, method and air input acquisition equipment |
| CN103543826A (en) * | 2013-07-30 | 2014-01-29 | 广东工业大学 | Method for recognizing gesture based on acceleration sensor |
| CN104978022A (en) * | 2014-04-10 | 2015-10-14 | 联发科技股份有限公司 | Ultrasound-based methods for touchless gesture recognition, and apparatuses using the same |
| CN106200964A (en) * | 2016-07-06 | 2016-12-07 | 浙江大学 | A kind of method carrying out man-machine interaction based on motion track identification in virtual reality |
-
2016
- 2016-12-14 CN CN201611157261.XA patent/CN106774995B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200519764A (en) * | 2002-12-26 | 2005-06-16 | Koninkl Philips Electronics Nv | Method and system for three-dimensional handwriting recognition |
| CN102810008A (en) * | 2012-05-16 | 2012-12-05 | 北京捷通华声语音技术有限公司 | Air input system, method and air input acquisition equipment |
| CN103543826A (en) * | 2013-07-30 | 2014-01-29 | 广东工业大学 | Method for recognizing gesture based on acceleration sensor |
| CN104978022A (en) * | 2014-04-10 | 2015-10-14 | 联发科技股份有限公司 | Ultrasound-based methods for touchless gesture recognition, and apparatuses using the same |
| CN106200964A (en) * | 2016-07-06 | 2016-12-07 | 浙江大学 | A kind of method carrying out man-machine interaction based on motion track identification in virtual reality |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
| CN110031827B (en) * | 2019-04-15 | 2023-02-07 | 吉林大学 | Gesture recognition method based on ultrasonic ranging principle |
| CN111862207A (en) * | 2020-06-05 | 2020-10-30 | 深圳市欣智旺电子有限公司 | Gesture judgment method and device for handwriting equipment, computer equipment and storage medium |
| CN111862207B (en) * | 2020-06-05 | 2023-11-07 | 深圳市欣威智能有限公司 | Gesture judging method and device of handwriting equipment, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106774995B (en) | 2019-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105807926B (en) | A kind of unmanned plane man-machine interaction method based on three-dimensional continuous dynamic hand gesture recognition | |
| Kjeldsen et al. | Toward the use of gesture in traditional user interfaces | |
| Murthy et al. | Hand gesture recognition using neural networks | |
| CN102402289B (en) | Mouse recognition method for gesture based on machine vision | |
| CN104616028B (en) | Human body limb gesture actions recognition methods based on space segmentation study | |
| CN105787471A (en) | Gesture identification method applied to control of mobile service robot for elder and disabled | |
| CN103529944B (en) | A kind of human motion recognition method based on Kinect | |
| Wu et al. | Human hand modeling, analysis and animation in the context of HCI | |
| CN106569613A (en) | Multi-modal man-machine interaction system and control method thereof | |
| CN105069413A (en) | Human body gesture identification method based on depth convolution neural network | |
| CN108509026B (en) | Remote maintenance support system and method based on enhanced interaction mode | |
| CN107992792A (en) | A kind of aerial handwritten Chinese character recognition system and method based on acceleration transducer | |
| US9063573B2 (en) | Method and system for touch-free control of devices | |
| CN102981742A (en) | Gesture interaction system based on computer visions | |
| CN106502390B (en) | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognition | |
| CN107632699A (en) | Natural human-machine interaction system based on the fusion of more perception datas | |
| CN109800676A (en) | Gesture identification method and system based on depth information | |
| CN105328701A (en) | Teaching programming method for series mechanical arms | |
| CN104881127A (en) | Virtual vehicle man-machine interaction method and system | |
| CN106468917A (en) | A kind of tangible live real-time video image remotely assume exchange method and system | |
| CN106774995B (en) | A kind of three-dimensional style of brushwork recognition methods based on localization by ultrasonic | |
| CN106774938A (en) | Man-machine interaction integrating device based on somatosensory device | |
| Kabir et al. | A novel dynamic hand gesture and movement trajectory recognition model for non-touch HRI interface | |
| CN106933385A (en) | A kind of implementation method of the low-power consumption sky mouse pen based on three-dimensional ultrasonic positioning | |
| CN107894834A (en) | Gesture identification method and system are controlled under augmented reality environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |