CN107633551B - The methods of exhibiting and device of a kind of dummy keyboard - Google Patents
The methods of exhibiting and device of a kind of dummy keyboard Download PDFInfo
- Publication number
- CN107633551B CN107633551B CN201710709726.6A CN201710709726A CN107633551B CN 107633551 B CN107633551 B CN 107633551B CN 201710709726 A CN201710709726 A CN 201710709726A CN 107633551 B CN107633551 B CN 107633551B
- Authority
- CN
- China
- Prior art keywords
- model
- finger
- hands
- keyboard
- standard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Input From Keyboards Or The Like (AREA)
Abstract
The invention discloses the methods of exhibiting and device of a kind of dummy keyboard, wherein, the method includes：The both hands of user and body surface are modeled respectively, to build the both hands model of user and surface keyboard model in virtual reality scenario；The finger model completion that will be blocked in the both hands model, to establish standard both hands model；According to the current spatial location relationship between finger in the standard both hands model, automatic adjusument, the adaptive keyboard model being adapted with the current spatial location relationship obtained between the finger are carried out to the bond length of the surface keyboard model；The adaptive keyboard model is shown in the virtual reality scenario.The technical solution that the application provides, can realize the higher interactive process of precision.
Description
Technical field
The present invention relates to human-computer interaction technique field, the methods of exhibiting and device of more particularly to a kind of dummy keyboard.
Background technology
Currently, in human-computer interaction, user use information input mode can usually include it is following several：
1. conventional entity interacts, i.e., complete to input, such as mouse, the key being widely used by true physical equipment
Disk, game paddle, remote controler etc..The disadvantages of this method is：Entity interaction size is larger to be not used to micromodule equipment；Function restriction
Property is very big, can only solve a kind of input demand (for example mouse cannot be used on mobile phone) of equipment；Equipment size fixation can not fit
Answer different crowds (the larger people of hand thinks that keyboard spacing is small, and the small people of hand then otherwise).
2. wearable entity interaction is completed to input by sensor (acceleration sensor, gyroscope, Magnetic Sensor etc.).
Such as wearable bracelet, wearable ring, motion capture gloves etc..The disadvantages of this method is：Must dress could interact；Only
It can identify the simple motion information at wearing position.
3. entity that is, by touch screen, is invented interface element by half virtual interacting.Such as mobile phone, tablet, touch screen
Deng.The disadvantages of this method is：Interactive region depends on the size of equipment；Hand blocks interface during interaction
4. three-dimensional interacts, i.e., interacted in three dimensions with dummy object, such as Leap Motion, Kinect etc.,
Or the interactive expanding based on both equipment.The disadvantages of this method is：Hand cannot interact on the support surface, otherwise supporting surface quilt
Hand is mistaken for so as to malfunction.
5. surface virtual interacting, by being touch tablet desktop virtualization, so as to complete the interaction of similar touch tablet.The party
The defects of method is：Support surface is necessary smooth (cannot be uneven)；Accuracy of identification is relatively low, easily accidentally touches；None- identified finger
Press the dynamics on surface.
Therefore a kind of man-machine interaction method for needing accuracy of identification higher at present.
Invention content
In order to solve problem of the prior art, an embodiment of the present invention provides the methods of exhibiting and dress of a kind of dummy keyboard
It puts.The technical solution is as follows：
A kind of methods of exhibiting of dummy keyboard, the method includes：
The both hands of user and body surface are modeled respectively, to build the both hands mould of the user in virtual reality scenario
Type and surface keyboard model；
The finger model completion that will be blocked in the both hands model, to establish standard both hands model；
According to the current spatial location relationship between finger in the standard both hands model, to the surface keyboard model
Bond length carries out automatic adjusument, the adaptive keyboard mould being adapted with the current spatial location relationship obtained between the finger
Type；
The adaptive keyboard model is shown in the virtual reality scenario.
Further, before the finger model completion that will be blocked in the both hands model, the method further includes：
The both hands model is clustered, corresponding two clusters of finger fingertip in the both hands model are obtained, by institute
The right-hand man stated in both hands model distinguishes.
Further, cluster is carried out to the both hands model to include：
From the finger sample S={ x of the both hands model1,x2...,xnIn, these samples are divided into k cluster S1,
S2,...SkIn so that quadratic sum is minimum in satisfaction group, i.e.,：
Wherein t refers to iterations, miIt refers to the average point of ith cluster, is updated according to following two steps each
A cluster and its average point：
Step 1：
Wherein, xpIt refers to being assigned to determining clusterElement；
Step 2：On calculating in the cluster that walks, the barycenter of each cluster is as newly
Average point.
Further, the finger model completion being blocked in the both hands model is included：
Determine the feature vector of target finger in the both hands model, described eigenvector includes and the target finger
Associated ten two dimensional character；
Based on the feature vector of standard palm that advance training obtains, the feature vector for enumerating the standard palm is corresponding
The various situations of finger are lacked, and are calculated similar between the feature vector of the target finger and the various situations of missing finger
Degree, and using the situation of similarity maximum as the situation of current missing finger.
Further, calculate according to the following equation the feature vector of the target finger and missing finger various situations it
Between similarity：
Wherein, sim functions refer to the similarity of the two, illustrate that the two is more similar closer to 1, and n refers to feature quantity,
XiRefer to the ith feature of current sample, Xi' refer to the ith feature of master pattern.
Further, automatic adjusument is carried out to the bond length of the surface keyboard model to include：
Determine complex transformation matrix, the complex transformation matrix includes spin matrix, translation matrix and scaled matrix；
It is adaptive keyboard model by the surface keyboard model conversion using the complex transformation matrix；Wherein, it is described
Parameter in complex transformation matrix is true according to the position relationship between the adaptive keyboard model and the standard both hands model
It is fixed.
Further, the complex transformation matrix is determined according to the following equation：
Wherein, M represents the complex transformation matrix, and θ is rotation angle, TxAnd TyRepresent translation parameters, SxAnd SyRepresent contracting
Put parameter；
Correspondingly, the position relationship is determined according to the following equation：
min[dist(L5PA)+dist(L4PS)+dist(L3PD)+dist(L2PF)+dist(R2PJ)+dist(R3PK)+
dist(R4PL)]
Wherein, L5、L4、L3、L2Left hand little finger, the third finger, middle finger and food in the standard both hands model are represented respectively
Refer to corresponding coordinate, PA、PS、PD、PFThe coordinate of left hand positioning key in the adaptive keyboard model, R are represented respectively2、R3、R4Point
Right hand forefinger in the standard both hands model, middle finger and nameless corresponding coordinate, P are not representedJ、PK、PLDescribed in representing respectively
The coordinate of right hand positioning key in adaptive keyboard model, dist (LP) represent to calculate between finger coordinate and positioning key coordinate away from
From；
Wherein, the bond length in the adaptive keyboard model after adjusting is expressed as dist (PAPS)。
Further, after surface keyboard model is built, the method further includes：
The surface keyboard model is split as to the piecemeal keyboard of specified quantity, wherein, the piecemeal keyboard with it is described double
Finger in hand model is corresponding.
Further, the method further includes：
The action that user applies is received, and calls and acts the scene to match with described；Wherein, the action includes both hands
It is placed on simultaneously, a finger shakes according to specified path and at least two fingers simultaneously on the move at least one
Kind；Correspondingly, the scene includes at least one of keyboard scene, mouse scene and gesture scene.
A kind of displaying device of dummy keyboard, described device include：
Modeling unit, for being modeled respectively to the both hands of user and body surface, to build in virtual reality scenario
The both hands model of middle user and surface keyboard model；
Finger completion unit, for the finger model completion that will be blocked in the both hands model, to establish standard both hands
Model；
Automatic adjusument unit, for according to the current spatial location relationship between finger in the standard both hands model,
Automatic adjusument is carried out to the bond length of the surface keyboard model, to obtain the current spatial location relationship between the finger
The adaptive keyboard model being adapted；
Display unit, for showing the adaptive keyboard model in the virtual reality scenario.
Therefore technical scheme of the present invention is compared with traditional input mode, and it is more free and efficient, it can be in office
Expectation face (plane, curved surface etc.) inputs (word, character, number, gesture etc.), and advantageous effect is：
1. require no knowledge about the position of dummy keyboard (keyboard can be adaptive)；
2. the position of sensing positioning keyboard is not needed to (i.e. right-hand man's forefinger is individually placed on two buttons of F and J)；
It 3. can be according to the size adjust automatically keyboard size of hand；
4. body surface is supporting surface, user can be allowed to input for a long time；
5. automatic identification and switching interaction scenarios, such as input through keyboard scene, gesture input scene, mouse control scene
6. be the effective solution of virtual reality, during due to having on virtual implementing helmet, entity, therefore tradition can't see
Input (either physical keyboard or dummy keyboard) it is just no longer applicable, and by inputting and showing separation, realization passes through
Head display hand and keyboard, achieve the purpose that input, it is achieved thereby that the interactive process that precision is higher.
Description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is the flow chart of virtual interacting method in embodiment of the present invention；
Fig. 2 is the movement interval diagram of active body in embodiment of the present invention；
Fig. 3 is the structure diagram of active body in embodiment of the present invention；
Fig. 4 is point cloud data new in embodiment of the present invention and the point cloud data schematic diagram to disappear；
Fig. 5 is the interaction schematic diagram of active body Model and passive body Model in embodiment of the present invention；
Fig. 6 is the structure diagram of the displaying device of dummy keyboard in embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
Referring to Fig. 1, the application provides a kind of methods of exhibiting of dummy keyboard, the method includes：
S1：The both hands of user and body surface are modeled respectively, to build pair of the user in virtual reality scenario
Hand model and surface keyboard model；
S2：The finger model completion that will be blocked in the both hands model, to establish standard both hands model；
S3：According to the current spatial location relationship between finger in the standard both hands model, to the surface keyboard mould
The bond length of type carries out automatic adjusument, the adaptive key being adapted with the current spatial location relationship obtained between the finger
Disk model；
S4：The adaptive keyboard model is shown in the virtual reality scenario.
In the present embodiment, before the finger model completion that will be blocked in the both hands model, the method is also
Including：
The both hands model is clustered, corresponding two clusters of finger fingertip in the both hands model are obtained, by institute
The right-hand man stated in both hands model distinguishes.
In the present embodiment, cluster is carried out to the both hands model to include：
From the finger sample S={ x of the both hands model1,x2...,xnIn, these samples are divided into k cluster S1,
S2,...SkIn so that quadratic sum is minimum in satisfaction group, i.e.,：
Wherein t refers to iterations, miIt refers to the average point of ith cluster, is updated according to following two steps each
A cluster and its average point：
Step 1：
Wherein, xpIt refers to being assigned to determining clusterElement；
Step 2：On calculating in the cluster that walks, the barycenter of each cluster is as newly
Average point.
In the present embodiment, the finger model completion being blocked in the both hands model is included：
Determine the feature vector of target finger in the both hands model, described eigenvector includes and the target finger
Associated ten two dimensional character；
Based on the feature vector of standard palm that advance training obtains, the feature vector for enumerating the standard palm is corresponding
The various situations of finger are lacked, and are calculated similar between the feature vector of the target finger and the various situations of missing finger
Degree, and using the situation of similarity maximum as the situation of current missing finger.
In the present embodiment, the feature vector of the target finger is calculated according to the following equation with lacking the various of finger
Similarity between situation：
Wherein, sim functions refer to the similarity of the two, illustrate that the two is more similar closer to 1, and n refers to feature quantity,
XiRefer to the ith feature of current sample, Xi' refer to the ith feature of master pattern.
In the present embodiment, automatic adjusument is carried out to the bond length of the surface keyboard model to include：
Determine complex transformation matrix, the complex transformation matrix includes spin matrix, translation matrix and scaled matrix；
It is adaptive keyboard model by the surface keyboard model conversion using the complex transformation matrix；Wherein, it is described
Parameter in complex transformation matrix is true according to the position relationship between the adaptive keyboard model and the standard both hands model
It is fixed.
In the present embodiment, the complex transformation matrix is determined according to the following equation：
Wherein, M represents the complex transformation matrix, and θ is rotation angle, TxAnd TyRepresent translation parameters, SxAnd SyRepresent contracting
Put parameter；
Correspondingly, the position relationship is determined according to the following equation：
min[dist(L5PA)+dist(L4PS)+dist(L3PD)+dist(L2PF)+dist(R2PJ)+dist(R3PK)+
dist(R4PL)]
Wherein, L5、L4、L3、L2Left hand little finger, the third finger, middle finger and food in the standard both hands model are represented respectively
Refer to corresponding coordinate, PA、PS、PD、PFThe coordinate of left hand positioning key in the adaptive keyboard model, R are represented respectively2、R3、R4Point
Right hand forefinger in the standard both hands model, middle finger and nameless corresponding coordinate, P are not representedJ、PK、PLDescribed in representing respectively
The coordinate of right hand positioning key in adaptive keyboard model, dist (LP) represent to calculate between finger coordinate and positioning key coordinate away from
From；
Wherein, the bond length in the adaptive keyboard model after adjusting is expressed as dist (PAPS)。
In the present embodiment, after surface keyboard model is built, the method further includes：
The surface keyboard model is split as to the piecemeal keyboard of specified quantity, wherein, the piecemeal keyboard with it is described double
Finger in hand model is corresponding.
In the present embodiment, the method further includes：
The action that user applies is received, and calls and acts the scene to match with described；Wherein, the action includes both hands
It is placed on simultaneously, a finger shakes according to specified path and at least two fingers simultaneously on the move at least one
Kind；Correspondingly, the scene includes at least one of keyboard scene, mouse scene and gesture scene.
In practical application scene, the present invention obtains the both hands of people and the threedimensional model on surface by optical sensor, double
Subordinate side does not have actual keyboard, touch screen or display, but in the display can simultaneous display dummy keyboard and virtual both hands mould
The distance of type (such as Fig. 2), hand and dummy keyboard is exactly the distance of hand and desktop in reality.Since hand model is with hand in reality
Movement and move, people when in use, has feeling of immersion, and the physical feedback for tapping surface further increases its feeling of immersion, allows people
Seem to feel that there is true keyboard in lower section in one's hands.
In the present embodiment, the position of every finger is obtained by sensor first, binocular vision, structure can be passed through
Three-dimensionals/the two dimensional techniques such as light, hash hot spot, radar, ultrasonic wave, laser scanning, a word laser obtain the position of finger.It is assuming that left
Hand thumb refers to tail：L={ L1,L2,L3,L4,L5, the right hand refers to from thumb to tail is respectively：R={ R1,R2,R3,R4,
R5, the technology that Freely input needs are used is described below.
1. keyboard is adaptive
When both hands are placed in desktop, due to there is no keyboard, although user can see void by display
Intend the relative position of keyboard and both hands, but due to the Boundary Feedback of not true keyboard, the input posture of user and position meeting
Unconscious change by the adaptive technology of keyboard, can improve user's input efficiency and input is experienced.Specifically, keyboard
Adaptive technique includes following aspect：
1) right-hand man is distinguished：
It is 2 clusters finger fingertip cluster, for finger tip sample { x by clustering algorithm(1),...,x(m)},x(m)∈R3,
Specifically carry out as follows：
Randomly select 2 cluster center of mass point μ1、μ2；
Procedure below is repeated until convergence：
For each sample i, calculate which class it should belong to
c(i):=argmin | | x(i)-μj||2
For each class j, such barycenter is recalculated
2) finger completion：
It is not identified assuming that there is several fingers because blocking, the finger three-dimensional information blocked is needed this when to predict
Out, the palm simplified model of standard as shown in Figure 3 is established.Due to the limitation of human hand bone and skin, model should also have
Suitable qualifications, condition include：Different fingers are defined to x ∈ [α respectively along x, y, z-axis rotation anglemin,αmax], y ∈
[βmin,βmax], z ∈ [γmin,γmax], each user of angle of restriction is different, needs to obtain when user calibrates；
Each finger-joint length is fixed, but can differ；
Arthrogryposis direction be defined to palm of the hand direction and
Finger corresponds to finger tip and is connected, and sequentially remains unchanged clockwise；
Finger corresponding joint is connected, and sequentially remains unchanged clockwise.
Specifically, feature vector, X=(x of finger can be taken1,x2,...,x12) it is following feature：
Joint length；
Joint number；
First arthrogryposis degree；
Second joint bending degree；
Third arthrogryposis degree；
Finger and left adjacent Space Angle；
Finger faces Space Angle with the right side；
Palm and x-axis Space Angle；
Palm and y-axis Space Angle；
Palm and z-axis Space Angle；
Palm space coordinate；
Palm and previous Frame Properties deviation.
In addition, by being trained to great amount of samples, feature vector, X '=(x' of standard palm simplified model is obtained1,x
‘2,...,x’12), enumerate all possibilities of missing finger, by KNN algorithms calculate each possibility and standard feature to
The similarity of amount, highest similarity is best possibility：
2. calculation key is away from (fixed keyboard)
When user needs to breathe out keyboard typing, both hands is needed to put on the table, at this point, passing through the space between finger
Position relationship can acquire best keyboard key away from.Assuming that four angular coordinates of keyboard are respectively：Pc1,Pc2,Pc3,Pc4, left hand is fixed
Position key coordinate be respectively：PA,PS,PD,PF, corresponding finger is L5,L4,L3,L2, the right hand elements of a fix are respectively：PJ,PK,PL, it is corresponding
Finger be R2,R3,R4。
In order to simplify operation, it is believed that input face is approximate plane, since QWERTY keyboard is by appropriate rotation, is put down
It moves, scaling can obtain new keyboard, enable spin matrix：
Translation matrix
Scaled matrix
If complex transformation is
Step-length is set, is acquired by iteration and converges to following relationship：
min[dist(L5PA)+dist(L4PS)+dist(L3PD)+dist(L2PF)+dist(R2PJ)+dist(R3PK)+
dist(R4PL)]
Then dist (PAPS) it is bond length.
3. it is laid out adaptive-piecemeal keyboard
The fixed most apparent advantage of keyboard is to be easily adapted to, but do not meet the design of ergonomics, because of different hands
Refer to different in size, and the corresponding standard key position of finger is that spacing is consistent, be easy to cause certain fingers bend and stretch and its
Its finger is inconsistent, and tired degree is caused to have notable difference.Therefore set forth herein a kind of concept of piecemeal keyboard, that is, button cloth is allowed
The suitable ergonomics of office.
Each piece of piecemeal keyboard and different fingers are bound, and are divided into 8 pieces altogether, such as Fig. 4.When finger position changes, key
Disk is also with change.
4. scene automatically switches
Since Freely input has very strong autgmentability, as virtual scene is more and more, switching will become increasingly
Complexity, therefore present invention introduces a kind of mechanism of automatic switchover, and just this can be used without active handoff scenario to reach user
The purpose of scape function.
Referring to Fig. 5, when user wants to input word, it is only necessary to which both hands are past to be put down naturally, you can triggers the field of button
Scape, it is desirable that be：After being modeled to palm, it is necessary to which the model of the keyboard of the exhalation shown in Fig. 5 can be constructed by meeting, and meet its restriction
Condition：Two frame finger tips correspond to finger space and attitudes vibration no more than threshold value.In addition, when one finger of user is when shaking
It waits, is judged by identifying that finger shakes rule, mouse scene can be switched to；When user refers to movement more, hand is automatically switched to
Potential field scape.
Referring to Fig. 6, the application also provides a kind of displaying device of dummy keyboard, described device includes：
Modeling unit 100, for being modeled respectively to the both hands of user and body surface, to build in virtual reality field
The both hands model of user and surface keyboard model in scape；
Finger completion unit 200, it is double to establish standard for the finger model completion that will be blocked in the both hands model
Hand model；
Automatic adjusument unit 300, for being closed according to the current spatial location between finger in the standard both hands model
System carries out automatic adjusument, to obtain the current spatial location between the finger to the bond length of the surface keyboard model
The adaptive keyboard model that relationship is adapted；
Display unit 400, for showing the adaptive keyboard model in the virtual reality scenario.
Therefore technical scheme of the present invention is compared with traditional input mode, and it is more free and efficient, it can be in office
Expectation face (plane, curved surface etc.) inputs (word, character, number, gesture etc.), and advantageous effect is：
1. require no knowledge about the position of dummy keyboard (keyboard can be adaptive)；
2. the position of sensing positioning keyboard is not needed to (i.e. right-hand man's forefinger is individually placed on two buttons of F and J)；
It 3. can be according to the size adjust automatically keyboard size of hand；
4. body surface is supporting surface, user can be allowed to input for a long time；
5. automatic identification and switching interaction scenarios, such as input through keyboard scene, gesture input scene, mouse control scene
6. be the effective solution of virtual reality, during due to having on virtual implementing helmet, entity, therefore tradition can't see
Input (either physical keyboard or dummy keyboard) it is just no longer applicable, and by inputting and showing separation, realization passes through
Head display hand and keyboard, achieve the purpose that input, it is achieved thereby that the interactive process that precision is higher.
The apparatus embodiments described above are merely exemplary, wherein the unit illustrated as separating component can
To be or may not be physically separate, the component shown as unit may or may not be physics list
Member, you can be located at a place or can also be distributed in multiple network element.It can be selected according to the actual needs
In some or all of module realize the purpose of this embodiment scheme.Those of ordinary skill in the art are not paying creativeness
Labour in the case of, you can to understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It is realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on such understanding, on
Technical solution is stated substantially in other words to embody the part that the prior art contributes in the form of software product, it should
Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including several fingers
It enables and (can be personal computer, server or the network equipment etc.) so that computer equipment is used to perform each implementation
Method described in certain parts of example or embodiment.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.
Claims (9)
1. a kind of methods of exhibiting of dummy keyboard, which is characterized in that the method includes：
The both hands of user and body surface are modeled respectively, with build in virtual reality scenario user both hands model with
And surface keyboard model；
The finger model completion that will be blocked in the both hands model, to establish standard both hands model；
According to the current spatial location relationship between finger in the standard both hands model, to the bond length of the surface keyboard model
Carry out automatic adjusument, the adaptive keyboard model being adapted with the current spatial location relationship obtained between the finger；
The adaptive keyboard model is shown in the virtual reality scenario；
The finger model completion being blocked in the both hands model is included：
Determine the feature vector of target finger in the both hands model, described eigenvector includes related to the target finger
Ten two dimensional characters of connection；
Based on the feature vector of standard palm that advance training obtains, the corresponding missing of feature vector of the standard palm is enumerated
The various situations of finger, and the similarity between the feature vector of the target finger and the various situations for lacking finger is calculated,
And using the situation of similarity maximum as the situation of current missing finger.
2. it according to the method described in claim 1, it is characterized in that, is mended in the finger model that will be blocked in the both hands model
Before complete, the method further includes：
The both hands model is clustered, obtains corresponding two clusters of finger fingertip in the both hands model, it will be described double
Right-hand man in hand model distinguishes.
3. according to the method described in claim 2, include it is characterized in that, carrying out cluster to the both hands model：
From the finger sample S={ x of the both hands model1,x2...,xnIn, these samples are divided into k cluster S1,S2,
...SkIn so that quadratic sum is minimum in satisfaction group, i.e.,：
Wherein t refers to iterations, miIt refers to the average point of ith cluster, updates each cluster according to following two steps
And its average point：
Step 1：
Wherein, xpIt refers to being assigned to determining clusterElement；
Step 2：In the cluster walked on calculating, the barycenter of each cluster is as new mean value
Point.
4. according to the method described in claim 1, it is characterized in that, calculate according to the following equation the feature of the target finger to
Similarity between amount and the various situations of missing finger：
Wherein, sim functions refer to the similarity of the two, illustrate that the two is more similar closer to 1, n refers to feature quantity, XiRefer to
Be current sample ith feature, Xi' refer to the ith feature of master pattern.
5. according to the method described in claim 1, it is characterized in that, the bond length of the surface keyboard model is adaptively adjusted
Section includes：
Determine complex transformation matrix, the complex transformation matrix includes spin matrix, translation matrix and scaled matrix；
It is adaptive keyboard model by the surface keyboard model conversion using the complex transformation matrix；Wherein, it is described compound
Parameter in transformation matrix is determined according to the position relationship between the adaptive keyboard model and the standard both hands model.
6. according to the method described in claim 5, it is characterized in that, the complex transformation matrix is determined according to the following equation：
Wherein, M represents the complex transformation matrix, and θ is rotation angle, TxAnd TyRepresent translation parameters, SxAnd SyRepresent scaling ginseng
Number；
Correspondingly, the position relationship is determined according to the following equation：
min[dist(L5PA)+dist(L4PS)+dist(L3PD)+dist(L2PF)+dist(R2PJ)+dist(R3PK)+dist
(R4PL)]
Wherein, L5、L4、L3、L2Left hand little finger, the third finger, middle finger and forefinger pair in the standard both hands model are represented respectively
The coordinate answered, PA、PS、PD、PFThe coordinate of left hand positioning key in the adaptive keyboard model, R are represented respectively2、R3、R4Table respectively
Show right hand forefinger in the standard both hands model, middle finger and nameless corresponding coordinate, PJ、PK、PLRepresent described adaptive respectively
The coordinate of right hand positioning key in keyboard model is answered, dist (LP) represents to calculate the distance between finger coordinate and positioning key coordinate；
Wherein, the bond length in the adaptive keyboard model after adjusting is expressed as dist (PAPS)。
7. according to the method described in claim 1, it is characterized in that, after surface keyboard model is built, the method is also wrapped
It includes：
The surface keyboard model is split as to the piecemeal keyboard of specified quantity, wherein, the piecemeal keyboard and the both hands mould
Finger in type is corresponding.
8. according to the method described in claim 1, it is characterized in that, the method further includes：
The action that user applies is received, and calls and acts the scene to match with described；Wherein, the action includes both hands simultaneously
It is placed on, a finger shakes according to specified path and at least two fingers at least one on the move simultaneously；Phase
Ying Di, the scene include at least one of keyboard scene, mouse scene and gesture scene.
9. the displaying device of a kind of dummy keyboard, which is characterized in that described device includes：
Modeling unit for being modeled respectively to the both hands of user and body surface, is used with structure in virtual reality scenario
The both hands model at family and surface keyboard model；
Finger completion unit, for the finger model completion that will be blocked in the both hands model, to establish standard both hands model；
Automatic adjusument unit, for according to the current spatial location relationship between finger in the standard both hands model, to institute
The bond length for stating surface keyboard model carries out automatic adjusument, is mutually fitted with the current spatial location relationship obtained between the finger
The adaptive keyboard model matched；
Display unit, for showing the adaptive keyboard model in the virtual reality scenario；
The finger model completion being blocked in the both hands model is included：
Determine the feature vector of target finger in the both hands model, described eigenvector includes related to the target finger
Ten two dimensional characters of connection；
Based on the feature vector of standard palm that advance training obtains, the corresponding missing of feature vector of the standard palm is enumerated
The various situations of finger, and the similarity between the feature vector of the target finger and the various situations for lacking finger is calculated,
And using the situation of similarity maximum as the situation of current missing finger.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710709726.6A CN107633551B (en) | 2017-08-18 | 2017-08-18 | The methods of exhibiting and device of a kind of dummy keyboard |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710709726.6A CN107633551B (en) | 2017-08-18 | 2017-08-18 | The methods of exhibiting and device of a kind of dummy keyboard |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107633551A CN107633551A (en) | 2018-01-26 |
| CN107633551B true CN107633551B (en) | 2018-07-06 |
Family
ID=61100256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710709726.6A Active CN107633551B (en) | 2017-08-18 | 2017-08-18 | The methods of exhibiting and device of a kind of dummy keyboard |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107633551B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108646910A (en) * | 2018-03-20 | 2018-10-12 | 重庆邮电大学 | A kind of Three-Dimensional Dynamic finger text input system and method based on depth image |
| CN110472396B (en) * | 2018-08-17 | 2022-12-30 | 中山叶浪智能科技有限责任公司 | Somatosensory gesture touch method, system, platform and storage medium |
| CN115904056A (en) * | 2021-08-09 | 2023-04-04 | 华为技术有限公司 | Display method of virtual input element, electronic device and readable storage medium |
| CN117827016A (en) * | 2024-03-06 | 2024-04-05 | 深圳市黑爵同创电子科技有限公司 | Key cap arrangement method of ergonomic keyboard |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4930100B2 (en) * | 2007-02-27 | 2012-05-09 | ソニー株式会社 | Force / tactile display, force / tactile display control method, and computer program |
| CN102541420A (en) * | 2010-12-31 | 2012-07-04 | 上海优熠电子科技有限公司 | Self-adaption virtual keyboard system |
| CN102693025B (en) * | 2011-03-21 | 2015-07-08 | 中国科学院软件研究所 | Touch finger identification method for multi-touch interaction system |
| CN102778951B (en) * | 2012-06-15 | 2016-02-10 | 惠州华阳通用电子有限公司 | Use input equipment and the input method of virtual key |
| JP2015535628A (en) * | 2012-11-27 | 2015-12-14 | トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ | Adaptive virtual keyboard |
| CN103186241B (en) * | 2013-04-03 | 2016-07-06 | 中国科学院软件研究所 | A kind of interactive desktop contact right-hand man's recognition methods |
| CN105224069B (en) * | 2014-07-03 | 2019-03-19 | 王登高 | A kind of augmented reality dummy keyboard input method and the device using this method |
-
2017
- 2017-08-18 CN CN201710709726.6A patent/CN107633551B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN107633551A (en) | 2018-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11983326B2 (en) | Hand gesture input for wearable system | |
| CN108052202B (en) | 3D interaction method and device, computer equipment and storage medium | |
| Wacker et al. | Arpen: Mid-air object manipulation techniques for a bimanual ar system with pen & smartphone | |
| CN107633551B (en) | The methods of exhibiting and device of a kind of dummy keyboard | |
| EP3090331B1 (en) | Systems with techniques for user interface control | |
| JP2020528597A (en) | Neural network training using user interface device representations | |
| Chaconas et al. | An evaluation of bimanual gestures on the microsoft hololens | |
| KR101318244B1 (en) | System and Method for Implemeting 3-Dimensional User Interface | |
| Matulic et al. | Phonetroller: Visual representations of fingers for precise touch input with mobile phones in vr | |
| EP4307096A1 (en) | Key function execution method, apparatus and device, and storage medium | |
| Caputo et al. | The Smart Pin: An effective tool for object manipulation in immersive virtual reality environments | |
| Zhao et al. | Comparing head gesture, hand gesture and gamepad interfaces for answering Yes/No questions in virtual environments | |
| Raees et al. | VEN-3DVE: vision based egocentric navigation for 3D virtual environments | |
| Chun et al. | A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment | |
| Segen et al. | Look ma, no mouse! | |
| Choi et al. | 3D hand pose estimation on conventional capacitive touchscreens | |
| CN109960404A (en) | A kind of data processing method and device | |
| Tran et al. | Easy-to-use virtual brick manipulation techniques using hand gestures | |
| Cabric et al. | Touchglass: Raycasting from a glass surface to point at physical objects in public exhibits | |
| Gil et al. | ThumbAir: In-Air Typing for Head Mounted Displays | |
| Liu et al. | COMTIS: Customizable touchless interaction system for large screen visualization | |
| Caputo | Gestural interaction in virtual environments: User studies and applications | |
| Mahdikhanlou et al. | Object manipulation and deformation using hand gestures | |
| CN108415571A (en) | A kind of somatosensory device implementation method moving caused data analysis based on thumb | |
| CN116185205B (en) | Non-contact gesture interaction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |