CN109240508B - Method for restoring finger motion by hand back skin deformation - Google Patents
Method for restoring finger motion by hand back skin deformation Download PDFInfo
- Publication number
- CN109240508B CN109240508B CN201811159627.6A CN201811159627A CN109240508B CN 109240508 B CN109240508 B CN 109240508B CN 201811159627 A CN201811159627 A CN 201811159627A CN 109240508 B CN109240508 B CN 109240508B
- Authority
- CN
- China
- Prior art keywords
- finger
- ultraviolet fluorescent
- hand
- fluorescent ink
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
Abstract
The invention discloses a method for restoring finger motion by skin deformation of a hand back, which belongs to the field of man-machine interaction and comprises the following specific steps: (1) making marks on the back of the hand, and wearing a camera with a function of acquiring mark changes on the wrist; (2) the optitrack camera is worn on the finger of a person; (3) acquiring a buckling curve of each finger through an optitrack camera, and acquiring the change of a mark on the back of a hand through a camera on the wrist; (4) comparing the buckling curve obtained in the step (3) with the change of the identification to obtain the relation between the buckling curve and the identification; (5) and (3) taking down the optitrack camera worn on the finger of the person, and directly acquiring the motion change of the finger through the mark on the back of the hand by using the data acquired in the step (4).
Description
Technical Field
The invention relates to the field of human-computer interaction, in particular to a method for restoring finger motion by skin deformation of a hand back.
Background
The abundance of computer resources and low power consumption make electronic devices more portable and ubiquitous. Therefore, there is an increasing interest in novel human-computer interaction models that go beyond traditional input strategies such as keyboards, mice and touch screens. However, the limited interaction space (e.g., small screen) impairs the user experience and prevents the portable device from functioning adequately.
The scheme aims at the finger motion recognition technology, for example, data gloves are adopted at present, the buckling error of a proximal phalanx of the data gloves is about 5 degrees, but the accuracy of the data gloves needs to be at the cost of comfort and convenience, and other solutions such as a commercial depth camera, a wrist-wearing camera and a head-wearing camera are adopted, but a finger motion detection system based on the camera can face the shielding problem, high power consumption and light limitation. To address these problems directly related to finger movement, researchers have used sensing techniques and different signals to the relevant area, such as the wrist, including capacitive sensing, electromyographic sensing (EMG), and pressure sensing. However, these are too noisy and can only support a small set of discrete gestures.
In view of the problems in the prior art, the present professional design a method for restoring finger movement by skin deformation on the back of the hand with the assistance of too strong professional knowledge to overcome the above-mentioned drawbacks.
Disclosure of Invention
For the problems in the prior art, the method for restoring the finger motion by the skin deformation of the back of the hand can measure the skin stretching of the back of the hand, and achieves the purpose of detecting the position and the movement of the finger.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows: the method for restoring finger motion by hand back skin deformation comprises the following specific steps:
(1) making marks on the back of the hand, and wearing a camera with a function of acquiring mark changes on the wrist;
(2) the optitrack camera is worn on the finger of a person;
(3) acquiring a buckling curve of each finger through an optitrack camera, and acquiring the change of a mark on the back of a hand through a camera on the wrist;
(4) comparing the buckling curve obtained in the step (3) with the change of the identification to obtain the relation between the buckling curve and the identification;
(5) and (3) taking down the optitrack camera worn on the finger of the person, and directly acquiring the motion change of the finger through the mark on the back of the hand by using the data acquired in the step (4).
As a preferential technical scheme, the marks are a plurality of ultraviolet fluorescent oil patterns, and the camera is a bracelet which can be worn on the wrist of a human hand and has an ultraviolet camera function.
As a preferential technical scheme, the ultraviolet fluorescent oil pattern is a plurality of ultraviolet fluorescent ink dots, and at least two ultraviolet fluorescent ink dots are arranged on the back of the hand corresponding to each finger;
as a preferred technical scheme, the ultraviolet fluorescent ink pattern is an ultraviolet fluorescent ink line, and the ultraviolet fluorescent ink line is perpendicular to each finger.
As a preferred technical scheme, the ultraviolet fluorescent ink pattern is a plurality of ultraviolet fluorescent ink lines, and each ultraviolet fluorescent ink line is arranged in the opposite direction of the corresponding finger.
As a preferred technical solution, the step (3) needs to collect and count data for multiple times.
As a preferred technical solution, when the marks are uv fluorescent ink dots, the degree of flexion of the fingers is in a linear relationship with a change in distance between two uv fluorescent ink dots corresponding to each finger.
As a preferred technical solution, when the mark is an ultraviolet fluorescent ink line, the change of the angle at the intersection of the finger opposite direction and the ultraviolet fluorescent ink line has a linear relationship with the degree of flexion of the finger.
As a preferred technical solution, when the mark is a plurality of uv fluorescent ink lines, the average width of each uv fluorescent ink line is linearly related to the degree of flexion of the corresponding finger.
The invention has the advantages that:
(1) according to the invention, a brand-new man-machine interaction method is completed by researching the deformation of the skin on the back of the hand and the buckling change rule of the fingers, and the method is comfortable and convenient, and the camera can not be shielded by the fingers.
(2) The invention utilizes the ultraviolet fluorescent ink as the mark, on one hand, the ultraviolet fluorescent ink can be well combined with the camera and has small interference, and on the other hand, the ultraviolet fluorescent ink can not cause discomfort to hands.
(3) According to the invention, the actual data is firstly acquired by the optitrack camera and compared with the finger movement generated by the deformation of the skin on the back of the hand, so that the correctness of the invention is verified. Of course, the invention can be used for acquiring the human-computer interaction only through the finger movement generated by the deformation of the skin on the back of the hand at the later stage.
(4) A series of experiments prove that the deformation error of the finger curvature and the skin on the back of the hand is about 3-7 degrees, and the requirements of man-machine interaction application are completely met.
Drawings
FIG. 1 is a diagram of the practical use of the method for restoring finger motion by skin deformation of the back of the hand;
FIG. 2 is a diagram of an ultraviolet camera in a method of hand back skin deformation reduction finger movement;
FIG. 3 is a diagram of the theoretical operation of the method for restoring finger movement by skin deformation of the back of the hand;
fig. 4 is a simulated three-dimensional coordinate system established by optitrack on the back of the hand in the method for restoring finger motion by skin deformation of the back of the hand.
Detailed Description
In order to facilitate understanding for those skilled in the art, the present invention will be further described with reference to the accompanying drawings.
It is well known that human movement is driven by connected bones and muscles. This will result in skin stretching, and different human movements will cause different skin movement patterns, and it is therefore possible to recombine human movements from the pattern of skin stretching. Researchers have explored this approach well in facial recognition to detect the user's facial expressions. The human-computer interaction of the human fingers is different from facial recognition and is more intuitive, and the action range and the degree of freedom of the fingers are incomparable with the face.
The method for restoring the finger movement by the skin deformation of the back of the hand is specifically described as follows, and comprises the following specific steps:
(1) making marks on the back of the hand, and wearing a camera with a function of acquiring mark changes on the wrist;
(2) the optitrack camera is worn on the finger of a person;
(3) acquiring a buckling curve of each finger through an optitrack camera, and acquiring the change of a mark on the back of a hand through a camera on the wrist;
(4) comparing the buckling curve obtained in the step (3) with the change of the identification to obtain the relation between the buckling curve and the identification;
(5) and (3) taking down the optitrack camera worn on the finger of the person, and directly acquiring the motion change of the finger through the mark on the back of the hand by using the data acquired in the step (4).
The mark is preferably a plurality of ultraviolet fluorescent oil patterns, the camera is a bracelet with an ultraviolet camera function and capable of being worn on the wrist of a human hand, so that the carrying problem of the camera can be solved, the collection of mark information is not influenced when the mark is worn on the wrist, discomfort is not felt, and as shown in figure 1, figure 2 shows the content of the ultraviolet fluorescent ink dots in figure 1 acquired by the ultraviolet camera.
Furthermore, the ultraviolet fluorescent oil pattern is a plurality of ultraviolet fluorescent ink dots, at least two ultraviolet fluorescent ink dots are arranged on the back of the hand corresponding to each finger, and the bending degree of the finger and the change of the distance between the two ultraviolet fluorescent ink dots corresponding to each finger are found to be in a linear relation through experiments;
or the ultraviolet fluorescent ink pattern is an ultraviolet fluorescent ink line which is perpendicular to each finger, and similarly, the change of the angle at the intersection of the finger opposite direction and the ultraviolet fluorescent ink line is in linear relation with the bending degree of the finger.
Or, the ultraviolet fluorescent ink pattern is a plurality of ultraviolet fluorescent ink lines, each ultraviolet fluorescent ink line is arranged in the opposite direction of the corresponding finger, and the average width of each ultraviolet fluorescent ink line is in linear relation with the bending degree of the corresponding finger.
Of course, the step (3) of the invention needs to collect statistics for multiple times.
In order to further verify the feasibility of the invention, the verification is carried out by specific experimental data.
12 participants (6 females, 6 males) were enrolled to participate in the experiment, ranging from 20 to 34 years of age, and a full set of finger gestures was designed, with a 4 x 4 uv-phosphor ink dot matrix on the back of the hand for convenience of data accuracy, as shown in figure 3,
the method comprises the steps of establishing a simulated three-dimensional coordinate system on the back of a hand, taking the back of the hand as one reference surface, and enabling a normal line to be perpendicular to the back of the hand, wherein it is necessary to emphasize that a thumb needs to be established independently to form a thumb coordinate system, and the specifically established coordinate system is shown in fig. 4, is convenient for an OptiTrack camera to acquire three-dimensional coordinates and finger flexion of a finger, and is operated on line by utilizing OptiTrack motion software (running on a personal computer (Intel i7-4765T @2.0GHz, 8G RAM)).
The relevant data errors were obtained, and in the table, the flexion range refers to the maximum degree of flexion of the proximal phalanx of each finger averaged over 12 participants, the maximum error is in the posterior parenthesis, and the extension distance refers to the maximum distance of the uv-phosphor ink dot on the back of the hand closest to the finger of each finger averaged over 12 participants from the uv-phosphor ink dot that follows, the maximum error is in the posterior parenthesis.
| Flexion range average (Standard) | Extended distance average (Standard) | |
| Thumb (thumb) | 59.47°(10.41°) | 1.52(0.63)mm |
| Index finger | 103.84°(6.19°) | 2.72(1.23)mm |
| Middle finger | 112.35°(7.71°) | 2.34(1.44)mm |
| Ring finger | 115.00°(8.72°) | 1.88(0.96)mm |
| Little finger | 121.35°(13.71°) | 2.77(1.12)mm |
And then, data errors between the stretching angles of the skin of the back of the hand and the bending angles of the fingers are analyzed, and under the same stretching distance, the average error of the thumb reaches 3.77 degrees, the index finger 5.87 degrees, the middle finger 6.43 degrees, the ring finger 4.5 degrees and the little finger 6.22 degrees.
In conclusion, the error basically reaches the error range of the traditional data glove, namely, the human-computer interaction application can be satisfied.
It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Furthermore, it should be understood that various changes, modifications and/or alterations can be made to the present invention by those skilled in the art after reading the technical disclosure of the present invention, for example, whether the head is swung can be judged by stretching the skin on the neck, the limb movement can be judged by stretching the skin on the back, and the like, and all such equivalents fall within the scope of the present invention as defined by the appended claims.
Claims (9)
1. The method for restoring the skin deformation of the back of the hand to the finger movement is characterized by comprising the following specific steps of:
(1) making marks on the back of the hand, and wearing a camera with a function of acquiring mark changes on the wrist;
(2) the optitrack camera is worn on the finger of a person;
(3) acquiring a buckling curve of each finger through an optitrack camera, and acquiring the change of a mark on the back of a hand through a camera on the wrist;
(4) comparing the buckling curve obtained in the step (3) with the change of the identification to obtain the relation between the buckling curve and the identification;
(5) and (4) taking down the optitrack camera worn on the finger of the person, and directly acquiring the motion change of the finger through the mark on the back of the hand by using the data of the connection between the buckling curve acquired in the step (4) and the change of the mark.
2. The method for skin deformation reduction of finger movement on back of hand according to claim 1, wherein: the identification is a plurality of ultraviolet fluorescent oil patterns, and the camera is designed as a bracelet with an ultraviolet camera function and capable of being worn on the wrist of a human hand.
3. The method for skin deformation reduction of finger movement on back of hand according to claim 2, wherein: the ultraviolet fluorescent oil pattern is a plurality of ultraviolet fluorescent ink dots, and at least two ultraviolet fluorescent ink dots are arranged on the back of the hand corresponding to each finger.
4. The method for skin deformation reduction of finger movement on back of hand according to claim 2, wherein: the ultraviolet fluorescent ink pattern is an ultraviolet fluorescent ink line, and the ultraviolet fluorescent ink line is perpendicular to each finger.
5. The method for skin deformation reduction of finger movement on back of hand according to claim 2, wherein: the ultraviolet fluorescent ink patterns are a plurality of ultraviolet fluorescent ink lines, and each ultraviolet fluorescent ink line is arranged in the opposite direction of the corresponding finger.
6. The method for skin deformation reduction of finger movement on back of hand according to claim 1, wherein: and (3) collecting and counting data for multiple times.
7. The method for skin deformation reduction of finger movement on back of hand according to claim 3, wherein: when the marks are ultraviolet fluorescent ink dots, the bending degree of the fingers is in a linear relation with the change of the distance between the two ultraviolet fluorescent ink dots corresponding to each finger.
8. The method for skin deformation reduction of finger movement on back of hand according to claim 4, wherein: when the mark is an ultraviolet fluorescent ink line, the change of the angle of the intersection of the reverse direction of the finger and the ultraviolet fluorescent ink line is in a linear relation with the bending degree of the finger.
9. The method for skin deformation reduction of finger movement on back of hand according to claim 5, wherein: when the mark is a plurality of ultraviolet fluorescent ink lines, the average width of each ultraviolet fluorescent ink line is in linear relation with the bending degree of the corresponding finger.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811159627.6A CN109240508B (en) | 2018-09-30 | 2018-09-30 | Method for restoring finger motion by hand back skin deformation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811159627.6A CN109240508B (en) | 2018-09-30 | 2018-09-30 | Method for restoring finger motion by hand back skin deformation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109240508A CN109240508A (en) | 2019-01-18 |
| CN109240508B true CN109240508B (en) | 2020-01-24 |
Family
ID=65054894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811159627.6A Active CN109240508B (en) | 2018-09-30 | 2018-09-30 | Method for restoring finger motion by hand back skin deformation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109240508B (en) |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2775934C (en) * | 2005-02-16 | 2013-10-29 | Orica Explosives Technology Pty Ltd | Blasting methods and apparatus with reduced risk of inadvertent or illicit use |
| CN103777752A (en) * | 2013-11-02 | 2014-05-07 | 上海威璞电子科技有限公司 | Gesture recognition device based on arm muscle current detection and motion sensor |
| EP3102996B1 (en) * | 2014-02-06 | 2019-12-04 | Sony Corporation | Device and method for detecting gestures on the skin |
| CN103853333A (en) * | 2014-03-21 | 2014-06-11 | 上海威璞电子科技有限公司 | Gesture control scheme for toy |
| CN105094338A (en) * | 2015-08-19 | 2015-11-25 | 联想(北京)有限公司 | Control method, controller and electronic equipment |
| CN105824431B (en) * | 2016-06-12 | 2018-12-04 | 齐向前 | Message input device and method |
| CN206378818U (en) * | 2017-01-22 | 2017-08-04 | 无锡吾成互联科技有限公司 | A kind of Hand gesture detection device based on wireless self-networking pattern |
| CN107765850A (en) * | 2017-09-22 | 2018-03-06 | 上海交通大学 | A kind of sign Language Recognition based on electronic skin and multi-sensor fusion |
-
2018
- 2018-09-30 CN CN201811159627.6A patent/CN109240508B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109240508A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Guo et al. | Human-machine interaction sensing technology based on hand gesture recognition: A review | |
| Lin et al. | Backhand: Sensing hand gestures via back of the hand | |
| US6861945B2 (en) | Information input device, information processing device and information input method | |
| CN112789577B (en) | Neuromuscular text input, writing and drawing in augmented reality systems | |
| CN103315744B (en) | Hand tremor detection method | |
| Jiang et al. | Stretchable e-skin patch for gesture recognition on the back of the hand | |
| EP3203350A1 (en) | Wearable controller for wrist | |
| CN103440101B (en) | Processing method, system and the mobile phone of a kind of original handwriting data | |
| US20060028457A1 (en) | Stylus-Based Computer Input System | |
| CN105022471A (en) | Device and method for carrying out gesture recognition based on pressure sensor array | |
| Zhang et al. | Recognizing hand gestures with pressure-sensor-based motion sensing | |
| CN103677289A (en) | Intelligent interactive glove and interactive method | |
| CN110618754B (en) | Surface electromyogram signal-based gesture recognition method and gesture recognition armband | |
| CN103605433B (en) | A kind of Multifunctional somatological input device | |
| CN114153317A (en) | Information processing method, device and equipment based on electromyographic signals and storage medium | |
| CN109240508B (en) | Method for restoring finger motion by hand back skin deformation | |
| Blumrosen et al. | Back to finger-writing: Fingertip writing technology based on pressure sensing | |
| CN106020442A (en) | Sensing method for intelligent sensing glove | |
| CN109409236B (en) | Three-dimensional static gesture recognition method and device | |
| Widodo et al. | Design and evaluation of upper-arm mouse using inertial sensor for human-computer interaction | |
| CN104484073A (en) | Hand touch interaction system | |
| Carrino et al. | Gesture segmentation and recognition with an EMG-based intimate approach-an accuracy and usability study | |
| Hogg | Design of thumb keyboards: Performance, effort and kinematics | |
| CN102221876B (en) | Airflow-inductive computer cursor signal generation device and airflow-inductive mouse device | |
| CN108897444A (en) | The method and system of cursor control are realized using wearable wristband type universal serial mouse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |