CN105759967B - A kind of hand overall situation attitude detecting method based on depth data - Google Patents
A kind of hand overall situation attitude detecting method based on depth data Download PDFInfo
- Publication number
- CN105759967B CN105759967B CN201610093720.6A CN201610093720A CN105759967B CN 105759967 B CN105759967 B CN 105759967B CN 201610093720 A CN201610093720 A CN 201610093720A CN 105759967 B CN105759967 B CN 105759967B
- Authority
- CN
- China
- Prior art keywords
- point
- palm
- hand
- finger
- finger tip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of hand overall situation attitude detecting method based on depth data, it includes following sub-step: S1: finger three-dimensional detection: using hand depth data as input, detect palm center to finger general direction, simultaneously by calculating point cloud center as palm center, current D translation is obtained；S2: the three-dimensional normal vector of palm the detection of palm three-dimensional: is obtained by fit Plane；S3: palm overall situation posture indicates: palm overall situation posture shows as the D translation of palm and the three-dimensional rotation of palm, and three-dimensional rotation is combined to obtain by the rotation of finger orientation and the rotation of palm normal vector.The present invention has counted a set of simple effective method on the basis of hand depth data, and the detection of hand overall situation posture is realized using three steps, and clear in structure to be illustrated, algorithm overall simple, execution efficiency is high, has preferable practicability.
Description
Technical field
The present invention relates to field of human-computer interaction more particularly to a kind of hand overall situation attitude detection sides based on depth data
Method.
Background technique
Human-computer interaction wants communication channel as the group between people and robot, technology also constantly by operate it is feasible to
Convenient, comfortable direction is developed, in terms of more and more researchs are placed on the interaction technique based on hand, compared to other human body portions
Position, hand freedom and flexibility are responsible for a large amount of interworking in the routine work of people, and the operand completed by hand is not
Win number, but based on traditional interactive mode mostly operated with hand, and three-dimension gesture interaction is compared to traditional mode, not only
It is more naturally comfortable, it can also promote the most direct interactive experience of user.And in recent years, with the increasingly stream of depth transducer technology
Row, the technical threshold of three-dimension gesture technology compared to being greatly reduced in the past, while various demands are continuously increased, and are promoted
The further development of three-dimension gesture technical research.
The technical foundation of three-dimension gesture is exactly hand gestures estimation, at present the hand gestures estimation method master under three-dimension gesture
Be divided into discriminate and production, the method combination threedimensional model of production carries out the mode of energy function optimization, often according to
Rely and is accelerated with GPU, it is computationally intensive.Production method is trained model by using database, then is estimated, for
Untrained posture is often ineffective, and precision is general.In actual three-dimension gesture problem, the posture of hand can be described
For the posture of palm and the posture of finger, the posture of palm is global posture, and the posture of finger is influenced by the posture of palm,
Therefore the posture of palm is carried out according to a preliminary estimate, the difficulty of hand entirety Attitude estimation capable of being substantially reduced.Part Methods at present
In estimation hand gestures, i.e., global direction, the method for use has cloud PCA method, the Return Law, model fitting method etc..Its midpoint cloud
PCA method will appear the case where apparent error judges for the hand of three-dimensional deformation, and the recurrence mode based on model is by random character
Be affected, and model fitting method then needs to generate the threedimensional model under a large amount of hand overall situations direction, be not still it is optimal and
Simplest method.
In conjunction at present to three-dimensional hand overall situation direction determining method advantage and disadvantage on the basis of, the present invention proposes on this basis
Hand overall situation direction detection method based on depth data.
The present invention is directed to hand overall situation Attitude estimation problem, and problem is decomposed, and the finger for detecting hand respectively is three-dimensional
Direction and palm three-dimensional calculate the global posture of hand, i.e. three-dimensional rotation and D translation in the result estimated herein, real
Now to hand overall situation direction estimation.Method proposed by the present invention is clear in structure to be illustrated, and algorithm overall simple, execution efficiency is high, tool
There is preferable practicability.
Application No. is the patents of invention of CN201410254196.7 to disclose a kind of tracking of real time hand, posture classification and interface
Control.This invention proposes the devices that Electronic Design operation is carried out based on gesture, and main functional modules include finger tracking, appearance
State classification, Interface Control and related circuit design etc..In gesture posture categorization module, the invention simultaneously extracts layout areas sift spy
The form for levying point, carries out the classification of gesture posture, based on sift feature using SVM to determine hand posture using classification results.This
Invention propose method be hand global posture detection, therefore with this invention towards the problem of different, the method to be solved
Thinking is also made a world of difference.
Application No. is 201180075255.9 patents of invention to disclose a kind of man-machine friendship of the short distance based on dynamic posture
Mutually.The device that the invention proposes includes the parts such as posture starting detecting and alarm, gesture recognition engine and user command, wherein posture
Identify that hand gestures and hand track in engine implementation depth image, hand track are examined using masstone and depth histogram
It surveys, and gesture recognition engine then extracts bias, rectangularity, compactness, orientation, center etc. profile spy to hand bianry image
Sign, then using classifier carry out gesture posture classification, it can be seen that the method have with patent 1 it is a little similar, with this patent
Problem direction to be solved does not have to, and method is also different.
Application No. is 201210438642.0 patents of invention to disclose a kind of space gesture posture based on depth camera
Command processing method.The device that the invention proposes obtains hand point cloud data using Kinect sensor, and is registrated according to plane
Data later extract outline data, realize the identification of gesture posture.The method utilizes three-dimensional after obtaining hand depth data
Plane carries out point cloud registering, that is, seeks the position and direction of palm plane, then projects to a cloud and obtains two in plane
It is worth image, then extracts finger tip information and calculate hand gestures.Wherein this mode is to only determining simply by cloud plane
Palm plane, the effect and bad for the method under deformation gesture is different with the problem to be solved in the present invention target, and
The method used in volar direction detection is also different, and the method that the present invention uses is simple, while can adapt to more
Complicated gesture situation.
Application No. is the patents of invention of 20131047231.X to disclose a kind of Fingertip Detection based on depth information.
The invention carries out finger tip detection using depth data, mainly includes palm segmentation and palm of the hand positioning, refers to root positioning and finger tip inspection
Survey and etc..Wherein when referring to root positioning, the invention uses outline concave as hand root, but deformation gesture bottom profiled is recessed
Can not be as the judgment basis for referring to root, the method that graph model is used for detection the invention of finger tip.And this patent is also adopted
Finger tip detection has been carried out with profile, but only detection part finger tip, method is different, and is not distinguish to finger tip, and to refer to
Point removes detection finger substantially three-dimensional, therefore problem to be solved is different, and method is also different.
Summary of the invention
The hand overall situation posture based on depth data that it is an object of the invention to overcome the deficiencies of the prior art and provide a kind of
Detection method, most important problem are how to utilize the hand data of depth transducer, rapidly estimate the current gesture of user
Under hand overall situation posture, i.e. the 3 d pose of palm lays the foundation for later hand gestures estimation.
The purpose of the present invention is achieved through the following technical solutions: a kind of hand overall situation posture based on depth data
Detection method, it includes following sub-step:
S1: the detection of finger three-dimensional: using hand depth data as input, detect palm center to finger substantially
Direction Vy, while by calculating point cloud center as palm center, obtain current D translation Tg；
S2: the three-dimensional normal vector V of palm the detection of palm three-dimensional: is obtained by fit Planez；
S3: palm overall situation posture indicates: palm overall situation posture shows as the D translation T of palmgWith the three-dimensional rotation of palm
Turn Tr, three-dimensional rotation is combined to obtain by the rotation of finger orientation and the rotation of palm normal vector.
The step S1 includes following sub-step:
S11: image preprocessing: pre-processing depth image, i.e., thresholding processing is first carried out, by image background regions
Zero setting only retains hand shape area；Closed operation is carried out on this basis；
S12: it calculates hand images center: for the image of hand depth data, carrying out hand center calculation, as point cloud
Central point Pcen, wherein calculation is the mean value of each dimension of hand region pixel；
The detection of S13:2D finger tip point, including following sub-step:
S131: the bianry image obtained for pretreatment carries out profile C detection, finds the corresponding profile C of maximum perimeter,
Convex polygon detection is carried out to profile C；
S132: carrying out convex closure to convex polygon and defect point detect, and the convex closure point includes finger tip point and non-finger tip point,
The defect point then contains concave point and noise spot between finger；
S133: it is screened to reconnaissance: for convex closure point, calculating separately the distance between two two o'clocks, excluded distance and be less than threshold value
Point, that is, exclude continuous and extra convex closure point；For defect point, calculate first each defect point to hand picture centre away from
From, then calculate hand rotation Rectangular Bounding Volume, obtain the corresponding square boundary of hand region, wherein rectangle it is long with it is wide in it is larger
Value is L, and the defect point by distance greater than 1/2L excludes, i.e. noise drawbacks point on exclusion profile；
S134: after obtaining defect point and convex closure point, location index C of each convex closure point in profile is calculatedt, similarly lack
Location index C of the trapping spot in profileq；
S135: according to convex closure order traversal convex closure point, if the corresponding outline position of some convex closure point indexes CtPositioned at two
Between the outline position of a Adjacent defect point, it may be assumed that Cq1< Ct< Cq2, then it is considered as finger tip point PTip, it is otherwise non-finger tip point PNTip；
As the finger tip point P metTipWhen less than two, the outline position of each defect point is indexed, detection and its most phase
The outline position index of adjacent convex closure point, meets Ct1> Cq, Ct2< Cq1When, this convex closure point is calculated at a distance from 2D centroid, if
It is greater than 1/3L, then is considered as finger tip point T2d, it otherwise just abandons, while carrying out the judgement of finger tip point number:
When being greater than 5, stop finger tip point detection.And if finger tip point number is 0 after this step calculates, at this time
The vertical straight-on camera of hand, finger orientation are set as particular value, towards lens direction；
S14: the 2D fingertip location P of initial option is determinedTipLater, image is transformed in conjunction with profile C give directions Ttip2d, because
For PTipFor finger tip dot profile index value；It is determined 3D finger tip direction, including following sub-step at this time:
S141: in conjunction with Ttip2dCorresponding depth Z is changed into the 3D point T under camera coordinates system under depth datatip3d,
To a cloud central point PcenAlso P under camera coordinates system is transformed tocen3D；
S142: each 3D finger tip is calculated separately to Pcen3DThree-dimensional vector: Pvi=T3di-Pcen3D, then VyIt is each
The mean value of finger tip three-dimensional:
It is normalized simultaneously.
It further include a treatment on special problems step after the step S135, for three-dimensional hand, there are thumbs to erect
It rises, and the case where remaining digital flexion, the finger tip point detected at this time is not actual finger general direction, needs to add thus
Enter approximate wrist point detection；During step S135 detection, if the defect point of detection is 1, or the finger tip of detection
When point number is 1, approximate wrist point detection, including following sub-step are carried out at this time:
S1011: for all non-finger tip convex closure point PNTip, calculate separately the depth value Z (P of its corresponding pixel pointsNTip),
Select maximum Z (PNTip) corresponding PNTip_minAs preliminary wrist point；
S1012: approximate wrist point at this time is the convex closure point on profile, needs to detect its neighborhood point, finds neighborhood depth
It is worth bigger profile point；P thuswrist=minZ (PNTip_min+ Neb), Neb=1: ± N, wherein PwristIt is expressed as quasi- wrist point
PNTip_minThe point that depth value is bigger within the scope of neighborhood N is as approximate wrist point.
When the point number of 2D finger tip detection is 1, the information that approximate wrist point direction is added at this time is corrected, including with
Lower sub-step:
PwristBe converted to three-dimensional point P under camera spacewrist3D, then orientation Vw=Pcen3D-Pwrist3D, and utilize VwIt is right
VyIt is corrected:
V after normalizing at this timeyIt is exactly the substantially three-dimensional of finger, while Tg=Pcen3D。
The step S2 includes following sub-step:
S21: non-finger tip point determines: during 2D finger tip detection, the non-finger tip point P that recordsNTipWith defect point one
And it is denoted as palm point Ppalm, image coordinate point T is obtained by profile Cpalm, and camera is converted to according to depth value and camera parameter and is sat
3D point T under mark systempalm3D；
S22: plane fitting is carried out to the non-finger tip point of 3D: sets spatial plane equation as Ax+By+Cz+D=0, remembers the non-finger tip of 3D
Point is Tpalm3Di=(xi, yi, zi), then seek objective function are as follows:
Using least square method, the parameter { A, B, C, D } of plane equation is solved, plane normal vector is { A, B, C }, at this time hand
Slapping direction is approximately Vz={ A, B, C }.
Palm overall situation direction is expressed as three-dimensional rotation Tr, since the freedom degree of palm rotation is higher, revolved by finger orientation
Transfering the letter breath and volar direction rotation information, calculate three-dimensional rotation Tr；The step S3 includes following sub-step:
S31: primary standard finger orientation upward, is denoted as Vy0, and finger straight-on camera, direction are denoted as Vz0；It is counted by step S1
Obtained finger substantially three-dimensional Vy, find out Ty；The TyIt is indicated by spin matrix or quaternary number, calculation
To calculate Vy、Vy0Cross product and angle；
S32: for the rotation information of volar direction, V is first carried outzUpdate Vz=Ty*Vz, and with side identical with step S31
Formula obtains rotation information Tz；
S33: the three-dimensional rotation T of entire handr=Ty*Tz, to detect the global posture of hand.
The beneficial effects of the present invention are:
Method proposed by the present invention is based on the hand depth data of depth transducer (such as Kinect2) shooting, in hand depth
For degree on the basis of, the present invention devises a set of simple effective method, realizes hand overall situation posture using three steps
Detection: the detection of finger three-dimensional, the detection of palm three-dimensional and the expression of palm overall situation posture.For hand overall situation appearance
State shows as the D translation and three-dimensional rotation of palm.Wherein D translation is expressed as the movement of palm, and three-dimensional rotation indicates
Palm plane is varied under current pose by first state of value.Due to the flexibility of hand, freedom degree is higher, and according to the spy of palm
Point, using the two normal vectors, can realize opponent by detecting finger substantially three-dimensional and palm three-dimensional normal vector respectively
The estimation of portion's overall situation posture.Method proposed by the present invention is clear in structure to be illustrated, and algorithm overall simple, execution efficiency is high, have compared with
Good practicability.
Detailed description of the invention
Fig. 1 is the method for the present invention flow chart.
Specific embodiment
Technical solution of the present invention is described in further detail with reference to the accompanying drawing:
The present invention includes 3 steps:
(1) finger three-dimensional detects: finger three-dimensional indicates the direction from palm center to middle fingertip, and three
In the case that finger shape changes under dimension gesture, such as digital flexion, and palm rotates, relatively it is difficult to estimate finger at this time
General direction.Through analyzing in the depth map of hand, often observe that part finger tip still than more prominent, therefore can pass through
Finger tip and the line at hand center obtain finger three-dimensional.
(2) palm three-dimensional detects: palm three-dimensional is expressed as the normal vector of palm plane, since palm can be close
Like being plane, normal vector is by palm direction outside.Three-dimension gesture assistant's palm point still in approximate plane, therefore
The present invention obtains the three-dimensional normal vector of palm by fit Plane.
(3) expression of palm overall situation posture: palm overall situation posture shows as the three-dimensional rotation of the D translation and palm of palm
Turn, D translation is obtained by hand center, and three-dimensional rotation then passes through the rotation of finger orientation and the rotation phase group of palm normal vector
Conjunction obtains, to detect hand overall situation posture by above method, not only algorithm is simple, but also is to the three-dimensional of some of complex
Also there is certain applicability, there is good practical value.
As shown in Figure 1, a kind of hand overall situation attitude detecting method based on depth data, it includes following sub-step:
S1: the detection of finger three-dimensional: finger orientation detection is mainly used as input using hand depth data, the purpose is to
Detected in the case where angle is changeable and gesture is changeable palm center to finger general direction Vy, the present invention is with practical true
Data are analyzed, and are found the corresponding depth map of most of gesture, are still remained all or part of tip shape, use thus
The detection of 3D finger tip detection method progress finger orientation.
Using hand depth data as input, detect palm center to finger general direction Vy, while passing through calculating
Point cloud center obtains current D translation T as palm centerg；
The step S1 includes following sub-step:
S11: image preprocessing: pre-processing depth image, i.e., thresholding processing is first carried out, by image background regions
Zero setting only retains hand shape area；It carries out closed operation on this basis, not only reduces noise and small empty profile in this way
It influences, can more eliminate the influence in invalid depth region.
S12: it calculates hand images center: for the image of hand depth data, carrying out hand center calculation, as point cloud
Central point Pcen, wherein calculation is the mean value of each dimension of hand region pixel；
The detection of S13:2D finger tip point, including following sub-step:
S131: the bianry image obtained for pretreatment carries out profile C detection, finds the corresponding profile C of maximum perimeter,
Convex polygon detection is carried out to profile C；The convex polygon being achieved in that approximate can replace hand profile, can be avoided using wheel
Wide C bring not robustness.
S132: carrying out convex closure to convex polygon and defect point detect, and the convex closure point includes finger tip point and non-finger tip point,
The defect point then contains concave point and noise spot between finger；Convex closure point has generally comprised finger tip point and non-finger tip at this time
Point, and defect point then contains concave point and noise spot between finger, it is therefore desirable to screened to reconnaissance.
S133: it is screened to reconnaissance:
For convex closure point, the distance between two two o'clocks are calculated separately, exclude the point that distance is less than threshold value, threshold value can be set
For 10 pixels, that is, exclude continuous and extra convex closure point；
For defect point, calculate first each defect point to hand picture centre distance, then calculate hand rotation rectangle
Bounding box obtains the corresponding square boundary of hand region, and wherein the long the larger value with width of rectangle is L, will be apart from greater than 1/2L
Defect point exclude, i.e., exclusion profile on noise drawbacks point；In true hand structure, recess between adjacent finger away from
It is less than the half of hand sizes with a distance from hand center.
S134: after obtaining defect point and convex closure point, location index C of each convex closure point in profile is calculatedt, similarly lack
Location index C of the trapping spot in profileq；
S135: according to convex closure order traversal convex closure point, if the corresponding outline position of some convex closure point indexes CtPositioned at two
Between the outline position of a Adjacent defect point, it may be assumed that Cq1< Ct< Cq2, then it is considered as finger tip point PTip, it is otherwise non-finger tip point PNTip；
As the finger tip point P metTipWhen less than two, the outline position of each defect point is indexed, detection and its most phase
The outline position index of adjacent convex closure point, meets Ct1> Cq, Ct2< Cq1When, this convex closure point is calculated at a distance from 2D centroid, if
It is greater than 1/3L, then is considered as finger tip point T2d, it otherwise just abandons, while carrying out the judgement of finger tip point number:
When being greater than 5, stop finger tip point detection.And if finger tip point number is 0 after this step calculates, at this time
The vertical straight-on camera of hand, finger orientation are set as particular value, towards lens direction；
It further include a treatment on special problems step after the step S135, for three-dimensional hand, there are thumbs to erect
It rises, and the case where remaining digital flexion, the finger tip point detected at this time is not actual finger general direction, needs to add thus
Enter approximate wrist point detection；During step S135 detection, if the defect point of detection is 1, or the finger tip of detection
When point number is 1, approximate wrist point detection, including following sub-step are carried out at this time:
S1011: for all non-finger tip convex closure point PNTip, calculate separately the depth value Z (P of its corresponding pixel pointsNTip),
Select maximum Z (PNTip) corresponding PNTip_minAs preliminary wrist point；Because wrist is more than finger under actual gesture operation
Far from camera, depth value is bigger.
S1012: approximate wrist point at this time is the convex closure point on profile, needs to detect its neighborhood point, finds neighborhood depth
It is worth bigger profile point；P thuswrist=minZ (PNTip_min+ Neb), Neb=1: ± 20, wherein PwristIt is expressed as quasi- wrist point
PNTip_minThe point that depth value is bigger in 20 range of neighborhood is as approximate wrist point.
S14: the 2D fingertip location P of initial option is determinedTipLater, image is transformed in conjunction with profile C give directions Ttip2d, because
For PTipFor finger tip dot profile index value；It is determined 3D finger tip direction, including following sub-step at this time:
S141: in conjunction with Ttip2dCorresponding depth Z is changed into the 3D point T under camera coordinates system under depth datatip3d,
To a cloud central point PcenAlso P under camera coordinates system is transformed tocen3D；
S142: each 3D finger tip is calculated separately to Pcen3DThree-dimensional vector: Pvi=T3di-Pcen3D, then VyIt is each
The mean value of finger tip three-dimensional:
It is normalized simultaneously.
When the point number of 2D finger tip detection is 1, the information that approximate wrist point direction is added at this time is corrected, including with
Lower sub-step:
PwristBe converted to three-dimensional point P under camera spacewrist3D, then orientation Vw=Pcen3D-Pwrist3D, and utilize VwIt is right
It is corrected:
V after normalizing at this timeyIt is exactly the substantially three-dimensional of finger, while Tg=Pcen3D。
S2: the three-dimensional normal vector V of palm the detection of palm three-dimensional: is obtained by fit Planez；Volar direction is by palm
Normal vector VzIt is indicated, is directed toward outside from palm.
The step S2 includes following sub-step:
S21: non-finger tip point determines: during 2D finger tip detection, the non-finger tip point P that recordsNTipWith defect point one
And it is denoted as palm point Ppalm, image coordinate point T is obtained by profile Cpalm, and camera is converted to according to depth value and camera parameter and is sat
3D point T under mark systempalm3D；Because the palm in-plane under three-dimension gesture is three-dimensional.
S22: carry out plane fitting to the non-finger tip point of 3D: all non-finger tip points represent the point on palm, and palm can
To be approximately plane, therefore the fitting of least square method space plane is carried out based on the non-finger tip point of 3D.
If spatial plane equation is Ax+By+Cz+D=0, the note non-finger tip point of 3D is Tpalm3Di=(xi, yi, zi), then it seeks
Objective function are as follows:
Using least square method, the parameter { A, B, C, D } of plane equation is solved, plane normal vector is { A, B, C }, at this time hand
Slapping direction is approximately Vz={ A, B, C }.
Simultaneously because the sequence of contour detecting be it is counterclockwise, the finger normal vector for calculating acquisition is also directed towards camera side
To, therefore volar direction proposed by the invention is estimated to be suitble to most of situation of the volar direction towards camera, does not consider palm
The case where backwards to camera.
S3: palm overall situation posture indicates: palm overall situation posture shows as the D translation T of palmgWith the three-dimensional rotation of palm
Turn Tr, three-dimensional rotation is combined to obtain by the rotation of finger orientation and the rotation of palm normal vector.
Palm overall situation direction is expressed as three-dimensional rotation Tr, since the freedom degree of palm rotation is higher, revolved by finger orientation
Transfering the letter breath and volar direction rotation information, calculate three-dimensional rotation Tr；The step S3 includes following sub-step:
Palm overall situation direction is expressed as three-dimensional rotation Tr, can be by finger side since the freedom degree of palm rotation is higher
To rotation information and volar direction rotation information, three-dimensional rotation T is calculatedr.Assuming that primary standard finger orientation is upward, it is denoted as
Vy0, and finger straight-on camera, direction are denoted as Vz0, the finger substantially three-dimensional V that is calculated at this time by step 1y, find out Ty,
And TyIt can be indicated, can also be indicated by quaternary number, calculation can calculate V by spin matrixy、Vy0Cross product and angle
It calculates.For the rotation information of volar direction, V is first carried outzUpdate Vz=Ty*Vz, and rotation information T is obtained in the same wayz,
The then three-dimensional rotation T of entire handr=Ty*Tz, to detect the global posture of hand.
Claims (5)
1. a kind of hand overall situation attitude detecting method based on depth data, it is characterised in that: it includes following sub-step:
S1: finger three-dimensional detection: using hand depth data as input, detect palm center to finger general direction
Vy, while by calculating point cloud center as palm center, obtain current D translation Tg；
S2: the three-dimensional normal vector V of palm the detection of palm three-dimensional: is obtained by fit Planez；
S3: palm overall situation posture indicates: palm overall situation posture shows as the D translation T of palmgWith the three-dimensional rotation T of palmr,
Three-dimensional rotation is combined to obtain by the rotation of finger orientation and the rotation of palm normal vector；
The step S1 includes following sub-step:
S11: image preprocessing: pre-processing depth image, i.e., first carries out thresholding processing, image background regions are set
Zero, only retain hand shape area；Closed operation is carried out on this basis；
S12: it calculates hand images center: for the image of hand depth data, carrying out hand center calculation, as point Yun Zhongxin
Point Pcen, wherein calculation is the mean value of each dimension of hand region pixel；
The detection of S13:2D finger tip point, including following sub-step:
S131: the bianry image obtained for pretreatment carries out profile C detection, the corresponding profile C of maximum perimeter is found, to wheel
Wide C carries out convex polygon detection；
S132: carrying out convex closure point to convex polygon and defect point detect, and the convex closure point includes finger tip point and non-finger tip point, institute
The defect point stated then contains concave point and noise spot between finger；
S133: it is screened to reconnaissance: for convex closure point, calculating separately the distance between two two o'clocks, exclude the point that distance is less than threshold value,
Exclude continuous and extra convex closure point；For defect point, calculate first each defect point to hand picture centre distance, then
It calculates hand and rotates Rectangular Bounding Volume, obtain the corresponding square boundary of hand region, wherein the long the larger value with width of rectangle is
L, the defect point by distance greater than 1/2L exclude, i.e. noise drawbacks point on exclusion profile；
S134: after obtaining defect point and convex closure point, location index C of each convex closure point in profile is calculatedt, similarly defect point
Location index C in profileq；
S135: according to convex closure order traversal convex closure point, if the corresponding outline position of some convex closure point indexes CtIt is adjacent positioned at two
Between the outline position of defect point, it may be assumed that Cq1< Ct< Cq2, then it is considered as finger tip point PTip, it is otherwise non-finger tip point PNTip；
As the finger tip point P metTipWhen less than two, the outline position of each defect point is indexed, detection is most adjacent with it
The outline position of convex closure point indexes, and meets Ct1> Cq, Ct2< CqWhen, two convex closure points are at a distance from 2D centroid at this time for calculating, such as
It is greater than 1/3L to fruit, then is considered as finger tip point PTip, otherwise just abandon；The judgement of finger tip point number is carried out simultaneously: when judging the finger
When sharp number is greater than 5, stops finger tip point detection and then confirm the vertical face of hand at this time when judging finger tip number for 0
Camera, finger orientation are set as particular value, towards lens direction；
S14: the 2D. fingertip location P of initial option is determinedTipLater, image is transformed in conjunction with profile C give directions Ttip2d, because
PTipFor finger tip dot profile index value；It is determined 3D finger tip direction, including following sub-step at this time:
S141: in conjunction with Ttip2dCorresponding depth Z is changed into the 3D point T under camera coordinates system under depth datatip3d, to a cloud
Central point PcenAlso P under camera coordinates system is transformed tocen3D；
S142: each 3D finger tip is calculated separately to Pcen3DThree-dimensional vector: Pvi=Ttip3d-Pcen3D, then VyFor each finger tip
The mean value of three-dimensional:
It is normalized simultaneously.
2. a kind of hand overall situation attitude detecting method based on depth data according to claim 1, it is characterised in that: institute
It further include a treatment on special problems step after the step S135 stated, for three-dimensional hand, there are thumbs to hold up, and remaining
The case where digital flexion, the finger tip point detected at this time are not actual finger general direction, need that approximate wrist is added thus
Point detection；During step S135 detection, if the defect point of detection is 1, or the finger tip point number of detection is 1
When, approximate wrist point detection, including following sub-step are carried out at this time:
S1011: for all non-finger tip convex closure point PNTip, calculate separately the depth value ZP of its corresponding pixel pointsNTip, selection is most
Big ZPNTipCorresponding PNTip_maxAs preliminary wrist point；
S1012: approximate wrist point at this time is the convex closure point on profile, needs to detect its neighborhood point, finds neighborhood depth value more
Big profile point；P thuswrist=minZ (PNTip_max+ Neb), Neb=1: ± N, wherein PwristIt is expressed as preliminary wrist point
PNTip_maxThe point that depth value is bigger within the scope of neighborhood N is as approximate wrist point.
3. a kind of hand overall situation attitude detecting method based on depth data according to claim 2, it is characterised in that: when
When the point number of 2D finger tip detection is 1, the information that approximate wrist point direction is added at this time is corrected, including following sub-step:
Pwrist is converted to three-dimensional point P under camera spacewrist3D, then orientation Vw=Pcen3D-Pwrist3D, and utilize VwTo VyInto
Row correction:
V after normalizing at this timeyIt is exactly the substantially three-dimensional of finger, while Tg=Pcen3D。
4. a kind of hand overall situation attitude detecting method based on depth data according to claim 2 or 3, feature exist
In: the step S2 includes following sub-step:
S21: non-finger tip point determines: during 2D finger tip detection, the non-finger tip point P that recordsNTipIt is denoted as together with defect point
Palm point Ppalm, image coordinate point T is obtained by profile Cpalm, and be converted under camera coordinates system according to depth value and camera parameter
3D point Tpalm3D；
S22: plane fitting is carried out to the non-finger tip point of 3D: sets spatial plane equation as Ax+By+Cz+D=0, the note non-finger tip point of 3D is
Tpalm3Di=(xi, yi, zi), then seek objective function are as follows:
Using least square method, the parameter { A, B, C, D } of plane equation is solved, plane normal vector is { A, B, C }, at this time palm side
To being approximately Vz={ A, B, C }.
5. a kind of hand overall situation attitude detecting method based on depth data according to claim 4, it is characterised in that: hand
It slaps global direction and is expressed as three-dimensional rotation Tr, since the freedom degree of palm rotation is higher, pass through finger orientation rotation information and hand
Direction rotation information is slapped, three-dimensional rotation T is calculatedr；The step S3 includes following sub-step:
S31: primary standard finger orientation upward, is denoted as Vy0, and finger straight-on camera, direction are denoted as Vz0；It is calculated by step S1
The finger arrived substantially three-dimensional Vy, find out Ty；The TyIt is indicated by spin matrix or quaternary number, calculation is meter
Calculate Vy、Vy0Cross product and angle；
S32: for the rotation information of volar direction, V is first carried outzUpdate Vz=Ty*Vz, and obtained in a manner of identical with step S31
To rotation information Tz；
S33: the three-dimensional rotation T of entire handr=Ty*Tz, to detect the global posture of hand.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610093720.6A CN105759967B (en) | 2016-02-19 | 2016-02-19 | A kind of hand overall situation attitude detecting method based on depth data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610093720.6A CN105759967B (en) | 2016-02-19 | 2016-02-19 | A kind of hand overall situation attitude detecting method based on depth data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105759967A CN105759967A (en) | 2016-07-13 |
| CN105759967B true CN105759967B (en) | 2019-07-09 |
Family
ID=56330196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610093720.6A Active CN105759967B (en) | 2016-02-19 | 2016-02-19 | A kind of hand overall situation attitude detecting method based on depth data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105759967B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106325509A (en) * | 2016-08-19 | 2017-01-11 | 北京暴风魔镜科技有限公司 | Three-dimensional gesture recognition method and system |
| CN106815578A (en) * | 2017-01-23 | 2017-06-09 | 重庆邮电大学 | A kind of gesture identification method based on Depth Motion figure Scale invariant features transform |
| TWI662482B (en) * | 2017-06-28 | 2019-06-11 | 緯創資通股份有限公司 | Method and Module of Gesture Recognition |
| CN110581987A (en) * | 2018-06-07 | 2019-12-17 | 宏碁股份有限公司 | Three-dimensional display with gesture sensing function |
| CN110880186A (en) * | 2018-09-06 | 2020-03-13 | 山东理工大学 | Real-time human hand three-dimensional measurement method based on one-time projection structured light parallel stripe pattern |
| CN109271931A (en) * | 2018-09-14 | 2019-01-25 | 辽宁奇辉电子系统工程有限公司 | It is a kind of that gesture real-time identifying system is pointed sword at based on edge analysis |
| CN109407848B (en) * | 2018-11-01 | 2021-07-13 | 广东源创智能科技有限公司 | Step detection method and device |
| CN109521879B (en) * | 2018-11-19 | 2022-02-18 | 杭州易现先进科技有限公司 | Interactive projection control method and device, storage medium and electronic equipment |
| CN110472507A (en) * | 2019-07-12 | 2019-11-19 | 中国地质大学（武汉） | Manpower depth image position and orientation estimation method and system based on depth residual error network |
| WO2021160024A1 (en) * | 2020-02-14 | 2021-08-19 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and system of identifying a user selection at a display of a user device |
| CN111694432B (en) * | 2020-06-11 | 2023-04-07 | 济南大学 | Virtual hand position correction method and system based on virtual hand interaction |
| CN116958146B (en) * | 2023-09-20 | 2024-01-12 | 深圳市信润富联数字科技有限公司 | Acquisition method and device of 3D point cloud and electronic device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368290A (en) * | 2011-09-02 | 2012-03-07 | 华南理工大学 | Hand gesture identification method based on finger advanced characteristic |
| CN102982557A (en) * | 2012-11-06 | 2013-03-20 | 桂林电子科技大学 | Method for processing space hand signal gesture command based on depth camera |
| CN103488972A (en) * | 2013-09-09 | 2014-01-01 | 西安交通大学 | Method for detection fingertips based on depth information |
-
2016
- 2016-02-19 CN CN201610093720.6A patent/CN105759967B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368290A (en) * | 2011-09-02 | 2012-03-07 | 华南理工大学 | Hand gesture identification method based on finger advanced characteristic |
| CN102982557A (en) * | 2012-11-06 | 2013-03-20 | 桂林电子科技大学 | Method for processing space hand signal gesture command based on depth camera |
| CN103488972A (en) * | 2013-09-09 | 2014-01-01 | 西安交通大学 | Method for detection fingertips based on depth information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105759967A (en) | 2016-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105759967B (en) | A kind of hand overall situation attitude detecting method based on depth data | |
| US9189855B2 (en) | Three dimensional close interactions | |
| US9330307B2 (en) | Learning based estimation of hand and finger pose | |
| CN106055091B (en) | A kind of hand gestures estimation method based on depth information and correcting mode | |
| Ma et al. | Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information | |
| CN105389539B (en) | A kind of three-dimension gesture Attitude estimation method and system based on depth data | |
| Hasan et al. | RETRACTED ARTICLE: Static hand gesture recognition using neural networks | |
| Ye et al. | Accurate 3d pose estimation from a single depth image | |
| WO2017219391A1 (en) | Face recognition system based on three-dimensional data | |
| Kanhangad et al. | Contactless and pose invariant biometric identification using hand surface | |
| CN108052942B (en) | Visual image recognition method for aircraft flight attitude | |
| Yilmaz et al. | A differential geometric approach to representing the human actions | |
| WO2012077286A1 (en) | Object detection device and object detection method | |
| CN109359514A (en) | A kind of gesture tracking identification federation policies method towards deskVR | |
| CN106446773A (en) | Automatic robust three-dimensional face detection method | |
| CN111160291A (en) | Human eye detection method based on depth information and CNN | |
| CN107479693A (en) | Real-time hand recognition methods based on RGB information, storage medium, electronic equipment | |
| CN103577792A (en) | Device and method for estimating body posture | |
| Kanaujia et al. | Part segmentation of visual hull for 3d human pose estimation | |
| Cohen et al. | 3D body reconstruction for immersive interaction | |
| Azad et al. | Accurate shape-based 6-dof pose estimation of single-colored objects | |
| CN108694348B (en) | Tracking registration method and device based on natural features | |
| Guo et al. | A hybrid framework based on warped hierarchical tree for pose estimation of texture-less objects | |
| Sunyoto et al. | Wrist detection based on a minimum bounding box and geometric features | |
| Wang | Hand gesture recognition based on fingertip detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |