CN102693025B - Touch finger identification method for multi-touch interaction system - Google Patents

Abstract

The invention discloses a touch finger identification method for a multi-touch interaction system, and belongs to the field of multi-touch technology. The method includes: 1) obtaining fingers F0, F1,...Fn detected in a frame, 2) fitting each finger Fj in the frame into an ellipse Ej; 3) according to the ellipse each finger corresponds to and set finger distance T, clustering fingers in the frame, and obtaining k types of fingers; 4) calculating the center of each class H{F0, F1,...Fp} and a convex hull region R of the class; and 5) considering each class of the current frame and coverage area of each class as touch finger information (H{F0, F1,..., Fp}, R). The touch finger identification method for the multi-touch interaction system has high accuracy and good robustness in touch finer information identification, makes touch finger identification in the multi-touch interaction system of a touch screen more accurate, and makes the touch screen more practical.

Description

A kind of touch hand recognition methods of multi-touch interaction system

Technical field

The invention belongs to field of multi-touch technology, relate to a kind of touch hand recognition methods of multi-touch interaction system particularly.

Background technology

Multi-point touch system progressively moves towards practical recent years, and in multi-point touch system, user operates with multiple finger usually, such as by multiple finger sliding, rotation, translation.

Traditional touch desktop can only finger on identification desktop, can not obtain touching the information of hand and the movable information of hand, but the motion of hand can provide translation, the rotation information of hand more accurately usually.If the movable information touching hand directly can be obtained, will system be user-friendly to, enhance man-machine interaction concordance.

In the middle of multi-point touch system, when desktop is larger, the user of operation increases, when finger number increases, obtain the hand information that each finger is corresponding how effectively, accurately, comprise each concrete corresponding left hand or the right hand that touch finger, and how touch right-hand man moves, left and right touches the overlay area of hand, and these information all can not directly obtain.Therefore in the urgent need to a kind of quick, stable touch hand recognizer.

Summary of the invention

In order to solve the problem identifying in multi-touch interaction system in prior art and touch hand information, the object of the invention is in multi-touch interaction system, identify touch hand information.

For achieving the above object, the invention provides a kind of touch hand recognition methods in multi-touch interaction system, the method in original state to contact cluster, to same class, other touches finger and identifies right-hand man, in the process of contact movement, Continuous Tracking is carried out to contact, and dynamic conditioning is carried out to cluster result.Finger spacing maximum in single hand is set to T.

Technical scheme of the present invention is:

A touch hand recognition methods for multi-touch interaction system, the steps include:

1) finger F detected in a certain frame is obtained ₀, F ₁... F _n; Wherein, each finger F _j={ f ₀, f ₁... f _m, f _iit is pixel in finger; Wherein, m, n are positive integer;

2) by each finger F in this frame _jfit to an oval E _j;

3) according to each ellipse of finger correspondence and the finger distance T of setting, cluster is carried out to the finger in this frame, obtain k class finger; Wherein, k is natural number;

4) each class H{F is calculated ₀, F ₁..., F _pcenter and such convex closure region R; Wherein, p is positive integer and p≤n;

5) each class of present frame and overlay area thereof are touched hand information (H{F as one ₀, F ₁..., F _p, R).

Further, by finger F _jfit to an oval E _jmethod be:

1) according to finger F _jinterior pixel f _icoordinate (x _i, y _i), calculate elliptical center point coordinate (x _c, y _c);

2) to F _jeach coordinate of interior pixel is normalized, and x after calculating normalization _i, y _icovariance matrix, obtain oval major and minor axis a, b, and the angle theta of transverse and x-axis; Obtain finger F _jthe oval E of matching _j(x _c, y _c, a, b, θ).

Further, described step 3) clustering method be:

A) each finger in this frame is initially a class H _i;

B) H is calculated _iany one finger fitted ellipse E in class _mwith H _jany one finger fitted ellipse E in class _ncentral point distance L, if L maximal value be less than setting finger distance T, then by H _iwith H _jmerge into a class;

C) step b is repeated), until there is no annexable class, obtain k class finger.

Further, according to class H{F ₀, F ₁..., F _pcenter and in such center of each finger obtain such convex closure region R.

Further, the computing method of described R are: by H{F ₀, F ₁..., F _pcenter and the center (x of each finger fitted ellipse in such _c, y _c) the point set determination convex closure region R that forms.

Further, each class H{F of described calculating ₀, F ₁..., F _pthe method at center be:

1) for class H{F ₀, F ₁..., F _pin the fitted ellipse E (x of each finger _c, y _c, a, b, θ), according to the sequence counterclockwise of θ value;

2) calculate the major axis intersection point of all fitted ellipse in such, calculate the mean value of all intersection points, be such H{F ₀, F ₁..., F _pcenter (x _h, y _h).

Main contents of the present invention are:

Step S1: be located at the finger F that a certain frame detects ₀, F ₁... F _n, wherein each finger comprises F _j={ f ₀, f ₁... f _m, f _iit is pixel in finger.To each finger F _jmatching oval E _j(x _c, y _c, a, b, θ), x _c, y _cbe elliptical center point coordinate, a, b are oval major and minor axis, and θ is oval major axis and the angle of x-axis.

Our fitting algorithm is as follows, and the method complexity is low, easily realizes:

If finger F _jan interior pixel f _icoordinate is (x _i, y _i), calculate (x _c, y _c):

$x_c=\frac{1}{n}\underset{i=0}{\overset{n}{\mathrm{\Σ}}}{x}_i,y_c=\frac{1}{n}\underset{i=0}{\overset{n}{\mathrm{\Σ}}}{y}_i$

By f _inormalization:

x _i＝x _i-x _c

y _i＝y _i-y _c

Calculate x _i, y _icovariance matrix:

$\left[\begin{array}{cc}\mathrm{\Σ}{x}_i{x}_i& -\mathrm{\Σ}{x}_i{y}_i\\ -\mathrm{\Σ}{x}_i{y}_i& \mathrm{\Σ}{y}_i{y}_i\end{array}\right]$

λ is obtained to this Matrix Calculating eigenwert ₀, λ ₁, (λ ₀> λ ₁), λ ₀character pair vector (v ₀, v ₁) then

$a=2\sqrt{{\mathrm{\λ}}_0}$

$b=2\sqrt{{\mathrm{\λ}}_1}$

θ＝atan(v ₁/v ₀)

Step S2: according to maximum contact spacing from T, to F ₀, F ₁... F _nits cluster is H by cluster ₀, H ₁..., H _k, k class altogether.

Clustering algorithm:

1. initial each finger is a class H _i.

2. for each class H _i, finding its nearest class is H _j, for arbitrary F _m∈ H _i, F _n∈ H _jif, | F _m-F _n| < T (| F _m-F _n| represent finger distance, refer to the distance of the elliptical center point of touch area matching), then by H _i, H _jand be a class.

3. repeat the 2nd step, until do not have class to merge.

Step S3: for the fitted ellipse E (x of each finger in each class H _c, y _c, a, b, θ) in θ sort in counterclockwise mode.Calculate the intersection point of all transverses in such, be set to (x ₁, y ₁), (x ₂, y ₂) ..., (x _n, y _n), get the center of such H; The center of such H is (x _h, y _h): wherein

$x_h=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{x}_i,y_h=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{y}_i$

Calculate the convex closure region R of the point set of each finger centre composition in the center of H and class H, representative touches the region of hand.

Step S4: using the touch hand of each class as present frame, the H{F obtained ₀, F ₁, F _n, R, as touch hand information, it comprises F ₀, F ₁..., F _nindividual finger, its center is x _h, y _h, the region covered is R.

After initial clustering, each contact uniquely indicates, and in Continuous Tracking process, the minimum distance according to front and back two frame point carries out Continuous Tracking to sign point, when there being new hand to point out that existing or original finger disappears, and adopts above-mentioned algorithm to recalculate.

The invention has the beneficial effects as follows:

The invention provides the method identifying in multi-touch interaction system and touch hand information.Only need the area information obtaining touching finger, Fast Fitting is oval, just can obtain the direction of calculating finger, to finger cluster, identifies and touches hand information.By fitted ellipse, finger cluster, recognize touch hand information accuracy high, robustness is good.The invention enables in the multi-touch interaction system of touch screen and identify that touch hand is more accurate, make touch-screen more practical.

Accompanying drawing explanation

Fig. 1 is system layout's schematic diagram;

1, desktop, 2, infrared lamp, 3, video camera, 4, projector, 5, main frame;

Fig. 2 is finger fitted ellipse, Clustering Effect figure;

Fig. 3 is process flow diagram of the present invention.

Embodiment

Each detailed problem involved in technical solution of the present invention is described in detail below in conjunction with accompanying drawing.Be to be noted that described embodiment is only intended to be convenient to the understanding of the present invention, and any restriction effect is not play to it.

In order to realize method of the present invention, adopt a CPU to be 1.6G during enforcement, internal memory is 512M, the computing machine of hard disk 320G or the computing machine of other types, and the length of multiple point touching desktop is 1.5 meters, and width is 1.15 meters.System layout's schematic diagram as shown in Figure 1.Fig. 2 gives us the diagram of embodiment, and as can be seen from Figure 2, the carrying out that we the are correct cluster of three hands, Rectangular Bounding Volume represents the result recognized.Adopt Matlab to work out relative program on computers, the process flow diagram of the inventive method refers to Fig. 3, and the concrete implementation step of the inventive method is as follows:

Step S1: being located at the finger that a certain frame detects is F ₀, F ₁... F ₁₅, 15 touch finger areas altogether, and wherein each finger comprises F _j={ f ₀, f ₁... f _i... f _m, f _iit is pixel in finger.To each finger F _jmatching oval E _j(x _c, y _c, a, b, θ)

Fitting algorithm is as follows:

If f _icoordinate is (x _i, y _i), calculate (x _c, y _c):

$x_c=\frac{1}{n}\underset{i=0}{\overset{n}{\mathrm{\&Sigma;}}}{x}_i,y_c=\frac{1}{n}\underset{i=0}{\overset{n}{\mathrm{\&Sigma;}}}{y}_i$

By f _inormalization:

x _i＝x _i-x _c

y _i＝y _i-y _c

Calculate x _i, y _icovariance matrix:

$\left[\begin{array}{cc}\mathrm{\&Sigma;}{x}_i{x}_i& -\mathrm{\&Sigma;}{x}_i{y}_i\\ -\mathrm{\&Sigma;}{x}_i{y}_i& \mathrm{\&Sigma;}{y}_i{y}_i\end{array}\right]$

λ is obtained to this Matrix Calculating eigenwert ₀, λ ₁, (λ ₀> λ ₁), λ ₀character pair vector (v ₀, v ₁), then

$a=2\sqrt{{\mathrm{\&lambda;}}_0}$

$b=2\sqrt{{\mathrm{\&lambda;}}_1}$

θ＝atan(v ₁/v ₀)

Step S2: distance T=30 pixel distance in selected maximum finger, is also in other words more than 30 pixel distances, then not thinks and belong to a class; To F ₀, F ₁... F ₁₅its cluster is H by cluster ₀, H ₁, H ₂, 3 classes altogether.

Clustering algorithm:

1. initial each finger is a class H _i.

2. for each class H _i, finding its nearest class is H _j, for arbitrary F _m∈ H _i, F _n∈ H _jif, | F _m-F _n| < T (| F _m-F _n| represent finger distance, refer to the distance of the elliptical center point of touch area matching), then by H _i, H _jand be a class.

3. repeat the 2nd step, until do not have class to merge.

Step S3: for each class H (i.e. H ₀, H ₁, H ₂) the fitted ellipse E (x of each finger _c, y _c, a, b, θ) in θ with

Counterclockwise mode sorts.Calculate the intersection point of the major axis of each ellipse, be the center x of H _h, y _h, calculate the center of H

With the convex closure region R of the point set of finger centre composition each in class H, representative touches the region of hand.

Step S4: using the touch hand of each class as present frame, the H{F obtained ₀, F ₁, F _n, R, as touch hand information, it comprises F ₀, F ₁, F _nindividual finger, its center is x _h, y _h, the region covered is R.

To the touch finger in Fig. 2, fitted ellipse, each as shown in Figure 2 finger areas comprises an ellipse, and to 15 finger areas clusters, it is three classes that result is gathered, and adopts rectangle frame to draw in fig. 2.

The above; be only the embodiment in the present invention; but protection scope of the present invention is not limited thereto; any people being familiar with this technology is in the technical scope disclosed by the present invention; the conversion or replacement expected can be understood; all should be encompassed in and of the present inventionly comprise within scope, therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.

Claims (4)

1. a touch hand recognition methods for multi-touch interaction system, the steps include:

1) finger F detected in a certain frame is obtained ₀, F ₁... F _n; Wherein, each finger F _j={ f ₀, f ₁... f _m, f _iit is pixel in finger; Wherein, m, n are positive integer;

2) by each finger F in this frame _jfit to an oval E _j; Wherein, approximating method is: to each finger F _jmatching oval E _j(x _c, y _c, a, b, θ), x _c, y _cbe elliptical center point coordinate, a, b are oval major and minor axis, and θ is oval major axis and the angle of x-axis; If finger F _jan interior pixel f _icoordinate is (x _i, y _i), calculate (x _c, y _c): by f _inormalization: x _i=x _i-x _c, y _i=y _i-y _c; Calculate x _i, y _icovariance matrix: $\left[\begin{array}{cc}\mathrm{\&Sigma;}{x}_i{x}_i& -\mathrm{\&Sigma;}{x}_i{y}_i\\ -\mathrm{\&Sigma;}{x}_i{y}_i& \mathrm{\&Sigma;}{y}_i{y}_i\end{array}\right],$ λ is obtained to this Matrix Calculating eigenwert ₀, λ ₁, (λ ₀> λ ₁), λ ₀character pair vector (v ₀, v ₁), then θ=atan (v ₁/ v ₀);

3) according to each ellipse of finger correspondence and the finger distance T of setting, cluster is carried out to the finger in this frame, obtain k class finger; Wherein, k is natural number; Described clustering method is: a) each finger in this frame is initially a class H _i; B) H is calculated _iany one finger fitted ellipse E in class _mwith H _jany one finger fitted ellipse E in class _ncentral point distance L, if L maximal value be less than setting finger distance T, then by H _iwith H _jmerge into a class; C) step b is repeated), until there is no annexable class, obtain k class finger;

4) each class H{F is calculated ₀, F ₁..., F _pcenter and such convex closure region R; Wherein, p is positive integer and p≤n;

5) each class of present frame and overlay area thereof are touched hand information (H{F as one ₀, F ₁..., F _p, R).

2. the method for claim 1, is characterized in that according to class H{F ₀, F ₁..., F _pcenter and in such center of each finger obtain such convex closure region R.

3. method as claimed in claim 2, is characterized in that the computing method of described R are: by H{F ₀, F ₁..., F _pcenter and the center (x of each finger fitted ellipse in such _c, y _c) the point set determination convex closure region R that forms.

4. method as claimed in claim 3, is characterized in that each class H{F of described calculating ₀, F ₁..., F _pthe method at center be:

1) for class H{F ₀, F ₁..., F _pin the fitted ellipse E (x of each finger _c, y _c, a, b, θ), according to the sequence counterclockwise of θ value;

2) calculate the major axis intersection point of all fitted ellipse in such, calculate the mean value of all intersection points, be such H{F ₀, F ₁..., F _pcenter (x _h, y _h).