CN103092439B - Improved identification method for infrared interactive touch spot - Google Patents

Abstract

Disclosed is an improved identification method for an infrared interactive touch spot. The improved identification method for the infrared interactive touch spot comprises the following steps: (1) coordinate setting is carried out for a camera based on consistency of random sampling; five groups of point correspondence relationships are randomly sampled in a set S, a projection matrix HC corresponding to the five groups of point correspondence relationships is counted out, when a consistent set CC of the HC surpasses a preset threshold value T, then the current point correspondence relationships in the consistent set meet requirements of ideal point correspondence relationships; otherwise, the step is repeated until the consistent set meets the requirements; according to the point correspondence relationships in the consistent set, a final projection matrix H is counted out through direct linear transformation; (2) screen border touch spot detection is carried out through a dynamic parameter method.

Description

A kind of infrared interactive touch spot recognition methods of improvement

Technical field

The present invention relates to a kind of contact identification, particularly a kind of contact identification being applicable to infrared multi-point touch interactive system.

Background technology

Multi-touch interaction system, as a kind of novel man-machine interactive system, to be reached to image applications through several fingers by computer user when not having conventional input device and control.This brand-new interactive mode establishes the natural interactive interface of human and computer, achieve multiple spot, multi-user at one time directly and environment carry out interaction, enhance Consumer's Experience.For making user obtain good interactive experience, need must carry out contact tracking and identification accurate, rapidly, therefore contact recognition technology is the key that interactive system realizes.

Contact recognition technology at present based on computer vision mainly contains: laser plane technology (LLP), frustrated total internal reflection (FTIR) technology and back infrared illumination (DI) technology.LLP sets up simply, equipment cheaply, not needs the advantages such as locking device, but do not have real pressure sensitivity, and is easy to the situation that occurs that contact stops.FTIR limits due to its design feature, is suitable only for the situation individual desktop only having finger or pen input, needs compatible layer and the sign of None-identified object.Characteristics such as DI sets up simply with environment, the kind of identifiable design contact is many, the favorable expandability of multi-screen splicing and be subject to the welcome of numerous researcher, current DI technology becomes the first-selection of multiple point touching developer already.

Due to the impact of external environmental light and noise, the collection image based on DI technology has hot spot, and infrared light is according to being difficult to evenly, even if take multi-cam to splice the recognition effect being also difficult to obtain at display screen edge.Under this specific environment, existing contact identifies that solution cannot meet the accuracy requirement of identification, needs to adopt new method head it off.

Summary of the invention

Object of the present invention is just the defect solving existing DI technology, designs, studies a kind of improving one's methods based on back infrared illumination technology identification contact.Utilize stochastic sampling consistance to position camera coordinate, error can be reduced greatly, and adopt dynamic parameter to realize screen border contact to detect, can the contact of successfully recognition screen corner, can processing speed be improved.

Technical scheme of the present invention is: infrared (back infrared illumination) interactive touch spot recognition methods of a kind of improvement, is characterized in that comprising the following steps:

(1) based on stochastic sampling consistance to camera coordinate setting;

(1-1) set of the two plane corresponding relations setting S to be formed as projector all in position fixing process and camera pixel, wherein contains the point that may there is error right;

Projective transformation formula x _i'=Hx _i(formula 1)

Wherein x _iand x _i' be the coordinate sequence of two-dimensional points between a series of two given planes, H is projective transform matrix;

(1-2) stochastic sampling 5 groups of point correspondences from S set, calculate this 5 groups of point correspondences according to (formula 1) corresponding projection matrix H _c, initialization H _ccorresponding consistent collection C _c0=0;

(1-3) point correspondences all in S is calculated and carry out following judgement:

(1-3-1) H is calculated _cx _i;

If (1-3-2) || H _cx _i-x _i' || < σ, then C _ci+1=C _ci+ 1, wherein σ is allowable error;

If (1-4) H _cconsistent collection C _cexceed the threshold value T preset, then think that the current consistent point correspondence concentrated all meets the requirement of ideal point corresponding relation; Otherwise, repeat above-mentioned steps, until consistent energy collecting enough satisfies condition;

(1-5) corresponding according to consistent centrostigma relation, utilizes direct linear transformation to obtain final projection matrix H;

(2) dynamic parameter method is adopted to carry out the detection of screen border contact

For each pixel determination image enhancement coefficient A _{(X, Y)}with brightness of image threshold value T _{(X, Y)}, this needs to set up brightness of image B _{(X, Y)}with A _{(X, Y)}, T _{(X, Y)}corresponding relation; Adopt the dynamic programming method based on parameter matrix, can B be drawn _{(X, Y)}with A _{(X, Y)}, T _{(X, Y)}meet following relationship:

$A_{(X,Y)}=A_{\max }-(B_{(X,Y)}-B_{\min })\&times;\frac{A_{\max }-A_{\min }}{B_{\max }-B_{\min }}$ (formula 2)

$T_{(X,Y)}=T_{\min }-(B_{(X,Y)}-B_{\min })\&times;\frac{T_{\max }-T_{\min }}{B_{\max }-B_{\min }}$ (formula 3)

Wherein B _max, B _minmaximal value and the minimum value of the brightness of desktop pixels point, A _max, A _minthe minimum and maximum image enhancement coefficient allowed, T _max, T _minbe the minimum and maximum brightness of image threshold value allowed, these three kinds of values of setting are set according to desktop environment;

(3) denoising of dual threshold Binarization methods is adopted

(3-1) two brightness of image threshold value T are set _upper, T _lower(T _upper>T _lower), maximal value and the minimum value of the desktop pixels point brightness after binaryzation are B _upper, B _lower(B _upper>B _lower);

(3-2) each pixel of traversing graph picture is started from position (0,0) place;

(3-3) as the brightness value B of pixel (x, y) _{(X, Y)}>T _uppertime, mark this brightness value B _{(X, Y)}=B _upper, forward to (3-4); Otherwise mark this brightness value B _{(X, Y)}=B _lower; If to image base, then return;

(3-4) for pixel (x+i, y+j) in this eight neighborhood, wherein (i, j=0, ± 1; i ²+ j ²≠ 0), if B _{(x+i, y+j)}> T _lower, then B _{(x+i, y+j)}=B _upper, otherwise B _{(x+i, y+j)}=B _lower; Recurrence (3-4) is performed for point (x+i, y+j) until do not have the value of luminance point to be greater than T around _lowerpoint, forward to (3-3);

(4) radial symmetry method is adopted to carry out contact detection and Identification:

First to pixel (original image) gradient operator compute gradient figure, then respectively the every bit in gradient map is calculated to accumulation direction mapping graph and the amplitude mapping graph of certain limit, obtain final radial symmetry intensity level SN finally by convolution summation.

Specific algorithm is as follows:

(4-1) input the pixel binary picture split, by horizontal and vertical sobel gradient operator convolution, draw the gradient map G of this both direction;

(4-2) minimum and cumulative maximum radius is set according to the mean distance of finger contact on camera and projector, min_r < n < max_r, initialization accumulation related pixel point direction mapping graph O _nwith amplitude mapping graph M _n;

(4-3) the radial symmetry result S of each contact _nand meet following relationship between mapping graph:

S _n=F _n* A _n(formula 4)

$F_n=\frac{M_n}{k_n}{\left(\frac{O_n}{k_n}\right)}^{\mathrm{\&alpha;}}$ Retain (formula 5)

Wherein A _nfor the dimensional Gaussian window of fixed measure, (formula 5) is F _ncomputing formula, α is radial controling parameters, k _nfor scale factor, the two is for normalization different radii yardstick;

(4-4) to each n in minimum and maximum radius: calculate convolution S according to (formula 4) _n; (4-5) to S _nadd up and try to achieve SN.

Beneficial effect of the present invention is:

1. utilize stochastic sampling consistance to position camera coordinate, error can be reduced greatly, and radial distortion is little and tangential distortion is less, automatically can complete, not need manual intervention, and then reach good mapping accuracy.

2. adopt dynamic parameter to realize screen border contact to detect, can the contact of successfully recognition screen corner, also the track disconnection phenomenon occurred when finger streaks different touch area can not be produced, simultaneously because the mobilism situation of this algorithm with parameter has nothing to do, only calculate when program starts, therefore can improve processing speed.

3. when adopting dual threshold Binarization methods to carry out denoising to image, just bright spot is thought when brightness value is greater than 24, and then recursively surrounding brightness value is greater than the pixel of 10 as bright spot, so just the interference of noise can both be eliminated, more point can be judged to be bright spot again simultaneously, after being conducive to, judge effective contact.

4., when adopting radial symmetry method contact to identify, utilize the features of shape of finger, the suspicious points of similar for contact region finger tip is accurately got rid of, define contact detection method that is stable, robust, expense is little, sets up simple, can keep good reliability under Low SNR.Other advantages of the present invention and effect also can continue to embody in an embodiment.

Accompanying drawing explanation

Fig. 1---contact recognizer process flow diagram.

The geometric maps procedure chart of Fig. 2---projector camera chain.

Fig. 3---physics cross-hatch pattern.

Fig. 4---camera coordinates locating effect figure.

Fig. 5---each pixel brightness value, threshold value, enhancing coefficient figure.

Fig. 6---adopt the recognition effect figure of dynamic parameter method.Fig. 6 (a) is original image, has indicated finger touch area, respectively in desktop-centric position and corner.Fig. 6 (b) is the image after dynamic parameter process.

Fig. 7---Fig. 7 (a) and Fig. 7 (b) is spot zone each brightness value pixel quantity statistical graph.Fig. 7 (a) and Fig. 7 (b) all adopt brightness value be 10 threshold value, pixel brightness value being greater than 10 is labeled as bright spot, be then identified as bright spot after having subregion binaryzation.

Fig. 8---two kinds of Binarization methods recognition effect comparison diagrams.Fig. 8 (a) is the image after adopting simple binaryzation, after Fig. 8 (b) adopts dual threshold binaryzation, and the image of the noise before eliminating.

Fig. 9---radial symmetry transform algorithm block diagram.

Figure 10---(a) and (b) radial symmetry finger tip Detection results figure is the detection of radial symmetry method to similar circular object.

Embodiment

Below in conjunction with drawings and Examples, technical solutions according to the invention are further elaborated.

Fig. 1 is algorithm flow chart of the present invention.In FIG, first carry out camera coordinate setting to screen when program starts, calibration obtains, accurately after coordinate position, starting to collect background frames, then continuously acquired image frames subtracting each other with background frames, obtains error image.After the process such as, high-pass filtering smoothing to this error image, image enhaucament, dynamic parameter method is adopted to carry out the detection of screen border contact, carry out binaryzation denoising again, thus the image comprising a lot of spot zone can be obtained, adopt radial symmetry method to carry out contact detection and Identification, after the Effective judgement of contact, send contact information to upper procedure process.

(1) camera coordinate setting

Camera coordinate setting is the basis of camera subsequent operation, and the geometric maps process of projector and camera chain as shown in Figure 2.Computer input to certain 1 M (X, Y) for the treatment of in projects images P of projector, in a P place imaging and through screen reflection when projecting on projecting apparatus screen.When camera being shot is caught, I (R, C) pixel in synthetic image.The present invention adopts based on stochastic sampling consistance camera coordinate setting, use physics gridiron pattern (7 × 5 as shown in Figure 3, have 35 points), the known checkerboard grid of projector projects, video camera photographs the scene containing projection figure, the mapping relations of target area in perspective view to scene graph are solved by projective transformation, calculate the inverse transformation of scene graph to perspective view simultaneously, the result that calculating switches back to, with the difference between actual result, then utilizes stochastic sampling coherence method to obtain least error point pair.Fig. 4 is the chessboard of the projector projects that camera captures, and calculate mapping relations according to projective transform matrix, green grid is the reposition of original projection figure in scene graph in PC, although can see that the image of camera and projector projects is not parallel, cause geometric deformation, but map set up still very accurately.

(2) screen border contact is detected

In tradition DI mode, to be that uneven illumination is even cause the contact signal intensity of desktop corner very low and None-identified for main problem.Given this problem, the present invention adopts dynamic parameter method to carry out the detection of screen border contact.For this reason, the pixel brightness value of analysis chart inconocenter 310 × 10 pixel region, arranges the contact region of 31 10 × 10 pixels, arranges suitable threshold value and strengthen coefficient to reach the object identifying contact to each contact region.In Figure 5, dotted line is the brightness value of each pixel, can find out that both sides are darker; Dual slope is the enhancing coefficient of each pixel, contrary with brightness value, and it is larger that both sides strengthen coefficient; Single broken line is the threshold value of each pixel, consistent with the change of brightness.Corresponding value is set according to the relation of brightness value, image enhancement coefficient and threshold value.By this method, can the contact of successfully identification desktop corner, also can not produce the track disconnection phenomenon occurred when finger streaks different contact region.Fig. 6 (a) is original image, has indicated finger touch area, respectively in desktop-centric position and corner.Fig. 6 (b) is the image after dynamic parameter process, can find out the contact having identified corner.

(3) dual threshold binaryzation

Noise is mainly derived from two aspects: one is the sudden change of ambient lighting, and arm when being finger touch desktop is on the impact of imaging.Around statistical study the same area contact and noise each brightness value of 20 × 20 pixel region pixel quantity after find: if Fig. 7 (a) and Fig. 7 (b) all adopt brightness value be 10 threshold value, pixel brightness value being greater than 10 is labeled as bright spot, be then identified as bright spot after having subregion binaryzation; According to brightness value 24 as threshold value, although can the noise in nonrecognition Fig. 7 (b) be bright spot, the spot zone area that Fig. 7 (a) recognizes be very little, is unfavorable for follow-uply determining whether effective contact and identifying contact configuration; If adopt two threshold values, just think bright spot when brightness value is greater than 24, and then recursively surrounding brightness value is greater than the pixel of 10 as bright spot, just both can eliminate the interference of noise, more point can be judged to be bright spot again simultaneously, after being conducive to, judge effective contact.Dual threshold Binarization methods is adopted to effectively eliminate noise.Fig. 8 (a) is the image after adopting simple binaryzation, has a small amount of environment noise in image, after adopting dual threshold binaryzation, and the noise (Fig. 8 (b) circle mark place) before eliminating.

(4) radial symmetry method

When finger moves in the plane, its shape facility roughly remains unchanged, and whole finger tip can regard the combination of parallel lines and circular primitive as.If the target area that binaryzation obtains is 1, background area is 0, and fingertip area has obvious feature: (1) finger tip end is similar to circle, and its diameter is finger width; (2), in the region of search centered by the finger tip circumscribed circle center of circle, finger tip is surrounded by one section of non-1 pixel occupied the majority and the enclosed region that the non-zero pixel accounting for fraction is formed.

The present invention proposes to utilize radial symmetry method first to do pre-service to the hand foreground area split, and then obtain gradient map and carry out radial symmetry transform obtaining possible fingertip location, finally utilize the colouring information of finger to get rid of, whole process can describe with Fig. 9.

By changing shade of gray direction, can detect for bright symmetrical region and dark symmetrical region respectively, supposing that the positive dirction of shade of gray is from inferring to bright, only just need can be detected several candidate points of sub-circular by its corresponding map point location in compute gradient direction.

But, in actual use, probably there is the circular object that background exists and skin color is similar.Meanwhile, also there is the position that some may be judged to the similar circle of finger tip in staff itself.Therefore, also need to get rid of certain candidate point.Analyze finger shape can see, because finger is for slender rod shaped, is therefore the circle in the center of circle with finger tip, can runs through finger, the radian crossing with finger is in certain limit.For independent round dot, this circle there will not be crossing situation; For the joint that digital flexion protrudes, this radian can be greater than common finger.Accordingly, we can do further screening to the candidate's finger tip filtered out.

Radian is relevant with the thickness chosen and point of detection radius of a circle, and according to finger tip round diameter, the size of the width d also namely pointed, determines an adaptive detection radius of circle R.The circular fan-shaped angle formed that overlaps of finger and detection is represented, as shown in (formula 6) with θ.The θ of normal finger is distributed between 30 degree to 60 degree, is greater than this scope, or coincidence scope is the candidate point of 0, can get rid of.

$\mathrm{\&theta;}=2\&times;\arcsin \left(\frac{d}{2R}\right)$ (formula 6)

Can see the detection of radial symmetry method to similar circular object from Figure 10 (a) (b), and the exclusion process of candidate point is all very effective.

The scope of request protection of the present invention is not limited only to the description of this embodiment.

Claims (1)

1. the infrared interactive touch spot recognition methods improved, is characterized in that comprising the following steps:

(1) based on stochastic sampling consistance to camera coordinate setting;

(1-1) set of the two plane corresponding relations setting S to be formed as projector all in position fixing process and camera pixel, wherein contains the point that may there is error right;

Projective transformation formula x ' _i=Hx _i(formula 1)

Wherein x _iwith x ' _ibe the coordinate sequence of two-dimensional points between a series of two given planes, H is projective transform matrix;

(1-2) stochastic sampling 5 groups of point correspondences from S set, calculate this 5 groups of point correspondences according to (formula 1) corresponding projection matrix H _c, initialization H _ccorresponding consistent collection C _c0=0;

(1-3) point correspondences all in S is calculated and carry out following judgement:

(1-3-1) H is calculated _cx _i;

If (1-3-2) || H _cx _i-x ' _i|| < σ, then C _ci+1=C _ci+ 1, wherein σ is allowable error;

If (1-4) H _cconsistent collection C _cexceed the threshold value T preset, then think that the current consistent point correspondence concentrated all meets the requirement of ideal point corresponding relation; Otherwise, repeat above-mentioned steps, until consistent energy collecting enough satisfies condition;

(1-5) corresponding according to consistent centrostigma relation, utilizes direct linear transformation to obtain final projection matrix H;

(2) dynamic parameter method is adopted to carry out the detection of screen border contact

For each pixel determination image enhancement coefficient A _{(X, Y)}with brightness of image threshold value T _{(X, Y)}, this needs to set up brightness of image B _{(X, Y)}with A _{(X, Y)}, T _{(X, Y)}corresponding relation; Adopt the dynamic programming method based on parameter matrix, can B be drawn _{(X, Y)}with A _{(X, Y)}, T _{(X, Y)}meet following relationship:

$A_{(X,Y)}=A_{\max }-(B_{(X,Y)}-B_{\min })\&times;\frac{A_{\max }-A_{\min }}{B_{\max }-B_{\min }}$ (formula 2)

$T_{(X,Y)}=T_{\min }-(B_{(X,Y)}-B_{\min })\&times;\frac{T_{\max }-T_{\min }}{B_{\max }-B_{\min }}$ (formula 3)

Wherein B _max, B _minmaximal value and the minimum value of the brightness of desktop pixels point, A _max, A _minthe minimum and maximum image enhancement coefficient allowed, T _max, T _minbe the minimum and maximum brightness of image threshold value allowed, these three kinds of values of setting are set according to desktop environment;

(3) denoising of dual threshold Binarization methods is adopted

(3-1) two brightness of image threshold value T are set _upper, T _lower(T _upper>T _lower), maximal value and the minimum value of the desktop pixels point brightness after binaryzation are B _upper, B _lower(B _upper>B _lower);

(3-2) each pixel of traversing graph picture is started from position (0,0) place;

(3-3) as the brightness value B of pixel (x, y) _{(X, Y)}>T _uppertime, mark this brightness value B _{(X, Y)}=B _upper, forward to (3-4); Otherwise mark this brightness value B _{(X, Y)}=B _lower; If to image base, then return;

(3-4) for pixel (x+i, y+j) in this eight neighborhood, wherein (i, j=0, ± 1; i ²+ j ²≠ 0), if B _{(x+i, y+j)}> T _lower, then B _{(x+i, y+j)}=B _upper, otherwise B _{(x+i, y+j)}=B _lower; Recurrence (3-4) is performed for point (x+i, y+j) until do not have the value of luminance point to be greater than T around _lowerpoint, forward to (3-3);

(4) radial symmetry method is adopted to carry out contact detection and Identification:

First to original image pixels point gradient operator compute gradient figure, then respectively the every bit in gradient map is calculated to accumulation direction mapping graph and the amplitude mapping graph of certain limit, obtain final radial symmetry intensity level SN finally by convolution summation.