CN103092439A - Improved identification method for infrared interactive touch spot - Google Patents

Abstract

Disclosed is an improved identification method for an infrared interactive touch spot. The improved identification method for the infrared interactive touch spot comprises the following steps: (1) coordinate setting is carried out for a camera based on consistency of random sampling; five groups of point correspondence relationships are randomly sampled in a set S, a projection matrix HC corresponding to the five groups of point correspondence relationships is counted out, when a consistent set CC of the HC surpasses a preset threshold value T, then the current point correspondence relationships in the consistent set meet requirements of ideal point correspondence relationships; otherwise, the step is repeated until the consistent set meets the requirements; according to the point correspondence relationships in the consistent set, a final projection matrix H is counted out through direct linear transformation; (2) screen border touch spot detection is carried out through a dynamic parameter method.

Description

A kind of improved infrared mutual contact recognition methods

Technical field

The present invention relates to the recognition methods of a kind of contact, particularly a kind of contact recognition methods that is applicable to the infrared multi-point touch interactive system.

Background technology

The multiple point touching interactive system is as a kind of novel man-machine interactive system, can be in the situation that do not have traditional input equipment to see through several fingers by computer user to reach to image applications and control.This brand-new interactive mode has been set up the interactive interface of human and computer nature, realized multiple spot, multi-user at one time directly and environment carry out interaction, strengthened user's experience.For making the user obtain good interactive experience, need accurate, the rapid contact of must carrying out to follow the tracks of and identify, so the contact recognition technology is the key that interactive system realizes.

Contact recognition technology based on computer vision mainly contains at present: laser plane technology (LLP), frustrated total internal reflection (FTIR) technology and back infrared illumination (DI) technology.LLP sets up simply, equipment cheaply, does not need the advantages such as locking device, but there is no real pressure sensitivity, and is easy to occur the situation that the contact stops.FTIR is only suitable for only having the situation of finger or pen input due to its design feature restriction on individual desktop, need the sign of compatible layer and None-identified object.The characteristic such as the kind that DI set up simply, can identify the contact with environment is many, the favorable expandability of multi-screen splicing and be subject to numerous researchers' welcome, the DI technology becomes multiple point touching developer's first-selection already at present.

Due to the impact of external environment light and noise, have hot spot based on the collection image of DI technology, and infrared light is according to being difficult to evenly, even take the multi-cam splicing also to be difficult to the recognition effect that obtains at the display screen edge.Under this specific environment, existing contact identification solution can't satisfy the accuracy requirement of identification, needs to adopt the new method head it off.

Summary of the invention

Purpose of the present invention just is to solve the defective of existing DI technology, design, a kind of improving one's methods based on back infrared illumination technology identification contact of research.Utilize the stochastic sampling consistance that the camera coordinate is positioned, can reduce greatly error, and adopt dynamic parameter to realize screen border contact detection, successfully the contact of recognition screen corner, can improve processing speed.

Technical scheme of the present invention is: a kind of improved infrared (back infrared illumination) mutual contact recognition methods is characterized in that comprising the following steps:

(1) based on the stochastic sampling consistance to camera coordinate setting;

(1-1) establish the set that S is the two plane corresponding relations that form of projector all in position fixing process and camera pixel, wherein comprised and to exist the point of error right;

Projective transformation formula x _i'=Hx _i(formula 1)

X wherein _iAnd x _i' be the coordinate sequence of two-dimensional points between a series of given two planes, H is projective transform matrix;

(1-2) 5 groups of point correspondences of stochastic sampling from S set, calculate this 5 groups of point correspondences according to (formula 1)

Corresponding projection matrix H _C, initialization H _CCorresponding consistent collection C _C0=0;

(1-3) calculate all point correspondences in S

And carry out following judgement:

(1-3-1) calculate H _Cx _i;

If (1-3-2) || H _Cx _i-x _i' ||＜σ, C _Ci+1=C _Ci+ 1, wherein σ is allowable error;

If (1-4) H _CConsistent collection C _CSurpass predefined threshold value T, think that the current consistent point correspondence of concentrating all satisfies the requirement of ideal point corresponding relation; Otherwise, repeat above-mentioned steps, until consistent energy collecting enough satisfies condition;

(1-5) according to consistent relation corresponding to centrostigma, utilize the direct linear transformation to obtain final projection matrix H;

(2) adopt the dynamic parameter method to carry out screen border contact detection

For each pixel is determined image enhancement coefficient A _{(X, Y)}With brightness of image threshold value T _{(X, Y)}, this need to set up brightness of image B _{(X, Y)}With A _{(X, Y)}, T _{(X, Y)}Corresponding relation; Employing can draw B based on the dynamic programming method of parameter matrix _{(X, Y)}With A _{(X, Y)}, T _{(X, Y)}Satisfy following relationship:

$A_{(X,Y)}=A_{\max }-(B_{(X,Y)}-B_{\min })\×\frac{A_{\max }-A_{\min }}{B_{\max }-B_{\min }}$ (formula 2)

$T_{(X,Y)}=T_{\min }-(B_{(X,Y)}-B_{\min })\×\frac{T_{\max }-T_{\min }}{B_{\max }-B_{\min }}$ (formula 3)

B wherein _max, B _minMaximal value and the minimum value of the brightness of desktop pixel, A _max, A _minThe minimum and maximum image enhancement coefficient that allows, T _max, T _minBe the minimum and maximum brightness of image threshold value that allows, arrange according to desktop environment and set these three kinds of values;

(3) adopt the denoising of dual threshold Binarization methods

(3-1) two brightness of image threshold value T are set _Upper, T _Lower(T _UpperT _Lower), maximal value and the minimum value of the desktop pixel brightness after binaryzation are B _Upper, B _Lower(B _UpperB _Lower);

(3-2) (0,0) locates to begin each pixel of traversing graph picture from the position;

(3-3) as the brightness value B of pixel (x, y) _{(X, Y)}T _UpperThe time, this brightness value B of mark _{(X, Y)}=B _Upper, forward (3-4) to; Otherwise this brightness value B of mark _{(X, Y)}=B _LowerIf to the image bottom, return;

(3-4) for pixel (x+i, y+j), wherein (i, j=0, ± 1 in these point eight neighborhoods; i ²+ j ²≠ 0), if B _{(x+i, y+j)}＞T _Lower, B _{(x+i, y+j)}=B _Upper, otherwise B _{(x+i, y+j)}=B _LowerFor point (x+i, y+j) carry out recurrence (3-4) until the value that there is no luminance point on every side greater than T _LowerPoint, forward (3-3) to;

(4) adopt the radial symmetry method to carry out the contact detection and Identification:

At first to pixel (original image) gradient operator compute gradient figure, then respectively the every bit on gradient map is calculated accumulation direction mapping graph and the amplitude mapping graph of certain limit, sue for peace by convolution at last and obtain final radial symmetry intensity level SN.

Specific algorithm is as follows:

(4-1) good pixel binary picture is cut apart in input, with horizontal and vertical sobel gradient operator convolution, draws the gradient map G of this both direction;

(4-2) mean distance according to finger contact on camera and projector arranges minimum and cumulative maximum radius, min_r＜n＜max_r, initialization accumulation related pixel point direction mapping graph O _nWith amplitude mapping graph M _n

(4-3) radial symmetry of each contact S as a result _nAnd satisfy following relationship between mapping graph:

S _n=F _n* A _n(formula 4)

$F_n=\frac{M_n}{k_n}{\left(\frac{O_n}{k_n}\right)}^{\mathrm{\α}}$ Keep (formula 5)

A wherein _nBe the dimensional Gaussian window of fixed measure, (formula 5) is F _nComputing formula, α is for radially controlling parameter, k _nBe scale factor, both be used for normalization different radii yardstick;

(4-4) to each n in minimum and maximum radius: calculate convolution S according to (formula 4) _n(4-5) to S _nThe cumulative SN that tries to achieve.

Beneficial effect of the present invention is:

1. utilize the stochastic sampling consistance that the camera coordinate is positioned, can reduce greatly error, and radial distortion is little and tangential distortion is less, can automatically complete, and does not need manual intervention, and then reaches good mapping precision.

2. adopt dynamic parameter to realize screen border contact detection, the contact of recognition screen corner successfully, can not produce the interrupted phenomenon of track that occurs when finger streaks different touch area yet, simultaneously because this algorithm is irrelevant with the mobilism situation of parameter, only calculate when program start, therefore can improve processing speed.

3. when adopting the dual threshold Binarization methods to carry out denoising to image, just think bright spot greater than 24 the time when brightness value, and then recursively with the surrounding brightness value greater than 10 pixel as bright spot, so just both can eliminate the interference of noise, simultaneously can be judged to be bright spot to more point again, judge effective contact after being conducive to.

4. when employing radial symmetry method contact is identified, utilize the features of shape of finger, the suspicious points of the similar finger tip of contact region is accurately got rid of, formed contact detection method stable, robust, expense is little, sets up simply, can keep good reliability under Low SNR.Other advantages of the present invention and effect also can continue to embody in an embodiment.

Description of drawings

Fig. 1---contact recognizer process flow diagram.

The geometric maps procedure chart of Fig. 2---projector camera chain.

Fig. 3---physics cross-hatch pattern.

Fig. 4---camera coordinates locating effect figure.

Fig. 5---each pixel brightness value, threshold value, enhancing coefficient figure.

Fig. 6---adopt the recognition effect figure of dynamic parameter method.Fig. 6 (a) is original image, has indicated finger touch regional, respectively in desktop center and corner.Fig. 6 (b) is the image after processing for dynamic parameter.

Fig. 7---Fig. 7 (a) and Fig. 7 (b) are each brightness value pixel quantity statistical graph of spot zone.It is 10 threshold value that Fig. 7 (a) and Fig. 7 (b) adopt brightness value, and brightness value is labeled as bright spot greater than 10 pixel, is identified as bright spot after the subregion binaryzation is arranged.

Fig. 8---two kinds of Binarization methods recognition effect comparison diagrams.Fig. 8 (a) is the image after the simple binaryzation of employing, after Fig. 8 (b) adopts the dual threshold binaryzation, and the image of the noise before having eliminated.

Fig. 9---radial symmetry mapping algorithm block diagram.

Figure 10---(a) and (b) the radial symmetry finger tip detects design sketch, is that the radial symmetry method is to the detection of similar circular object.

Embodiment

Below in conjunction with drawings and Examples, technical solutions according to the invention are further elaborated.

Fig. 1 is algorithm flow chart of the present invention.In Fig. 1, at first screen is carried out camera coordinate setting during program start, calibration begins to collect background frames after obtaining accurately coordinate position, and then acquired image frames and subtracting each other with background frames continuously, obtain error image.By this error image is carried out smoothly, after high-pass filtering, figure image intensifying etc. process, adopt the dynamic parameter method to carry out screen border contact detection, carry out again the binaryzation denoising, thereby can obtain comprising the image of a lot of spot zones, adopt the radial symmetry method to come the contact detection and Identification, send contact information to upper procedure after the validity judgement through the contact and process.

(1) camera coordinate setting

Camera coordinate setting is the basis of camera subsequent operation, and the geometric maps process of projector and camera chain as shown in Figure 2.Computer input is to certain 1 M (X, Y) in projects images P for the treatment of of projector, when projecting on projecting apparatus screen in P place's imaging and through screen reflection.When being caught by video camera, I (R, C) pixel in synthetic image.The present invention adopts based on the stochastic sampling consistance camera coordinate setting, use physics gridiron pattern (7 * 5 as shown in Figure 3, have 35 points), the known chessboard grid of projector projects, video camera photographs the scene that contains projection figure, find the solution the mapping relations of perspective view target area in the scene graph by projective transformation, calculate simultaneously scene graph to the inverse transformation of perspective view, then the result that computational transformation returns utilizes the stochastic sampling coherence method to obtain least error point right with the difference between actual result.Fig. 4 is the chessboard of the projector projects that captures of camera, and calculate mapping relations according to projective transform matrix, green grid is the reposition of original projection figure in scene graph in PC, although can see that the image of camera and projector projects is not parallel, caused geometric deformation, but mapping set up still very accurately.

(2) the screen border contact is detected

In tradition DI mode, main problem is the very low and None-identified of the even contact signal intensity that causes the desktop corner of uneven illumination.Given this problem, the present invention adopts the dynamic parameter method to carry out the screen border contact and detects.For this reason, the pixel brightness value of analysis image center 310 * 10 pixel regions arranges the contact region of 31 10 * 10 pixels, and each contact region is arranged suitable threshold value and strengthens coefficient to reach the purpose of identification contact.In Fig. 5, dotted line is the brightness value of each pixel, can find out that both sides are darker; Dual slope is the enhancing coefficient of each pixel, and is opposite with brightness value, and it is larger that both sides strengthen coefficient; Single broken line is the threshold value of each pixel, and is consistent with the variation of brightness.Set corresponding value according to the relation of brightness value, image enhancement coefficient and threshold value.By this method, successfully the contact of identification desktop corner, can not produce the interrupted phenomenon of track that occurs when finger streaks different contact region yet.Fig. 6 (a) is original image, has indicated finger touch regional, respectively in desktop center and corner.Fig. 6 (b) is the image after processing for dynamic parameter, can find out the contact of having identified the corner.

(3) dual threshold binaryzation

Noise is mainly derived from two aspects: the one, and the sudden change of ambient lighting, the one, the impact of arm on imaging during the finger touch desktop.Find after the pixel quantity of 20 * 20 each brightness values of pixel region around statistical study the same area contact and noise: be 10 threshold value if Fig. 7 (a) and Fig. 7 (b) adopt brightness value, brightness value is labeled as bright spot greater than 10 pixel, is identified as bright spot after the subregion binaryzation is arranged; If adopt brightness value 24 as threshold value, although the noise in can nonrecognition Fig. 7 (b) is bright spot, the spot zone area that Fig. 7 (a) recognizes is very little, is unfavorable for follow-up effective contact and the identification contact shape of determining whether; If adopt two threshold values, just think bright spot when brightness value greater than 24 the time, and then recursively with the surrounding brightness value greater than 10 pixel as bright spot, just both can eliminate the interference of noise, simultaneously can be judged to be bright spot to more point again, judge effective contact after being conducive to.Adopt the dual threshold Binarization methods effectively to eliminate noise.Fig. 8 (a) is the image after the simple binaryzation of employing, and a small amount of environment noise is arranged in image, after adopting the dual threshold binaryzation, and the noise (Fig. 8 (b) circle mark place) before having eliminated.

(4) radial symmetry method

When finger moved in the plane, its shape facility roughly remained unchanged, and whole finger tip can be regarded the combination of parallel lines and circular primitive as.If the target area that binaryzation obtains is 1, the background area is 0, and fingertip area has obvious feature: (1) finger tip end is similar to circle, and its diameter is finger width; (2) in the region of search centered by the finger tip circumscribed circle center of circle, finger tip is by one section non-1 pixel that occupies the majority and account for the enclosed region that the non-zero pixel of fraction consists of and surround.

The present invention proposes to utilize the radial symmetry method first to do pre-service to cutting apart good hand foreground area, then obtains the gradient map conduct of going forward side by side and obtains possible fingertip location to symmetry transformation, utilizes at last the colouring information of finger to get rid of, and whole process can be described with Fig. 9.

By changing the shade of gray direction, can detect for bright symmetrical region and dark symmetrical region respectively, suppose that the positive dirction of shade of gray from inferring to bright, only need just can detect by its corresponding mapping point of compute gradient direction position several candidate points of sub-circular.

Yet, in actual use, background probably occurs and exist and the similar circular object of skin color.Simultaneously, staff itself also exists some may be judged to the position of the similar circle of finger tip.Therefore, also need get rid of certain candidate point.Analyzing the finger shape can see, for slender rod shaped, therefore the circle take finger tip as the center of circle, can run through finger due to finger, and the radian that intersects with finger is in certain limit.For independent round dot, the situation that this circle can not occur intersecting; For the crooked joint of protruding of finger, this radian can be greater than common finger.Accordingly, we can do further screening to the candidate's finger tip that filters out.

Radian is relevant with the thickness of choosing and pointing of surveying radius of a circle, and according to the finger tip round diameter, the size of the width d that also namely points is determined an adaptive detection radius of circle R.Represent to point and survey the circular fan-shaped angle that forms that overlaps with θ, as shown in (formula 6).The θ of normal finger is distributed in 30 degree between 60 degree, and greater than this scope, or the coincidence scope is 0 candidate point, can get rid of.

$\mathrm{\θ}=2\×\arcsin \left(\frac{d}{2R}\right)$ (formula 6)

Can see the radial symmetry method to the detection of similar circular object from Figure 10 (a) (b), and the exclusion process of candidate point is all very effective.

The scope that the present invention asks for protection is not limited only to the description of this embodiment.

Claims (3)

1. improved infrared mutual contact recognition methods is characterized in that comprising the following steps:

(1) based on the stochastic sampling consistance to camera coordinate setting;

(1-1) establish the set that S is the two plane corresponding relations that form of projector all in position fixing process and camera pixel, wherein comprised and to exist the point of error right;

Projective transformation formula x _i'=Hx _i(formula 1)

X wherein _iAnd x _i' be the coordinate sequence of two-dimensional points between a series of given two planes, H is projective transform matrix;

(1-2) 5 groups of point correspondences of stochastic sampling from S set, calculate this 5 groups of point correspondences according to (formula 1)

Corresponding projection matrix H _C, initialization H _CCorresponding consistent collection C _C0=0;

(1-3) calculate all point correspondences in S

And carry out following judgement:

(1-3-1) calculate H _Cx _i;

If (1-3-2) || H _Cx _i-x _i' ||＜σ, C _Ci+1=C _Ci+ 1, wherein σ is allowable error;

If (1-4) H _CConsistent collection C _CSurpass predefined threshold value T, think that the current consistent point correspondence of concentrating all satisfies the requirement of ideal point corresponding relation; Otherwise, repeat above-mentioned steps, until consistent energy collecting enough satisfies condition;

(1-5) according to consistent relation corresponding to centrostigma, utilize the direct linear transformation to obtain final projection matrix H;

(2) adopt the dynamic parameter method to carry out screen border contact detection

For each pixel is determined image enhancement coefficient A _{(X, Y)}With brightness of image threshold value T _{(X, Y)}, this need to set up brightness of image B _{(X, Y)}With A _{(X, Y)}, T _{(X, Y)}Corresponding relation; Employing can draw B based on the dynamic programming method of parameter matrix _{(X, Y)}With A _{(X, Y)}, T _{(X, Y)}Satisfy following relationship:

$A_{(X,Y)}=A_{\max }-(B_{(X,Y)}-B_{\min })\×\frac{A_{\max }-A_{\min }}{B_{\max }-B_{\min }}$ (formula 2)

$T_{(X,Y)}=T_{\min }-(B_{(X,Y)}-B_{\min })\×\frac{T_{\max }-T_{\min }}{B_{\max }-B_{\min }}$ (formula 3)

B wherein _max, B _minMaximal value and the minimum value of the brightness of desktop pixel, A _max, A _minThe minimum and maximum image enhancement coefficient that allows, T _max, T _minBe the minimum and maximum brightness of image threshold value that allows, arrange according to desktop environment and set these three kinds of values;

(3) adopt the denoising of dual threshold Binarization methods

(3-1) two brightness of image threshold value T are set _Upper, T _Lower(T _UpperT _Lower), maximal value and the minimum value of the desktop pixel brightness after binaryzation are B _Upper, B _Lower(B _UpperB _Lower);

(3-2) (0,0) locates to begin each pixel of traversing graph picture from the position;

(3-3) as the brightness value B of pixel (x, y) _{(X, Y)}T _UpperThe time, this brightness value B of mark _{(X, Y)}=B _Upper, forward (3-4) to; Otherwise this brightness value B of mark _{(X, Y)}=B _LowerIf to the image bottom, return;

(3-4) for pixel (x+i, y+j), wherein (i, j=0, ± 1 in these point eight neighborhoods; i ²+ j ²≠ 0), if B _{(x+i, y+j)}＞T _Lower, B _{(x+i, y+j)}=B _Upper, otherwise B _{(x+i, y+j)}=B _LowerFor point (x+i, y+j) carry out recurrence (3-4) until the value that there is no luminance point on every side greater than T _LowerPoint, forward (3-3) to;

(4) adopt the radial symmetry method to carry out the contact detection and Identification:

At first to pixel (original image) gradient operator compute gradient figure, then respectively the every bit on gradient map is calculated accumulation direction mapping graph and the amplitude mapping graph of certain limit, sue for peace by convolution at last and obtain final radial symmetry intensity level SN.

2. improved infrared mutual contact recognition methods according to claim 1 is characterized in that the specific algorithm of step (4) is as follows:

(4-1) good pixel binary picture is cut apart in input, with horizontal and vertical sobel gradient operator convolution, draws the gradient map G of this both direction;

(4-2) mean distance according to finger contact on camera and projector arranges minimum and cumulative maximum radius, min_r＜n＜max_r, initialization accumulation related pixel point direction mapping graph O _nWith amplitude mapping graph M _n

(4-3) radial symmetry of each contact S as a result _nAnd satisfy following relationship between mapping graph:

Sn=F _n* A _n(formula 4)

$F_n=\frac{M_n}{k_n}{\left(\frac{O_n}{k_n}\right)}^{\mathrm{\α}}$ (formula 5)

A wherein _nBe the dimensional Gaussian window of fixed measure, (formula 5) is F _nComputing formula, α is for radially controlling parameter, k _nBe scale factor, both be used for normalization different radii yardstick;

(4-4) to each n in minimum and maximum radius: calculate convolution S according to (formula 4) _n

(4-5) to S _nThe cumulative SN that tries to achieve.

3. improved infrared mutual contact recognition methods according to claim 1 and 2, is characterized in that according to the finger tip round diameter, and the size of the width d that also namely points is determined an adaptive detection radius of circle R; Represent to point and survey the circular fan-shaped angle that forms that overlaps with θ; The θ of normal finger is distributed in 30 degree between 60 degree, and greater than this scope, or the coincidence scope is 0 candidate point, can get rid of.

Images