CN103713738B - A kind of view-based access control model follows the tracks of the man-machine interaction method with gesture identification - Google Patents

Abstract

The invention discloses a kind of view-based access control model and follow the tracks of the man-machine interaction method with gesture identification, the present invention includes a varifocal high definition photographic head and multiple high definition photographic head.Apparatus of the present invention are arranged on above frame or the frame of screen.The information of wherein varifocal high definition camera collection vision, demarcate again through the position of Visual Observations Observations screen and the image of shooting, the identification of pupil is realized, thus realizing the first step visual tracking of the inventive method finally by the application specific processor of apparatus of the present invention.The second step of the present invention is after visual tracking, and multiple high definition photographic head realize the identification of gesture, i.e. finishing man-machine interaction in the region of Visual Observations Observations screen.Again through photographic head, screen is demarcated, gesture is carried out three-dimensional imaging, and judges position and the action of gesture touch point.Present invention achieves the gesture identification man-machine interactive system with visual tracking, it is provided that the contactless man-machine interaction mode of intelligence.

Description

A kind of view-based access control model follows the tracks of the man-machine interaction method with gesture identification

Technical field

The invention belongs to Artificial technical field of intelligence, follow the tracks of the man-machine interaction method with gesture identification more particularly, to a kind of view-based access control model.

Background technology

The progress of technology makes to become closer to the mutual of natural way alternately between people and computer, " natural interaction " that namely people vigorously advocate.This man-machine interaction mode easily of touching technique has been pushed to numerous fields, except being applied in portable personal digital product, it is also widely used in the fields such as information household appliances, public information, electronic game, office automation devices and industrial equipment.Utilize touching technique, user to have only to use gesture the word on touching screen gently or that icon can be realized as with computer is mutual so that mutual more intuitively convenient between people and machine.

And existing touching technique must flow through human contact's screen thus the process of finishing man-machine interaction.The touching technique of this contact cannot realize the natural interaction with screen when people is away from screen, it is necessary to controls screen by devices such as remote controllers, it is impossible to providing good man-machine interaction experience, therefore above-mentioned touching technique exists limitation in artificial intelligence application.Visual Tracking utilizes the change of vision to replace staff motion on the touchscreen, still can arbitrary region on positioning screen when making people away from screen.Use touch screen by the motion of eyes, decrease many steps, accelerate development and the realization of intelligent man-machine interaction focusing on people.The application of the aspects such as current this technology is also only limitted to eye tracker, recognition of face, is not also applied in touching technique field.

Summary of the invention

Disadvantages described above or Improvement requirement for prior art, the invention provides a kind of view-based access control model and follow the tracks of the man-machine interaction method with gesture identification, its object is to, can arbitrarily have on the screen such as computer liquid crystal displayer of screen characteristics, common liquid crystals screen, projecting apparatus screen, giant display and realize visual tracking, and realize the man-machine interaction mode of Untouched control screen.

For achieving the above object, according to one aspect of the present invention, it is provided that a kind of view-based access control model follows the tracks of the man-machine interaction method with gesture identification, comprises the following steps:

(1) by infrared light supply, it is used for carrying out the varifocal high definition photographic head of visual tracking and multiple high definition photographic head for carrying out gesture identification is arranged on screen frame place；

(2) varifocal high definition camera collection facial image, and the facial image gathered is carried out face contour extraction；

(3) pixel coordinate (u of calculation procedure (2) obtains facial contour middle left and right pupil center_eL,v_eL) and (u_eR,v_eR)；

(4) projection matrix Mel and the Mer of left and right pupil is calculated according to the pixel coordinate of left and right pupil center in facial contour and the coordinate at four angles of screen；

(5) projection matrix Mel and the Mer of the left and right pupil obtained by step (4) and the center pixel coordinate figure of left and right pupil calculate left and right pupil physical coordinates value on screen, and this region corresponding to physical coordinates value is the region that user performs gesture operation:

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}Xel\\ Yel\\ 1\end{array}\right]$

$\left[\begin{array}{c}{u}_{eR}\\ v_{eR}\\ 1\end{array}\right]=Mer\left[\begin{array}{c}Xer\\ Yer\\ 1\end{array}\right]$

Wherein (Xer, Yer) represents right pupil physical coordinates value on screen, and (Xel, Yel) represents left pupil physical coordinates value on screen；

(6) according to the principle of binocular vision, the screen being placed with high definition photographic head is carried out parameter calibration, to obtain projection matrix Ml and the Mr of left and right high definition photographic head respectively；

(7) high definition camera collection user gesture touches the image of screen, and the image gathered is carried out pretreatment, to obtain the gesture of the user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F)；

(8) according to the gesture operation of user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F) and the projection matrix Mr of the projection matrix Ml of left high definition photographic head and right high definition photographic head, and obtain the gesture of user three dimensional space coordinate (x on screen by below equation_f,y_f,z_f), wherein this gesture operation is in the region that the physical coordinates value obtained in above-mentioned steps (5) is corresponding:

$\left[\begin{array}{c}{u}_{1F}\\ v_{1F}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right],\left[\begin{array}{c}{u}_{2F}\\ v_{2F}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right]$

(9) the coordinate z that step (8) obtains is judged_fWhether less than threshold values γ, if z_fLess than γ, it can be determined that user's gesture generation click action, by USB interface by the three dimensional space coordinate (x of finger tip_f,y_f,z_f) output, else process terminates.

Preferably, step (2) includes following sub-step:

(2-1) utilize varifocal high definition camera collection facial image, and with mask method to gather facial image denoising；

(2-2) utilize Sobel operator that the pixel on facial image does gradient conversion, to obtain facial contour.

Preferably, step (3) is specifically, using the left and right pixel coordinate value that Sobel operator obtains left pupil in the facial contour that step (2) obtains is u_LeL、u_HeL, the pixel coordinate value up and down of left pupil is v_LeL、v_HeL, the center pixel coordinate figure (u of left pupil_eL,v_eL) it is (u_LeL+u_HeL/2,v_LeL+v_HeL/ 2), the center pixel coordinate figure (u of right pupil_eR,v_eR) it is (u_LeR+u_HeR/2,v_LeR+v_HeR/ 2), wherein u_LeR、u_HeRFor the left and right pixel coordinate value of right pupil, v_LeR、v_HeRPixel coordinate value up and down for right pupil.

Preferably, step (6) is specifically, utilize Zhang Zhengyou to demarcate and screen is demarcated, to obtain demarcation thing pixel coordinate on the high definition photographic head of left and right, each demarcation thing pixel coordinate (u on the high definition photographic head of left and right_1m,v_1m)、(u_2m,v_2m), wherein m is the number of fixed point, and utilizes below equation to obtain the projection matrix Ml of left high definition photographic head and the projection matrix Mr of right high definition photographic head respectively:

$\begin{array}{cc}\left[\begin{array}{c}{u}_{1m}\\ v_{1m}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]& \left[\begin{array}{c}{u}_{2m}\\ v_{2m}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]\end{array}$

Wherein (x_m,y_m,z_m) for the physical coordinates of circle fixed point.

Preferably, step (7) specifically includes following sub-step:

(7-1) left and right high definition photographic head gathers user's gesture respectively and touches the image of screen, and is subtracted each other by pixel in the image corresponding point of the image collected and initialization frame, to form new image；

(7-2) the new image that step (7-1) is obtained carries out image denoising；

(7-3) utilize Sobel operator that the pixel on image does gradient conversion, to obtain edge detection graph；

(7-4) pixel on left and right high definition photographic head is carried out K curvature differentiation by the edge detection graph obtained according to step (7-3), to obtain the gesture of the user imager coordinate (u on left and right high definition photographic head_1F,v_1F) and (u_2F,v_2F)。

It is another aspect of this invention to provide that provide a kind of view-based access control model to follow the tracks of the man-machine interaction method with gesture identification, comprise the following steps:

(1) by infrared light supply, it is used for carrying out the varifocal high definition photographic head of visual tracking and multiple high definition photographic head for carrying out gesture identification is arranged on screen frame place；

(2) varifocal high definition camera collection facial image, and the facial image gathered is carried out face contour extraction；

(3) pixel coordinate (u of calculation procedure (2) obtains facial contour middle left and right pupil center_eL,v_eL) and (u_eR,v_eR)；

(4) projection matrix Mel and the Mer of left and right pupil is calculated according to the pixel coordinate of left and right pupil center in facial contour and the coordinate at four angles of screen；

(5) projection matrix Mel and the Mer of the left and right pupil obtained by step (4) and the center pixel coordinate figure of left and right pupil calculate left and right pupil physical coordinates value on screen, and this region corresponding to physical coordinates value is the region that user performs gesture operation:

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}Xel\\ Yel\\ 1\end{array}\right]$

$\left[\begin{array}{c}{u}_{eR}\\ v_{eR}\\ 1\end{array}\right]=Mer\left[\begin{array}{c}Xer\\ Yer\\ 1\end{array}\right]$

Wherein (Xer, Yer) represents right pupil physical coordinates value on screen, and (Xel, Yel) represents left pupil physical coordinates value on screen；

(6) according to the principle of binocular vision, the screen being placed with high definition photographic head is carried out parameter calibration, to obtain projection matrix Ml and the Mr of left and right high definition photographic head respectively；

(7) high definition camera collection user gesture touches the image of screen, and the image gathered is carried out pretreatment, to obtain the gesture of the user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F)；

(8) when user slides touch screen, according to the gesture operation of user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F) and the projection matrix Mr of the projection matrix Ml of left high definition photographic head and right high definition photographic head, and gesture three dimensional space coordinate (x of the first frame finger tip on screen of user is obtained by below equation_f1,y_f1,z_f1), wherein this gesture operation is in the region that the physical coordinates value obtained in above-mentioned steps (5) is corresponding:

$\left[\begin{array}{c}{u}_{1F}\\ v_{1F}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_{f1}\\ y_{f1}\\ z_{f1}\\ 1\end{array}\right],\left[\begin{array}{c}{u}_{2F}\\ v_{2F}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_{f1}\\ y_{f1}\\ z_{f1}\\ 1\end{array}\right]$

(9) step (8) is repeated, to obtain the three dimensional space coordinate (x of follow-up D-1 frame finger tip image_f2,y_f2,z_f2) ..., (x_fD,y_fD,z_fD), wherein D represents that user slides the frame number of finger tip image collected when touching screen, thus obtaining gesture sliding trace on screen, is exported track by USB interface.

In general, by the contemplated above technical scheme of the present invention compared with prior art, it is possible to obtain following beneficial effect:

(1) present invention achieves and any screen (includes liquid crystal display screen, projecting apparatus screen or other screens etc.) realize that there is visual tracking location and the function of contactless touch；

(2) present invention uses simply, accurate positioning, it is simple to install.

Accompanying drawing explanation

Fig. 1 is the flow chart that view-based access control model of the present invention follows the tracks of the man-machine interaction method with gesture identification.

Fig. 2 is the schematic diagram of face contour detecting of the present invention.

Fig. 3 is the schematic diagram of visual tracking of the present invention.

Fig. 4 is the outline drawing of the device that gesture identification of the present invention uses.

Fig. 5 is the front view of the present invention.

Fig. 6 is the side view of screen of the present invention.

Fig. 7 is that the present invention demarcates thing schematic diagram.

Fig. 8 is that gesture of the present invention touches click schematic diagram.

Fig. 9 is gesture slip schematic diagram of the present invention.

Detailed description of the invention

In order to make the purpose of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein is only in order to explain the present invention, is not intended to limit the present invention.As long as just can be mutually combined additionally, technical characteristic involved in each embodiment of invention described below does not constitute conflict each other.

As it is shown in figure 1, a kind of view-based access control model of the present invention is followed the tracks of and the man-machine interaction method of gesture identification, comprise the following steps:

(1) by infrared light supply, it is used for carrying out the varifocal high definition photographic head of visual tracking and multiple high definition photographic head for carrying out gesture identification is arranged on the optional position of screen frame；In the present embodiment, carrying out the varifocal high definition photographic head of visual tracking, its characteristic is 10 times of zooms, and resolution is 720P, and frame per second is 60 frames/second, and angle lens is at 110 °；Infrared light supply selects the infrared light of 800nm-1200nm wavelength；Carrying out the frame per second of the high definition photographic head of gesture identification be 60 frames/second, resolution is 720P, and angle lens is at 110 °；Screen is the screen of arbitrary dimension or form, including liquid crystal display screen, projecting apparatus screen or other screens) photographic head is placed in the optional position, left and right of screen frame, as shown in Figs. 4-6, exemplarily, choose the center installation infrared light source of screen upper side frame, varifocal high definition photographic head and high definition photographic head, in the present embodiment, use two high definition photographic head, and a varifocal high definition photographic head, it should be understood that the photographic head quantity of the present invention is not limited to this.The present invention passes through infrared light supply as secondary light source.

(2) varifocal high definition camera collection facial image, and the facial image gathered is carried out face contour extraction；As in figure 2 it is shown, this step includes following sub-step:

(2-1) utilize varifocal high definition camera collection facial image, and with mask method to gather facial image denoising；Specifically, the mask of a 3*3 is initially set up $W=\left[\begin{array}{ccc}{w}_1& w_2& w_3\\ w_4& w_5& w_6\\ w_7& w_8& w_9\end{array}\right],$ Assume that on the facial image gathered, the pixel of certain point is a_j,k, wherein j and k represents the position of point on image；Then there is a_j,k=a_j-1,k-1w₁+a_j-1,kw₂+…+a_j,kw₅+…+a_j+1,kw₈+a_j+1,k+1w₉, thus obtaining new pixel a_j,k, in the present embodiment, $W=\frac{1}{9}\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right];$

(2-2) facial image after denoising is carried out rim detection, namely utilize Sobel operator that the pixel on facial image does gradient conversion, to obtain facial contour；Specifically, M sets Sobel operator $S_h=\left[\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ 1& 2& 1\end{array}\right]$ For transverse gradients operator, $S_v=\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ 1& 0& 1\end{array}\right]$ For longitudinal gradient operator, facial image is used respectively S_hAnd S_vCarry out convolution algorithm, with obtain this facial image at two horizontal and vertical on gradient map；

(3) pixel coordinate of left and right pupil center in the facial contour that calculation procedure (2) obtains；As it is shown on figure 3, this step is specifically, the Sobel operator that still calls in the facial contour that step (2) obtains in above-mentioned steps (2-2), the left and right pixel coordinate value obtaining left pupil is u_LeL、u_HeL, the pixel coordinate value up and down of left pupil is v_LeL、v_HeL, the therefore center pixel coordinate figure (u of left pupil_eL,v_eL) it is (u_LeL+u_HeL/2,v_LeL+v_HeL/ 2).In like manner can obtain the center pixel coordinate figure (u of right pupil_eR,v_eR) it is (u_LeR+u_HeR/2,v_LeR+v_HeR/ 2), wherein u_LeR、u_HeRFor the left and right pixel coordinate value of right pupil, v_LeR、v_HeRPixel coordinate value up and down for right pupil.

(4) projection matrix Mel and the Mer of left and right pupil is calculated according to the pixel coordinate of left and right pupil center in facial contour and the coordinate at four angles of screen, as shown in Figure 4, particularly as follows: first, when human eye vision watches the screen upper left corner attentively, (top left co-ordinate is (x to this step_A,y_A, 0)), it is possible to trying to achieve left and right pupil center pixel coordinate on high definition photographic head by step (3) is (u_1eL,v_1eL), (u_1eR,v_1eR), in like manner try to achieve and watch the screen upper right corner attentively (upper right corner coordinate is (x_B,y_B, 0)) time, left and right pupil center is at the pixel coordinate respectively (u of high definition photographic head_2eL,v_2eL), (u_2eR,v_2eR)；(lower left corner coordinate is (x to watch the screen lower left corner attentively_C,y_C, 0)) time, left and right pupil center is at the pixel coordinate respectively (u of high definition photographic head_3eL,v_3eL), (u_3eR,v_3eR)；(lower right corner coordinate is (x to watch the screen lower right corner attentively_D,y_D, 0)) time, left and right pupil center is at the pixel coordinate respectively (u of high definition photographic head_4eL,v_4eL), (u_4eR,v_4eR)；

Then, the principle according to binocular vision

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$

Coordinate and top left co-ordinate according to above-mentioned four angles of screen are (x_A,y_A), upper right corner coordinate is (x_B,y_B), lower left corner coordinate is (x_C,y_C), lower right corner coordinate is (x_D,y_D) bring on the right of above-mentioned equation, the pixel coordinate of the left pupil that four angles of screen are corresponding is (u_1eL,v_1eL), (u_2eL,v_2eL), (u_3eL,v_3eL), (u_4eL,v_4eL) bring the above-mentioned equation left side, simultaneous solution equation into

Can calculate the projection matrix obtaining left pupil is

$Mel=\left[\begin{array}{ccc}{\mathrm{mel}}_{11}& {\mathrm{mel}}_{12}& {\mathrm{mel}}_{13}\\ {\mathrm{mel}}_{21}& {\mathrm{mel}}_{22}& {\mathrm{mel}}_{23}\\ {\mathrm{mel}}_{31}& {\mathrm{mel}}_{32}& {\mathrm{mel}}_{33}\end{array}\right],$

The projection matrix that in like manner can try to achieve right pupil is

$Mer=\left[\begin{array}{ccc}{\mathrm{mer}}_{11}& {\mathrm{mer}}_{12}& {\mathrm{mer}}_{13}\\ {\mathrm{mer}}_{21}& {\mathrm{mer}}_{22}& {\mathrm{mer}}_{23}\\ {\mathrm{mer}}_{31}& {\mathrm{mer}}_{32}& {\mathrm{mer}}_{33}\end{array}\right].$

(5) projection matrix Mel and the Mer of the left and right pupil obtained by step (4) and the center pixel coordinate figure of left and right pupil calculate left and right pupil physical coordinates value on screen, and this region corresponding to physical coordinates value is the region that user performs gesture operation；Specifically, by the principle of following binocular vision

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}Xel\\ Yel\\ 1\end{array}\right]$

$\left[\begin{array}{c}{u}_{eR}\\ v_{eR}\\ 1\end{array}\right]=Mer\left[\begin{array}{c}Xer\\ Yer\\ 1\end{array}\right]$

Calculating and obtain left and right pupil physical coordinates value on screen, wherein (Xer, Yer) represents right pupil physical coordinates value on screen；(Xel, Yel) represents left pupil physical coordinates value on screen.When vision invests different screen areas, showing broken box as shown in Figure 3, can complete location and the tracking of vision on screen, the region that the physical coordinates value of this step acquisition is corresponding is exactly the operating area of user's gesture in subsequent step (8).

(6) according to the principle of binocular vision, the screen being placed with high definition photographic head is carried out parameter calibration, to obtain projection matrix Ml and the Mr of left and right high definition photographic head respectively；Specifically, by the such as demarcation thing shown in figure (7), utilize Zhang Zhengyou to demarcate screen is demarcated, to obtain demarcation thing pixel coordinate on the high definition photographic head of left and right, each demarcation thing pixel coordinate (u on the high definition photographic head of left and right_1m,v_1m)、(u_2m,v_2m), wherein m is the number of fixed point, as figure (7) be shown with 9, (x_m,y_m,z_m) for scheming the physical coordinates of the circle fixed point shown in (7).And utilize below equation to obtain the projection matrix Ml of left high definition photographic head and the projection matrix Mr of right high definition photographic head respectively:

$\begin{array}{cc}\left[\begin{array}{c}{u}_{1m}\\ v_{1m}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]& \left[\begin{array}{c}{u}_{2m}\\ v_{2m}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]\end{array}$

The final projection matrix obtained is respectively

$Ml=\left[\begin{array}{cccc}{\mathrm{ml}}_{11}& {\mathrm{ml}}_{12}& {\mathrm{ml}}_{13}& {\mathrm{ml}}_{14}\\ {\mathrm{ml}}_{21}& {\mathrm{ml}}_{22}& {\mathrm{ml}}_{23}& {\mathrm{ml}}_{24}\\ {\mathrm{ml}}_{31}& {\mathrm{ml}}_{32}& {\mathrm{ml}}_{33}& {\mathrm{ml}}_{34}\end{array}\right],Mr=\left[\begin{array}{cccc}{\mathrm{mr}}_{11}& {\mathrm{mr}}_{12}& {\mathrm{mr}}_{13}& {\mathrm{mr}}_{14}\\ {\mathrm{mr}}_{21}& {\mathrm{mr}}_{22}& {\mathrm{mr}}_{23}& {\mathrm{mr}}_{24}\\ {\mathrm{mr}}_{31}& {\mathrm{mr}}_{32}& {\mathrm{mr}}_{33}& {\mathrm{mr}}_{34}\end{array}\right]$

(7) high definition camera collection user gesture touches the image of screen, the image gathered is carried out pretreatment, including image subtraction, image denoising, edge extracting, the finger tip differentiated based on K curvature or nib image recognition, to obtain the gesture of the user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F)；As shown in Figure 8, this step specifically includes following sub-step:

(7-1) left and right high definition photographic head gathers user's gesture respectively and touches the image of screen, and is subtracted each other by pixel in the image corresponding point of the image collected and initialization frame, to form new image；

(7-2) the new image that step (7-1) is obtained carries out image denoising, and the process of image denoising is identical with above-mentioned steps (2-1), does not repeat them here；

(7-3) image after denoising is carried out rim detection, namely utilize Sobel operator that the pixel on image does gradient conversion, to obtain edge detection graph；The process of rim detection is identical with above-mentioned steps (2-2), does not repeat them here；

(7-4) pixel on left and right high definition photographic head is carried out K curvature differentiation by the edge detection graph obtained according to step (7-3), to obtain the gesture of the user imager coordinate on left and right high definition photographic head；Specifically, being the edge image that can extract gesture according to the edge detection graph obtained in (7-3), each edge coordinate point vector isIt is set to the K point that this is counted to for starting point by the clockwise direction at edgeThe K point counted to counterclockwise is set toThenK vector computing formula beWhen above-mentioned calculating α is more than 0 and more than setting threshold values β (its span as 0.5 to 1 between), then current vectorCorresponding pixel coordinate is the gesture of user imager coordinate (u on left high definition photographic head_1F,v_1F)；The process of right photographic head is same as described above, and the gesture obtaining user is (u at the pixel coordinate of right high definition photographic head_2F,v_2F)；

(8) according to the gesture operation of user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F) and the projection matrix Mr of the projection matrix Ml of left high definition photographic head and right high definition photographic head, and obtain the gesture of user three dimensional space coordinate (x on screen by below equation_f,y_f,z_f), wherein this gesture operation is in the region that the physical coordinates value obtained in above-mentioned steps (5) is corresponding:

$\left[\begin{array}{c}{u}_{1F}\\ v_{1F}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right],\left[\begin{array}{c}{u}_{2F}\\ v_{2F}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right]$

Three dimensional space coordinate (the x that can obtain gesture is solved by above-mentioned two matrix equation_f,y_f,z_f), namely complete three-dimensional imaging and the location of the gesture of user.

It should be noted that in this step, the mode that the gesture operation of user is click on touches screen.

(9) the coordinate z that step (8) obtains is judged_fWhether less than threshold values γ, wherein the span of γ and the length of screen are directly proportional, if z_fLess than γ, it can be determined that user's gesture generation click action, by USB interface by the three dimensional space coordinate (x of finger tip_f,y_f,z_f) output, else process terminates；

As shown in Figure 9, when user touches screen in a sliding manner, view-based access control model of the present invention location and the step included by gesture identification man-machine interaction method followed the tracks of are basic essentially identical with above-mentioned click mode, only difference is that above-mentioned steps (9) is replaced by:

Obtain the three dimensional space coordinate (x of continuous D frame finger tip_f1,y_f1,z_f1), (x_f2,y_f2,z_f2) ..., (x_fD,y_fD,z_fD), wherein D represents that user slides the frame number of finger tip image collected when touching screen, and be positive integer, thus obtaining gesture sliding trace on screen, is exported by USB interface by track, thus realizing the identification of gesture slip.

Those skilled in the art will readily understand; the foregoing is only presently preferred embodiments of the present invention; not in order to limit the present invention, all any amendment, equivalent replacement and improvement etc. made within the spirit and principles in the present invention, should be included within protection scope of the present invention.

Claims (6)

1. a view-based access control model follows the tracks of the man-machine interaction method with gesture identification, it is characterised in that comprise the following steps:

(1) by infrared light supply, it is used for carrying out the varifocal high definition photographic head of visual tracking and multiple high definition photographic head for carrying out gesture identification is arranged on screen frame place；

(2) varifocal high definition camera collection facial image, and the facial image gathered is carried out face contour extraction；

(3) pixel coordinate (u of calculation procedure (2) obtains facial contour middle left and right pupil center_eL,v_eL) and (u_eR,v_eR)；

(4) projection matrix Mel and the Mer of left and right pupil is calculated according to the pixel coordinate of left and right pupil center in facial contour and the coordinate at four angles of screen；

(5) projection matrix Mel and the Mer of the left and right pupil obtained by step (4) and the center pixel coordinate figure of left and right pupil calculate left and right pupil physical coordinates value on screen, and this region corresponding to physical coordinates value is the region that user performs gesture operation:

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}Xel\\ Yel\\ 1\end{array}\right]$

$\left[\begin{array}{c}{u}_{eR}\\ v_{eR}\\ 1\end{array}\right]=Mer\left[\begin{array}{c}Xer\\ Yer\\ 1\end{array}\right]$

Wherein (Xer, Yer) represents right pupil physical coordinates value on screen, and (Xel, Yel) represents left pupil physical coordinates value on screen；

(6) according to the principle of binocular vision, the screen being placed with high definition photographic head is carried out parameter calibration, to obtain projection matrix Ml and the Mr of left and right high definition photographic head respectively；

(7) high definition camera collection user gesture touches the image of screen, and the image gathered is carried out pretreatment, to obtain the gesture of the user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F)；

(8) according to the gesture operation of user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F) and the projection matrix Mr of the projection matrix Ml of left high definition photographic head and right high definition photographic head, and obtain the gesture of user three dimensional space coordinate (x on screen by below equation_f,y_f,z_f), wherein this gesture operation is in the region that the physical coordinates value obtained in above-mentioned steps (5) is corresponding:

$\left[\begin{array}{c}{u}_{1F}\\ v_{1F}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right],\left[\begin{array}{c}{u}_{2F}\\ v_{2F}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_f\\ y_f\\ z_f\\ 1\end{array}\right]$

(9) the coordinate z that step (8) obtains is judged_fWhether less than threshold values γ, if z_fLess than γ, it can be determined that user's gesture generation click action, by USB interface by the three dimensional space coordinate (x of finger tip_f,y_f,z_f) output, else process terminates.

2. man-machine interaction method according to claim 1, it is characterised in that step (2) includes following sub-step:

(2-1) utilize varifocal high definition camera collection facial image, and with mask method to gather facial image denoising；

(2-2) utilize Sobel operator that the pixel on facial image does gradient conversion, to obtain facial contour.

3. man-machine interaction method according to claim 2, it is characterised in that step (3) is specifically, using the left and right pixel coordinate value that Sobel operator obtains left pupil in the facial contour that step (2) obtains is u_LeL、u_HeL, the pixel coordinate value up and down of left pupil is v_LeL、v_HeL, the center pixel coordinate figure (u of left pupil_eL,v_eL) it is (u_LeL+u_HeL/2,v_LeL+v_HeL/ 2), the center pixel coordinate figure (u of right pupil_eR,v_eR) it is (u_LeR+u_HeR/2,v_LeR+v_HeR/ 2), wherein u_LeR、u_HeRFor the left and right pixel coordinate value of right pupil, v_LeR、v_HeRPixel coordinate value up and down for right pupil.

4. man-machine interaction method according to claim 1, it is characterized in that, step (6) is specially, utilize Zhang Zhengyou to demarcate screen is demarcated, to obtain demarcation thing pixel coordinate on the high definition photographic head of left and right, each demarcation thing pixel coordinate (u on the high definition photographic head of left and right_1m,v_1m)、(u_2m,v_2m), wherein m is the number of fixed point, and utilizes below equation to obtain the projection matrix Ml of left high definition photographic head and the projection matrix Mr of right high definition photographic head respectively:

$\begin{array}{cc}\left[\begin{array}{c}{u}_{1m}\\ v_{1m}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]& \left[\begin{array}{c}{u}_{2m}\\ v_{2m}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\\ 1\end{array}\right]\end{array}$

Wherein (x_m,y_m,z_m) for the physical coordinates of circle fixed point.

5. man-machine interaction method according to claim 1, it is characterised in that step (7) specifically includes following sub-step:

(7-1) left and right high definition photographic head gathers user's gesture respectively and touches the image of screen, and is subtracted each other by pixel in the image corresponding point of the image collected and initialization frame, to form new image；

(7-2) the new image that step (7-1) is obtained carries out image denoising；

(7-3) utilize Sobel operator that the pixel on image does gradient conversion, to obtain edge detection graph；

(7-4) pixel on left and right high definition photographic head is carried out K curvature differentiation by the edge detection graph obtained according to step (7-3), to obtain the gesture of the user imager coordinate (u on left and right high definition photographic head_1F,v_1F) and (u_2F,v_2F)。

6. a view-based access control model follows the tracks of the man-machine interaction method with gesture identification, it is characterised in that comprise the following steps:

(1) by infrared light supply, it is used for carrying out the varifocal high definition photographic head of visual tracking and multiple high definition photographic head for carrying out gesture identification is arranged on screen frame place；

(2) varifocal high definition camera collection facial image, and the facial image gathered is carried out face contour extraction；

(3) pixel coordinate (u of calculation procedure (2) obtains facial contour middle left and right pupil center_eL,v_eL) and (u_eR,v_eR)；

(4) projection matrix Mel and the Mer of left and right pupil is calculated according to the pixel coordinate of left and right pupil center in facial contour and the coordinate at four angles of screen；

(5) projection matrix Mel and the Mer of the left and right pupil obtained by step (4) and the center pixel coordinate figure of left and right pupil calculate left and right pupil physical coordinates value on screen, and this region corresponding to physical coordinates value is the region that user performs gesture operation:

$\left[\begin{array}{c}{u}_{eL}\\ v_{eL}\\ 1\end{array}\right]=Mel\left[\begin{array}{c}Xel\\ Yel\\ 1\end{array}\right]$

$\left[\begin{array}{c}{u}_{eR}\\ v_{eR}\\ 1\end{array}\right]=Mer\left[\begin{array}{c}Xer\\ Yer\\ 1\end{array}\right]$

Wherein (Xer, Yer) represents right pupil physical coordinates value on screen, and (Xel, Yel) represents left pupil physical coordinates value on screen；

(6) according to the principle of binocular vision, the screen being placed with high definition photographic head is carried out parameter calibration, to obtain projection matrix Ml and the Mr of left and right high definition photographic head respectively；

(7) high definition camera collection user gesture touches the image of screen, and the image gathered is carried out pretreatment, to obtain the gesture of the user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F)；

(8) when user slides touch screen, according to the gesture operation of user imager coordinate (u on left high definition photographic head_1F,v_1F) and imager coordinate (u on right high definition photographic head_2F,v_2F) and the projection matrix Mr of the projection matrix Ml of left high definition photographic head and right high definition photographic head, and gesture three dimensional space coordinate (x of the first frame finger tip on screen of user is obtained by below equation_f1,y_f1,z_f1), wherein this gesture operation is in the region that the physical coordinates value obtained in above-mentioned steps (5) is corresponding:

$\left[\begin{array}{c}{u}_{1F}\\ v_{1F}\\ 1\end{array}\right]=Ml\left[\begin{array}{c}{x}_{f1}\\ y_{f1}\\ z_{f1}\\ 1\end{array}\right],\left[\begin{array}{c}{u}_{2F}\\ v_{2F}\\ 1\end{array}\right]=Mr\left[\begin{array}{c}{x}_{f1}\\ y_{f1}\\ z_{f1}\\ 1\end{array}\right]$

(9) step (8) is repeated, to obtain the three dimensional space coordinate (x of follow-up D-1 frame finger tip image_f2,y_f2,z_f2) ..., (x_fD,y_fD,z_fD), wherein D represents that user slides the frame number of finger tip image collected when touching screen, thus obtaining gesture sliding trace on screen, is exported track by USB interface.