CN104571823B - A kind of contactless visual human's machine interaction method based on intelligent television - Google Patents

Abstract

A kind of contactless visual human's machine interaction method based on intelligent television of the present invention, comprise the following steps: step 1, intelligent television is connected image sensing apparatus, and in intelligent television, set up gesture recognition system and gesture information database, the intelligent television executive routine storing several appointment gesture and associate with appointment gesture one_to_one corresponding in gesture information database; Step 2, sets two kinds of operator schemes; Step 3, carries out operator scheme A, the two-dimensional menu interface directly switched out intelligent television by gesture and Three dimensional steerable interface operation; Step 4, carries out operator scheme B, keeps intelligent television current interface constant, calls the intelligent television executive routine that associate corresponding to gesture.The invention has the beneficial effects as follows: the mutual intention of the behavior model detecting user of the perception of combined with virtual interface location and user, propose insoluble in the man-machine system solved based on gesture interaction " Midas? Touch problem " new way.

Description

A kind of contactless visual human's machine interaction method based on intelligent television

Technical field

The present invention relates to a kind of contactless visual human's machine interaction method based on intelligent television.

Background technology

In recent years, three-dimensional (3D) natural gesture interaction technology is applied in the field such as Sign Language Recognition, finger mouse, dummy object control, household remote, Windows order control, finger drawing, robot controlling.Nature, efficient, intelligent, clog-free human-computer interaction interface (HCI, Human-ComputerInterface) become the main development direction of New Generation of Intelligent HCI, set up the main trend that man-machine interface that is intelligentized, natural, harmonious, hommization has become HCI of new generation development.

In now widely used graphic interface, main input mode or physical keyboard and touch-screen input, certainly, touch screen technology relies on its advantage to occupy main flow at mobile terminal.But society is still in development, and research human-computer interaction technology becomes Showed Very Brisk, and man-machine interaction receives rapidly the attention of Ge great producer, and becomes the another competitive field of mobile terminal industry.

Gesture, as natural interactive mode, achieves gratifying development in computer realm.At present, the hardware of mobile terminal is increasingly powerful, and multimedia technology is also developing rapidly, gesture is applied to mobile terminal and has become a thing of pushing the boat with the current.Undertaken marking and identifying by the form of finger, thus reach the object of control, the intuitive of gesture and powerful semanteme make man-machine interaction become particularly light.But because the performance identified has much room for improvement, also do not obtain at present applying very widely.

According to the difference of devices collect data data, current gesture identification can be divided into two kinds substantially, based on the gesture identification of data glove and the gesture identification of view-based access control model.Recognition system advantage based on data glove is that recognition efficiency is high, but requires that user must dress extremely inconvenient equipment due to it, and the cost of equipment is high, so this kind of recognition method can not be following main flow.And the gesture recognition system of view-based access control model relies on good applicability and do not need other hardware device, just by common camera, because of the main flow forming development.On the one hand, based on gesture under blank operation condition, the distance of user and physical interface is distant, and the mental model of this and user has larger gap; On the other hand, under this single input channel condition of gesture, cognitive load and the operational load of user are larger, and interface system not only has many performance constraints to user, even and if simple interactive task also may require that user completes complicated gesture command operation.

TV is becoming one and is carrying out mutual hinge with many contents.These are large-scale, high-resolution display can be used to browse digital photograph, select music, play games, program of seeing a film and watch TV.Present many TVs are connected with internet, and allow access online content and Social Media, this further causes the development of the new product as apple TV and Google's TV, they add the quantity from TV screen obtaining information and complicacy.Domestic pleasure is looked with millet also very fiery, and wherein pleasure is in first depending on having surmounted the Traditional Brand such as Hisense, Changhong after internet television last October, and it has interactive voice and the intelligent remote controller based on touch operation.In many cases, TV remote controller is a limiting factor inherently, and it only provides simple peg button to come to carry out alternately, lacking the dirigibility of mouse and gesture interaction with TV usually.NasserH.Dardas and MohammadAlhaj [7] utilizes Gesture Recognition to produce control command, utilize these control commands to control the motion of object in game, in this gesture recognition system, utilize word bag technique and support vector machine technology mutual to what realize between user and computing machine.The people such as Werner, when not violating user psychology model, control to show the needs adapting to active user by implicit expression adjustment, solve the precision problem of remote interaction pointing device.The people such as JoeWeakliam propose CoMPASS system, and it does not need the explicit input from user, but when user browses map particular space content according to the interested region of characteristic sum, the implicit expression operation of supervisory user.This system can analyze the implicit expression behavior of user, and is used for setting up user model by analysis result.The people such as KaoriFujinami are dissolved into enhancing technology in general fit calculation, adopt the method for nature or implicit expression to obtain user environment information, and user does not need to learn how obtaining information, has filled up the gap between user and complicated calculations environment.The people such as PaulDietz propose the implicit interactions technology based on multi-projector, they using projector as real-time output device, when user enters infrared spectral range, system implicitly can be paid close attention to user and viewing area display user, cyclically show related content by cartoon technology simultaneously.The people such as StavrosAntifakos devise the implicit interactions example of " intelligence " object that is synchronized with the movement towards non-accident, and system implied addressing controls when unlocking the door.

Generally speaking, the subject matter existed based on the contact interactive interface of gesture is at present cognitive load (needing user to remember many gesture commands) and the operational load heavier (user's long-time operation is understood very tired) of user, has had a strong impact on the applying of natural interaction technology based on gesture.

Summary of the invention

For solving above technical deficiency, the cognitive load that the invention provides a kind of user is little, the contactless visual human's machine interaction method based on intelligent television that operational load is lighter.

The present invention is achieved by the following measures:

A kind of contactless visual human's machine interaction method based on intelligent television of the present invention, comprises the following steps:

Step 1, intelligent television is connected image sensing apparatus, and in intelligent television, set up gesture recognition system and gesture information database, the intelligent television executive routine storing several appointment gesture and associate with appointment gesture one_to_one corresponding in gesture information database;

Step 2, if having user to draw rectangle in the aware space before image sensing apparatus observes intelligent television, then carry out step 3, if having user to draw circle in the aware space before image sensing apparatus observes intelligent television, then carry out step 4;

Step 3, carry out operator scheme A, intelligent television switches out two-dimensional menu interface and Three dimensional steerable interface, image sensing apparatus is observed the dynamic operation gesture of user and is identified by gesture recognition system, and the dynamic operation hand modelling utilizing behavior model perception algorithm user to be made carries out direct control on two-dimensional menu interface or Three dimensional steerable interface;

Step 4, carry out operator scheme B, keep intelligent television current interface constant, image sensing apparatus is observed the gesture data of user and is identified by gesture recognition system, in gesture information database, seek the one consistent with the gesture identified all over specify gesture, and call the intelligent television executive routine of corresponding association with it.

User draws the recognition methods of rectangle in step 2, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. by this gesture path polygon of fitting a straight line that this track is connected successively with four head and the tail, obtain a rectangle, make each point on gesture polygon arrive the mean value δ of the vertical range sum of the most near side (ns) of this rectangle minimum;

If e. mean value δ < ω, then this gesture path polygon is rectangle, and returns this rectangle; Otherwise this gesture path polygon is not rectangle, returns sky; Wherein, ω is a non-negative threshold of specifying.

User draws circular recognition methods in step 2, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. calculate this gesture path polygon barycentric coordinates position O a little;

E. each point P on this gesture path polygon is calculated _ithe distance r of distance O _i, r _i=|| P _i-O||;

F. each r is calculated _iaverage: wherein, n is the number that gesture path polygon is put;

G. square error r is calculated: if r< is γ, then gesture path polygon is circle; Otherwise be not circle; Wherein, γ is the non-negative threshold of specifying in advance.

Behavior model perception algorithm comprises the following steps in step 3:

A. the three-dimension gesture structure in N two field picture before particle filter algorithm acquisition current time is utilized; There is virtual interface in setting aware space, the normal direction vector of this virtual interface is T; Kth frame three-dimensional structure vector is G _k=(O _k, D _k,f, D _k,u, θ _k); Wherein, O _krepresent hand shape center of gravity vector, D _k,fand D _k,urepresent hand shape forward direction vector sum upward direction vector respectively, θ _krepresent the vector of each joint values composition of hand shape, 1≤k≤N;

B. the movement locus C of three-dimension gesture model at N frame is calculated _n, then, from present frame, at C _nthe line segment that upper reverse search is parallel with T, supposes that the initial time of this line segment is t ₀, then from t ₀set out, at C _nupper direction scounting line section, supposes that terminating point is t ₁, line segment t ₁t ₀direction vector be D _l; If D _lt=0 and at time period [t ₁, t ₀] interior gesture shape do not change, then detect " translation (Translation)-determine (Confirmation) " model (T-C), namely the current behavior of user is for selecting operation carrying out two-dimensional menu interface; Otherwise, carry out step c;

C. from present frame, oppositely joint angles vector θ _kcorresponding variable composition Vector Groups, if all Vector Groups all dullness do not increase and decrease, if the starting point that dullness does not increase is t ₀, then from t ₀set out, if search for all monotonic nondecreasing increasings of all Vector Groups, then detect " grab (Grasp)-handle (Manipulation)-put (Release) " model (G-M-R), namely the current behavior of user is for carry out the direct manipulation of Three dimensional steerable interface; Otherwise, return step a.

In step 4, the aware space before intelligent television marks off multiple functional area, and gesture identical in different functional areas associates different intelligent television executive routines; In aware space before intelligent television, multiple functional regional division method is:

Step a, there is virtual interface in setting aware space, this virtual interface solid region is G, virtual interface needs the functional area number of layout be m, G is divided into u × v functional block g _i,j(i=1,2 ..u, j=1,2 ..., v), u × v=m;

The center of gravity of step b, G is O, g _i,jcenter of gravity be O _i,j, definition gesture needs displacement weights to be to each functional area: η _i,j=γ exp (-|| O _i,j-O||); Displacement is larger, then displacement weights are less, and 0≤η i, j≤1 (1≤i≤u, 1≤j≤v);

Step c, further the ease for operation β of each functional area of definition _i,j(i=1,2 ..u, j=1,2 ..., v), they can obtain by the method for probability statistics, and meet: 0≤β i, j≤1 (1≤i≤u, 1≤j≤v);

The frequency of utilization ω of each function _i(i=1,2 ..m) is different, adopts the method for probability statistics to obtain, and meets: 0≤ω _i, j≤1 (1≤i≤u, 1≤j≤v); Each g _i,jthe overall assessment weights of position " quality " are: α _i,j=η _i,jβ _i,j;

Steps d, then by α _i,jsort according to descending, obtain sequence (α ₁, α ₂..., α _m), by ω _i,jalso sort according to descending order, obtain sequence (ω ₁, ω ₂..., ω _m); Thus, ω _kcorresponding function distributes to module α _k(1≤k≤m).

The invention has the beneficial effects as follows: 1. the gesture operation behavior model not only reflecting user from a side, and feature the mental model of user, and the contactless mutual interface paradigm of specification, make contactless interactive interface be can calculate, appreciable, therefore unite mutual for contact and contactless well alternately.2. optionally transparent switching between different operating attitude, different operating dimension, this switching objectively can reduce and is limited to the tension and sense of fatigue that a certain fixing operation pattern brings for a long time.3., in operator scheme A, the MentalModel of user is directly based upon on the basis that physical interface directly maps, and user just can complete interactive task by virtual interface without the need to study or memory, therefore the cognitive load of user is very low; In operator scheme B, because same gesture will be construed as different orders in different functional blocks, therefore can complete different semantic operation by same gesture, and remember a large amount of gesture commands without the need to user, therefore reduce the cognitive load of user.4. the mutual intention of the behavior model detecting user of the perception of combined with virtual interface location and user, proposes the new way of insoluble in the man-machine system solved based on gesture interaction " MidasTouch problem ".

Embodiment

A kind of contactless visual human's machine interaction method based on intelligent television of the present invention, in order to reduce user in the impact operating " MidasTouch problem " in virtual interface process, get rid of the interference that may exist between difference in functionality district, personal behavior model combines with virtual interface by the present invention, proposes the implicit interactions normal form based on personal behavior model and virtual interface.Comprise the following steps:

Step 1, intelligent television is connected image sensing apparatus, and in intelligent television, set up gesture recognition system and gesture information database, the intelligent television executive routine storing several appointment gesture and associate with appointment gesture one_to_one corresponding in gesture information database;

Step 2, if having user to draw rectangle in the aware space before image sensing apparatus observes intelligent television, then carry out step 3, if having user to draw circle in the aware space before image sensing apparatus observes intelligent television, then carry out step 4; Set up the two class basic operation models of user.The first, the behavior model in menu operation is expressed as: translation (Translation)-determine (Confirmation).In " translation " stage, the attitude of three-dimension gesture model can not change, and just the centre of gravity place of gesture changes; " determine " stage, user has the action of " pushing away forward ", and when the direction of motion of this action and translation motion, the direction of motion of center of gravity is perpendicular.Behavior model in the second, 3D direct manipulation is expressed as: grab (Grasp)-handle (Manipulation)-put (Release).Utilize " grabbing " and " putting " joint variation characteristic, just can identify " grabbing " and " put " gesture; " operation " in model, mainly comprises the basic operation such as translation, rotation.For shortcut calculation, if detect " grab-put " structure in user's gesture motion, then think that user is current and carry out " 3D direct manipulation ".Come that the whole physical interface of metaphor does not have function menu with " picture rectangle ", meet the mental model of user.

Particularly, user draws the recognition methods of rectangle, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. by this gesture path polygon of fitting a straight line that this track is connected successively with four head and the tail, obtain a rectangle, make each point on gesture polygon arrive the mean value δ of the vertical range sum of the most near side (ns) of this rectangle minimum;

If e. mean value δ < ω, then this gesture path polygon is rectangle, and returns this rectangle; Otherwise this gesture path polygon is not rectangle, returns sky; Wherein, ω is a non-negative threshold of specifying.

User draws circular recognition methods, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. calculate this gesture path polygon barycentric coordinates position O a little;

E. each point P on this gesture path polygon is calculated _ithe distance r of distance O _i, r _i=|| P _i-O||;

F. each r is calculated _iaverage: wherein, n is the number that gesture path polygon is put;

G. square error r is calculated: if r< is γ, then gesture path polygon is circle; Otherwise be not circle; Wherein, γ is the non-negative threshold of specifying in advance.

Step 3, carry out operator scheme A, intelligent television switches out two-dimensional menu interface and Three dimensional steerable interface, image sensing apparatus is observed the dynamic operation gesture of user and is identified by gesture recognition system, and the dynamic operation hand modelling utilizing behavior model perception algorithm user to be made carries out direct control on two-dimensional menu interface or Three dimensional steerable interface;

Behavior model perception algorithm comprises the following steps:

A. the three-dimension gesture structure in N two field picture before particle filter algorithm acquisition current time is utilized; There is virtual interface in setting aware space, the normal direction vector of this virtual interface is T; Kth frame three-dimensional structure vector is G _k=(O _k, D _k,f, D _k,u, θ _k); Wherein, O _krepresent hand shape center of gravity vector, D _k,fand D _k,urepresent hand shape forward direction vector sum upward direction vector respectively, θ _krepresent the vector of each joint values composition of hand shape, 1≤k≤N;

B. the movement locus C of three-dimension gesture model at N frame is calculated _n, then, from present frame, at C _nthe line segment that upper reverse search is parallel with T, supposes that the initial time of this line segment is t ₀, then from t ₀set out, at C _nupper direction scounting line section, supposes that terminating point is t ₁, line segment t ₁t ₀direction vector be D _l; If D _lt=0 and at time period [t ₁, t ₀] interior gesture shape do not change, then detect " translation (Translation)-determine (Confirmation) " model (T-C), namely the current behavior of user is for selecting operation carrying out two-dimensional menu interface; Otherwise, carry out step c;

C. from present frame, oppositely joint angles vector θ _kcorresponding variable composition Vector Groups, if all Vector Groups all dullness do not increase and decrease, if the starting point that dullness does not increase is t ₀, then from t ₀set out, if search for all monotonic nondecreasing increasings of all Vector Groups, then detect " grab (Grasp)-handle (Manipulation)-put (Release) " model (G-M-R), namely the current behavior of user is for carry out the direct manipulation of Three dimensional steerable interface; Otherwise, return step a.

In step 4, the aware space before intelligent television marks off multiple functional area, and gesture identical in different functional areas associates different intelligent television executive routines; In aware space before intelligent television, multiple functional regional division method is:

Step a, there is virtual interface in setting aware space, this virtual interface solid region is G, virtual interface needs the functional area number of layout be m, G is divided into u × v functional block g _i,j(i=1,2 ..u, j=1,2 ..., v), u × v=m;

The center of gravity of step b, G is O, g _i,jcenter of gravity be O _i,j, definition gesture needs displacement weights to be to each functional area: η _i,j=γ exp (-|| O _i,j-O||); Displacement is larger, then displacement weights are less, and 0≤η i, j≤1 (1≤i≤u, 1≤j≤v);

Step c, further the ease for operation β of each functional area of definition _i,j(i=1,2 ..u, j=1,2 ..., v), they can obtain by the method for probability statistics, and meet: 0≤β i, j≤1 (1≤i≤u, 1≤j≤v);

The frequency of utilization ω of each function _i(i=1,2 ..m) is different, adopts the method for probability statistics to obtain, and meets: 0≤ω _i, j≤1 (1≤i≤u, 1≤j≤v); Each g _i,jthe overall assessment weights of position " quality " are: α _i,j=η _i,jβ _i,j;

Steps d, then by α _i,jsort according to descending, obtain sequence (α ₁, α ₂..., α _m), by ω _i,jalso sort according to descending order, obtain sequence (ω ₁, ω ₂..., ω _m); Thus, ω _kcorresponding function distributes to module α _k(1≤k≤m).

In sum, long-time continuous uses gesture and handles interface and often easily cause the sense of fatigue of user.Evidence suggests, the one of the main reasons of human fatigue is that operating gesture mode is too single.In fact, on the one hand, user is only allowed to carry out 2D/3D operation, or require that user must operate continuously and can not stop, or gesture command operation is well-separated with direct manipulation, or gesture " enters " interface easily " going out " interface difficulty ... these restrictions itself not only allow user feel physical fatigue, and feel that psychology is tired out.On the other hand, in 3D interactive interface application system, often there is 2D interface (such as menu option or Operation system setting etc. generally present with 2D interface), therefore, 3D and 2D coexists the essential characteristic that will be the 3D interactive interfaces such as following 3D mobile device, intelligent 3D Digital Television---and this is the basic reason that inputs of this patent research multi-modal gesture of mixing (2D/3D, communication type gesture/manipulation type gesture) just.Therefore, with allowing user's real " nature " operation interface, inevitable requirement is optionally transparent switching between different operating attitude, different operating dimension, and this switching objectively can reduce and is limited to the tension and sense of fatigue that a certain fixing operation pattern brings for a long time.

Secondly, in operator scheme A, the mental model (MentalModel) of user is directly based upon on the basis that physical interface directly maps, and user just can complete interactive task by virtual interface without the need to study or memory, therefore the cognitive load of user is very low.In operator scheme B, because same gesture will be construed as different orders in different functional blocks, therefore can complete different semantic operation by same gesture, therefore reduce the cognitive load of user.Because only have the behavior model of the perception of combined with virtual interface location and user, therefore largely solve insoluble in the man-machine system based on gesture interaction " MidasTouch problem ".

The invisible virtual interface of this block near user's gesture has special meaning and function for the psychology of user.First, it is instruction " launch site "; Secondly, thinking model is abstract to one of carried out concrete active mechanism in user's brain, and the thinking model " distance " of this region and user is nearest; Say that it is the substitute of physical interface in a sense.With the consistance of thinking model, objectively just require that system needs the position of automatically reorientating virtual interface when " the operation center of gravity " of user marked change occurs.

By the perception of interactive device to personal behavior model, successfully build the 26S Proteasome Structure and Function distribution of virtual interface, critical data and the key model such as style, the type of virtual interface, windy lattice interface image matrix of virtual interface.Therefore, Intellisense realizes the core based on virtual interface implicit interactions, and virtual interface is the basis of carrying Intellisense.

The above is only the preferred implementation of this patent; it should be pointed out that for those skilled in the art, under the prerequisite not departing from the art of this patent principle; can also make some improvement and replacement, these improve and replace the protection domain that also should be considered as this patent.

Claims (4)

1., based on contactless visual human's machine interaction method of intelligent television, it is characterized in that, comprise the following steps:

Step 1, intelligent television is connected image sensing apparatus, and in intelligent television, set up gesture recognition system and gesture information database, the intelligent television executive routine storing several appointment gesture and associate with appointment gesture one_to_one corresponding in gesture information database;

Step 2, if having user to draw rectangle in the aware space before image sensing apparatus observes intelligent television, then carry out step 3, if having user to draw circle in the aware space before image sensing apparatus observes intelligent television, then carry out step 4;

Step 3, carry out operator scheme A, intelligent television switches out two-dimensional menu interface and Three dimensional steerable interface, image sensing apparatus is observed the dynamic operation gesture of user and is identified by gesture recognition system, and the dynamic operation hand modelling utilizing behavior model perception algorithm user to be made carries out direct control on two-dimensional menu interface or Three dimensional steerable interface;

Step 4, carry out operator scheme B, keep intelligent television current interface constant, image sensing apparatus is observed the gesture data of user and is identified by gesture recognition system, in gesture information database, seek the one consistent with the gesture identified all over specify gesture, and call the intelligent television executive routine of corresponding association with it;

In step 4, the aware space before intelligent television marks off multiple functional area, and gesture identical in different functional areas associates different intelligent television executive routines; In aware space before intelligent television, multiple functional regional division method is:

Step a, there is virtual interface in setting aware space, this virtual interface solid region is G, virtual interface needs the functional area number of layout be m, G is divided into u × v functional block g _i,j(i=1,2 ..u, j=1,2 ..., v), u × v=m;

The center of gravity of step b, G is O, g _i,jcenter of gravity be O _i,j, definition gesture needs displacement weights to be to each functional area: η _i,j=γ exp (-|| O _i,j-O||); Displacement is larger, then displacement weights are less, and $\underset{i=1}{\overset{u}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{v}{\mathrm{\&Sigma;}}}{\mathrm{\&eta;}}_{i,j}=\mathrm{1,0}\&le;\mathrm{\&eta;i},j\&le;1(1\&le;i\&le;u,1\&le;j\&le;v);$

Step c, further the ease for operation β of each functional area of definition _i,j(i=1,2 ..u, j=1,2 ..., v), they can obtain by the method for probability statistics, and meet: $\underset{i=1}{\overset{u}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{v}{\mathrm{\&Sigma;}}}{\mathrm{\&beta;}}_{i,j}=\mathrm{1,0}\&le;\mathrm{\&beta;i}.j\&le;1(1\&le;i\&le;u,$ $1\&le;j\&le;v);$

The frequency of utilization ω of each function _i(i=1,2 ..m) is different, adopts the method for probability statistics to obtain, and meets: $\underset{i=1}{\overset{u}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{v}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_{i,j}=\mathrm{1,0}\&le;{\mathrm{\&omega;}}_i,j\&le;1(1\&le;i\&le;u,1\&le;j\&le;v);$ Each g _i,jthe overall assessment weights of position " quality " are: α _i,j=η _i,jβ _i,j;

Steps d, then by α _i,jsort according to descending, obtain sequence (α ₁, α ₂..., α _m), by ω _i,jalso sort according to descending order, obtain sequence (ω ₁, ω ₂..., ω _m); Thus, ω _kcorresponding function distributes to module α _k(1≤k≤m).

2., according to claim 1 based on contactless visual human's machine interaction method of intelligent television, it is characterized in that: user draws the recognition methods of rectangle in step 2, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. by this gesture path polygon of fitting a straight line that this track is connected successively with four head and the tail, obtain a rectangle, make each point on gesture polygon arrive the mean value δ of the vertical range sum of the most near side (ns) of this rectangle minimum;

If e. mean value δ < ω, then this gesture path polygon is rectangle, and returns this rectangle; Otherwise this gesture path polygon is not rectangle, returns sky; Wherein, ω is a non-negative threshold of specifying.

3. according to claim 1 based on contactless visual human's machine interaction method of intelligent television, it is characterized in that, user draws circular recognition methods in step 2, comprises the following steps:

A. image sensing apparatus observes the image in intelligent television aware space, removes the background of every two field picture, obtains images of gestures;

B. the center of gravity of each images of gestures is calculated;

C. start the mark terminated with gesture using static gesture as gesture, by the center of gravity of each frame gesture according to time sequencing composition sequence, obtain gesture path polygon;

D. calculate this gesture path polygon barycentric coordinates position O a little;

E. each point P on this gesture path polygon is calculated _ithe distance r of distance O _i, r _i=|| P _i-O||;

F. each r is calculated _iaverage: wherein, n is the number that gesture path polygon is put;

G. square error r is calculated: if r< is γ, then gesture path polygon is circle; Otherwise be not circle; Wherein, γ is the non-negative threshold of specifying in advance.

4., according to claim 1 based on contactless visual human's machine interaction method of intelligent television, it is characterized in that, behavior model perception algorithm comprises the following steps in step 3:

A. the three-dimension gesture structure in N two field picture before particle filter algorithm acquisition current time is utilized; There is virtual interface in setting aware space, the normal direction vector of this virtual interface is T; Kth frame three-dimensional structure vector is G _k=(O _k, D _k,f, D _k,u, θ _k); Wherein, O _krepresent hand shape center of gravity vector, D _k,fand D _k,urepresent hand shape forward direction vector sum upward direction vector respectively, θ _krepresent the vector of each joint values composition of hand shape, 1≤k≤N;

B. the movement locus C of three-dimension gesture model at N frame is calculated _n, then, from present frame, at C _nthe line segment that upper reverse search is parallel with T, supposes that the initial time of this line segment is t ₀, then from t ₀set out, at C _nupper direction scounting line section, supposes that terminating point is t ₁, line segment t ₁t ₀direction vector be D _l; If D _lt=0 and at time period [t ₁, t ₀] interior gesture shape do not change, then detect " translation (Translation)-determine (Confirmation) " model (T-C), namely the current behavior of user is for selecting operation carrying out two-dimensional menu interface; Otherwise, carry out step c;

C. from present frame, oppositely joint angles vector θ _kcorresponding variable composition Vector Groups, if all Vector Groups all dullness do not increase and decrease, if the starting point that dullness does not increase is t ₀, then from t ₀set out, if search for all monotonic nondecreasing increasings of all Vector Groups, then detect " grab (Grasp)-handle (Manipulation)-put (Release) " model (G-M-R), namely the current behavior of user is for carry out the direct manipulation of Three dimensional steerable interface; Otherwise, return step a.