CN103916875B - Management and planning system based on WIFI wireless networks multiclass control terminal - Google Patents

Abstract

The present invention relates to the management based on WIFI wireless networks multiclass control terminal to include following sections with planning system：The hardware system based on WIFI is built first under intelligent space, makes all smart machines, intelligent terminal can be seamlessly connected with server；Then various control terminal is accessed, wherein including：Remote handset control, server end Voice command, the gesture control based on Kinect and inertial navigation module (AHRS) and the control of server end body；The optimizing scheduling algorithm of all smart machines of final design server end realizes control and service ubiquitous under intelligent space with mission planning algorithm.The present invention realizes the multiclass control mode of more smart machines based on WIFI wireless networks, realizes that the Optimized Operation of smart machine and high-quality service are planned using intellectual space technique.

Description

Management and planning system based on WIFI wireless networks multiclass control terminal

Technical field

The present invention relates to a kind of management based on WIFI wireless networks multiclass control terminal and planning system, belong to Intelligent air Between technical field.

Background technology

In recent years, with International Electro product, the continuous progress of smart machine, constantly set for intelligent space injection is fresh It is standby, substantially increase service and the service ability of intelligent space.With service field and the continuous development of radio network technique, The service for being provided with effect high quality under intelligent space for people has been possibly realized.Current various sophisticated equipments even service robot Research come into a highly intelligentized level, how being provided various smart machine organic coordinations nowhere for people Not service turn into study hotspot.

The research of intelligent space provides theoretical foundation for above mentioned problem, using intellectual space technique by multiclass smart machine It is uniformly controlled, and can needs to provide the user different device services according to different user different situations.Even intelligent space It is combined with internet, the right of remote control of Space Facilities can be realized in the distance by assigning user.

In all kinds of technologies mentioned above, either WIFI wireless networks, can connect internet hand at Kinect gestures detections Machine software, or server voice and algorithm process ability, all have quite ripe research theory and application in the present invention Achievement.Therefore the fusion of above-mentioned control mode and the implementation of control strategy become no longer difficult.

The content of the invention

In view of the shortcomings of the prior art, the present invention provide it is a kind of based on WIFI wireless networks multiclass control terminal management with Planning system.The system is advised including equipment scheduling in the application method of smart machine, intelligent space in intelligent space and task The method of drawing, ubiquitous control program is provided for user, provides the user intelligentized service, and the present invention utilizes Intelligent air Between technology realize smart machine Optimized Operation and high-quality service planning.

Technical scheme is as follows：

A kind of management and planning system based on WIFI wireless networks multiclass control terminal, including：

(1) hardware system based on WIFI is built under intelligent space, the hardware system based on WIFI includes intelligence Equipment, intelligent terminal and server, described smart machine, intelligent terminal respectively with the server wireless connection；

(2) terminal access server will be controlled, the control terminal includes：Remote handset control, server end voice control System, the gesture control based on Kinect and inertial navigation module and the control of server end body；The server end body controls： The smart machine in intelligent space, intelligent terminal are directly controlled by way of button at the management software interface of server end System；

(3) design server end is to all smart machines, the optimizing scheduling algorithm and mission planning algorithm of intelligent terminal, with Realize control and service ubiquitous under intelligent space；

The specific steps of hardware system of the structure based on WIFI comprise the following steps in the step (1):

A, the smart machine in intelligent space and intelligent terminal are matched using WIFI wireless network modules, made described By wireless network by remote auto control, described automatically controls including the smart machine, intelligence for smart machine, intelligent terminal Energy terminal is controlled by control terminal control or being serviced device end body；The smart machine and intelligent terminal are to be controlled by wireless network Electric light, TV, air-conditioning and curtain of network etc.；The WIFI wireless network modules use STM32 to pass through string for bottom control core Mouth communicates with RM04 modules, and the networking mode i.e. control core for taking many-one to broadcast corresponds to multiple servers and used Transmission Control Protocol carries out data transmission, and improves the stability of system in programming using Socket non-blocking modes, and the server includes Mobile phone, computer, improve the compatibility to control device；

B, the hardware of data transfer includes router and RM04 modules in step a,

The router effect is that the WIFI wireless networks of local are forwarded into public network so as to a public network as system Interface and cradle head camera is accessed by network by wds patterns；Dynamic Host Configuration Protocol server is by identifying RM04 modules in the router Then MAC Address distributes corresponding IP address of internal network, then internal address is forwarded on public network, the RM04 modules play by The effect of bottom control core couple in router, the RM04 modules are configured to network interface card pattern, are serviced in a local network as TCP Device is present；

C, the circuit of bottom control core produces different pulse signals to difference by STM32 single-chip microcomputers control relay Smart machine and intelligent terminal are controlled, and the circuit of the bottom control core also includes control interface for extending a variety of sensings Device；

D, in the communication of upper strata, each smart machine and intelligent terminal pass through the distribution of router with obtaining unique IP Location, server are controlled by IP address to smart machine and intelligent terminal；

In bottom communication, WIFI wireless network modules are communicated by serial mode and the STM32 single-chip microcomputers；Institute State STM32 single-chip microcomputers controls the switch of smart machine and intelligent terminal to run by way of control relay；

In communication process, each smart machine and intelligent terminal use with server by using TCP/IP communication agreement Socket programming modes realize the transmission of data or order；

E, according to step a, b, c, d and WIFI wireless networks and cable network are based on, centered on router, by Intelligent air Between in can be connected into each smart machine, intelligent terminal, server and the sensing equipment that can be communicated with server of network It is connected in a LAN, and each smart machine and intelligent terminal have the function of link internet；Make whole system Form tasks carrying part, device control unit point and server and data storage section；

It is as follows that Kinect gesture controls part described in the step (2) includes step：

F, Kinect is accessed into computer system；

G, by starting tracking of the Kinect to the palm of the hand, the positional information of the palm of the hand, and the depth information of the palm of the hand are obtained, The position of palm of the hand distance Kinect depth transducers center 1000-3000mm depth, enters to the two dimensional surface region where the palm of the hand The division of row functional area, laser pen region, area of isolation, control area are divided into, and define the palm of the hand in each region The instruction of the corresponding operation of diverse location, and make following provisions：Gauged distance be immediately ahead of the Kinect at 1 meter away from From；Standard flat is the plane where the four corner of palm of the hand movement at gauged distance；Standard area is the palm of the hand in standard Image range during planar movement on Kinect, and it is defined as rectangular area；The center of brandishing is that user stretches in front When going out the right hand, palm of the hand position residing in scope is entirely brandished, under different depth information；Reference area is palm of the hand movement Center of the scope on Kinect images, i.e. people caused by the center in Kinect directions on image perpendicular to waving Region；

H, computer is instructed according to gesture control, and the corresponding behaviour to smart machine and intelligent terminal is realized by network communication Make；The effect of contactless property operation is realized, this method avoids the research to gesture path, simplifies identification difficulty, and carry High discrimination；

It is as follows that inertial navigation module (AHRS) control section described in the step (2) includes step：

I, inertial navigation module (AHRS) is accessed into computer by communication；

J, merged by the Data Data of accelerometer and gyroscope magnetometer in inertial navigation module, the control of analysis and Control person Posture, inertial navigation module gather the signal of three axis MEMS gyro, and using quaternary number posture expression formula, integration tries to achieve gyroscope Attitude angle, while 3 axis MEMS accelerometer and the signal of three axle magnetometer are gathered, using gravitational field and geomagnetic field in geography Direction cosines conversion between coordinate system and body axis system carries out absolute angle resolving, obtains absolute pose angle, then basis The change frequency of absolute pose angle signal changes filtering parameter in real time, and the attitude angle obtained twice is carried out to be based on expanded type karr The data fusion of graceful filtering, the attitude angle and fusion acceleration of final output accurate stable, wherein, accelerometer and gyroscope Fusion principle repaiied with the value of accelerometer amendment gyroscope, when dynamic using the value of gyroscope when be mainly static The value of positive acceleration meter, the 3-axis acceleration (Ax, Ay, Az) exported according to the three axis accelerometer on movable body, is asked for respectively Angle of pitch pitch=tan^-1(- Ay ,-Az), inclination angleMerge accelerationMagnetometer is mainly to carry out magnetometer using accelerometer with the principle that merges of accelerometer Slope compensation, read the three-axle magnetic field intensity of magnetometer outputAsk for the magnetometer after slope compensation Output

WhereinExported for the magnetometer after slope compensation,For three axle magnetic of magnetometer output Field intensity, roll are inclination angle, and pitch is the angle of pitch；

Exported according to the magnetometer after the slope compensation calculated, ask for yaw angleObtain not After the angle value and acceleration of disconnected renewal, increase the reliability of data by the way of sliding window, related in the inertial navigation module And two kinds of control modes：Point to by control smart machine and intelligent terminal fire finger control and upward refer to fire finger control it is specific Pattern；When final control signal determines, by the way of threshold value：Judged according to the output valve of accelerometer, if the numerical value Then it is judged as that control instruction is sent more than threshold value.

K, effector position is carried out by distributed vision, the i.e. binocular with monopod video camera or multi-vision visual system Positioning, the effector are the sharp user that is possessed of control power to smart machine, intelligent terminal in intelligent space；According to inertial navigation mould The position of the angle information of block and current effector draw the angle that people faces, and judge what effector to be controlled by the angle Smart machine and intelligent terminal, after control signal is sent, controlled device is determined according to this angle information to be controlled；

It is as follows that Voice command part described in the step (2) includes step：

L, wireless microphone receiving terminal is accessed into computer；

M, user inputs voice command by wireless microphone, and voice command is carried out by the speech recognition part of computer Command recognition and parsing, and matched with the control command in voice command storehouse, if the match is successful, by controlling accordingly System order sends equipment control instruction, realizes the Voice command to equipment；

The control of remote handset described in the step (2) comprises the following steps：

N, server end uses Windows+Apache+SQLServer+PHP web application platform；Service uses B/S (Browser-Server) framework, effector are passed through clear using traditional PC, mobile device and any equipment for being equipped with browser Device of looking at accesses the server, has good cross-platform characteristic；

O, whole service application is divided into：Presentation layer (UI), Business Logic (BLL), data access layer (DAL)；Institute The responsible operation to database of data access layer is stated, data, services are provided for Business Logic；Business Logic concern business rule The designs relevant with business demand such as formulation then, the realization of operation flow；Presentation layer is used for display data and reception user is defeated The data entered, provide the user the interface of interactive operation；

P, response type Web designs (Responsive Web design)：The design and development of the page should be according to user's row For and facility environment (system platform, screen size, screen orientation etc.) carry out accordingly respond and adjust, no matter user Using traditional PC or smart mobile phone, Pad, the page should be able to all automatically switch resolution ratio, dimension of picture and associated script work( Can etc., to adapt to distinct device；I.e.：The page can adaptive user facility environment；Websocket is using independent based on TCP Offer full-duplex communication channel agreement, realize bidirectional real-time；

Step (3) the design server end includes step to the optimizing scheduling algorithm of all smart machines, intelligent terminal It is as follows：

All it has been connected in same LAN by various modes based on all smart machines, intelligent terminal, In scheduling portion, scheduled smart machine, intelligent terminal are divided into mobile device and fixed equipment, the tune on fixed equipment Degree, require to be controlled by using different control modes according to the timely position of user and control, the kind equipment is set Equal priority is set to, using requiring in dispatching algorithm, by the way of controlling；And the scheduling of mobile device is as described below：

P, each mobile device is linked into computer system by wireless network；Robot in intelligent space belongs to shifting Dynamic equipment；

Q, user is by client access server, and client can provide subscriber identity information to server, and to service Device sends user task application；The client is the software that user is controlled to mobile device, and it can be in server sheet Body can also be on the pc that can be communicated with server；

I, after server receives user's application, application is parsed, when obtaining user identity, the application of application task Between, the subtask that includes, the priority of task, after being parsed, task is loaded into task list according to task priority；

S, normal priority task, will be performed according to loading sequence；Higher priority task will be performed preferentially；Highest Priority tasks must perform in advance, and can interrupt the low priority task being carrying out, and preferentially be performed；

T, server will be carried out task scheduling to the task in task list, be finally completed user Shen according to dispatching algorithm Please；

U, the principle that dispatching algorithm follows：

I. user's equality, performance is both ways：When during the task of All factors being equal, preference will be give in list level, according to the application time successively Perform, it is unrelated with task object；Second, limit priority task only when currently without idle mobile devices, just forces to terminate just In the low priority task of execution, otherwise preferentially task is completed using idle mobile devices；

II. the energy most saves, if showing that task includes multiple subtasks, transfers to that these can be fully completed as far as possible The mobile device of subtask is completed, and avoids completing using multiple mobile devices；

V, dispatching algorithm flow：

Step1：A mission bit stream is obtained from task list；

Step2：The precedence information of the task is checked, if limit priority task, then Step3 is performed, otherwise performs Step6；

Step3：Check whether the mobile device for being not carried out limit priority task be present, if so, then moving these Equipment is divided into four classes, is respectively can complete all subtasks and idle mobile device, can complete part subtask and idle Mobile device, all subtasks and busy mobile device can be completed, part subtask can be completed and busy movement is set It is standby；Otherwise Step8 is performed；

Step4：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether to deposit In the second class mobile device, exist, can detect the second class mobile device complete whole subtasks, if can, moved by the second class Dynamic equipment performs the task, otherwise detects whether the 3rd class mobile device be present, exists then to be performed by the 3rd class mobile device and appoints Business, if, by the second class mobile device executable portion subtask, remaining subtask transfers to the 4th class mobile device to complete without if；If Without the second class mobile device, then when the 3rd class mobile device be present, there is the 3rd class mobile device to complete task, without the 3rd Class mobile device then completes task by the 4th class mobile device；

Step5：Step8 is performed after the completion of task；

Step6：Available free mobile device is checked whether, mobile device is divided into two classes if having, can respectively complete institute There is the mobile device of subtask and free time, the mobile device of part subtask and free time can be completed；

Step7：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether to deposit In the second class mobile device, task is performed by the second class mobile device if having；

Step8：If whole subtasks in the task have been performed both by, the task is deleted, if having only carried out part Subtask, then the subtask that executed is complete in the task is deleted, but the task still retains, if the task does not have mobile device can To perform, then the task is temporarily skipped；

Step9：Step1 is performed, obtains a new task, this process is repeated, until the task in list is all held OK；

Step (3) the design server end is to all smart machines, the mission planning algorithm of intelligent terminal, including step It is as follows：

W, each service addition Service name, such as RobotMove (robot movement) are given, driving parameter is added for the service As (E, F, G) (position, angle and velocity information etc.), and the service execution complete later output parameter (I, J, K) (figure Picture, target location and targeted attitude information etc.)；The working condition of also this service is respectively yes, no, work, system according to Its working condition selects to service：When the parameter that server receives meets at least one service, the service just by Driving, when the service execution complete can produce other parameter, can equally drive corresponding service, in this way formed planning stream Journey, in order to ensure the accuracy of flow and feasibility, algorithm part is designed as follows：

X, the specific steps in planning algorithm

Step1：Prepare parameter set, services set, active service collection and Backup collection, and final service list five is deposited Storage unit；

Step2：Whether cycle detection parameter set has parameter input to go to Step3 if parameter input, if without if Simultaneously cycle detection is waited in Step2；

Step3：According to the input parameter of parameter set, services set is scanned for, finding being capable of powered service deposit First layer, the index INDEX related to the number of plies add 1, go to Step4；

Step4：Duplicate checking is carried out to this layer of service just obtained, if the output of a service is defeated included in another In going out, this service is deleted from final service list, is placed in active service concentration；Parameter for driving this invalid service Deleted from parameter set and be placed in Backup concentration；To least recently used all parameters to prevent from repeating driving, will also delete simultaneously It is placed in Backup concentration；Go to Step5；

Step5：Detect whether this layer of service has output, if then going to Step2, if not going to Step6；

Step6：Calculate the wastage in bulk or weight of each service procedure, optimum selecting final flowsheet；

Step7：Terminate planning process, start to perform, and transfer to dispatch layer to run.

The present invention intentional value be

The equipment control flexibility ratio of intelligent space system is improved, aspect system user is to the intelligent space belonging to oneself Control.The addition of network control mode allows user's control to intelligent space anywhere or anytime all the time.Equipment is adjusted Spending the application with service quality achievement in research causes the operational efficiency of intelligent space to be substantially improved, and the quality of service user carries significantly It is high.Make the practical application of intelligent space further.

In hardware design and network connection, the networking mode i.e. control core for taking many-one to broadcast can be correspondingly more Individual server is simultaneously carried out data transmission using Transmission Control Protocol, and Socket non-blocking modes, the networking of this honeycomb fashion are used in programming Mode, the pattern of relatively large more central server, center caused by when avoiding mass data transmission are blocked, and work as Centroid The uses of other servers is not influenceed during paralysis so as to improve the reliability of system.

Monopod video camera is added in LAN using wds patterns, the trouble of cabling is avoided, makes the peace of monopod video camera Dress is more flexible.Controller sends unlike signal by relay according to distinct device and is controlled, and is held as electric light needs one Continuous high level or low level, and projecting apparatus needs two interval 1s or so pulse signal etc..

In the design of modules, first in Kinect Control System Designs, under standard flat, the palm of the hand is in equipment Movement can successful control device in the picture scope of return.But in actual applications, the hand of user is not all to exist all the time In standard flat, so the corresponding region on two-dimension picture can also produce difference.The depth moving range of user's palm of the hand is Fixed, but distance is more remote, moving range of the palm of the hand on picture can reduce, in order to solve problem above, as the present invention Further improvement, present invention additionally comprises in part 2 b step, when in user's palm of the hand distance Kinect depth transducers When the depth of the heart is not 1000mm, in addition to the two dimension letter by palm of the hand distance Kinect depth transducers center different depth position Breath is transformed into the process of the two-dimensional signal of palm of the hand distance Kinect depth transducers center 1000mm depth, may be selected by following Step is realized：First, moving range of the palm of the hand under any depth information is determined using least square method；Then, adoption rate Mode according to depth value the 2-D data of the palm of the hand be transformed into the palm of the hand distance Kinect depth transducers center 1000mm depth The 2-D data of degree.

User stands in Kinect center in standard area, and control limitation is larger, in order to allow a user to There is a range of movement on Kinect or so direction, as a further improvement on the present invention, present invention additionally comprises In in part 2 b step, when the palm of the hand of user deviates Kinect depth transducer centers, in addition to incite somebody to action inclined From Kinect depth transducers center palm of the hand two-dimensional signal be corrected to center palm of the hand two-dimensional signal process, and It can be realized by following steps：First, the distance difference that center is brandished before and after correcting is calculated；Then by difference actual Palm of the hand coordinates correction is to reference area.Updating formula isWherein (x_middle,y_middle) be Center is brandished before correction, (x₀,y₀) to brandish center after correction, (x_before,y_before) for conversion before the palm of the hand position, (x_now,y_now) for conversion after palm of the hand position.So far the control of the user obtained in Kinect parts by Kinect device Intention can be converted into control signal by server and the equipment in space is controlled.

Secondly in inertial navigation module (AHRS) control system, using the seamless of intelligent space distributed vision and inertial navigation module With reference to by the control information of the location information and inertial navigation module of distributed vision, both, which combine, improves control intention analysis Accuracy.

User inputs voice command, the language that voice command passes through computer system by wireless microphone in Voice command Sound identification division carries out command recognition and parsing, and is matched with the control command in voice command storehouse, wherein using Microsoft The mode that is combined with autonomous voice match file of speech recognition library, further increase the recognition efficiency of Chinese with it is accurate Degree.If the match is successful, equipment control instruction is sent by corresponding control command, realizes the Voice command to equipment.

In two-dimensional code scanning network mobile phone control section, service uses B/S (Browser-Server) framework, and client can make Adopted with traditional PC, mobile device and any equipment for being equipped with browser by the browser access service, key technology therein Designed with the response type based on Bootstrap frameworks；Response type Web designs (Responsive Web design)：The page is set Meter should be corresponding according to user behavior and facility environment (system platform, screen size, screen orientation etc.) progress with exploitation Response and adjustment.Specific practice mode is made up of many-side, including elastic mesh and layout, picture, CSS media query Use etc.；Bidirectional real-time based on WebSocket, when intelligent space equipment state changes, by real-time informing intelligence This notice is also synchronized to each service client by energy spatial server, server.In addition, its communication is complete by TCP80 ports Into this is advantageous to prevent the client network environment of non-Web Internet connections from using using fire wall.

In Software for Design, in scheduling portion, the mobile devices such as each robot are linked into calculating by wireless network Machine system.User by client access computer system, client can to computer system provide subscriber identity information, and to Computer system sends user task application.After computer system receives user's application, application is parsed, application is obtained and appoints The user identity of business, application time, the subtask included, the priority etc. of task, it is after being parsed, task is excellent according to task First level is loaded into task list.Normal priority task, it will be performed according to loading sequence；Higher priority task will be obtained preferentially It must perform；Limit priority task must perform in advance, and can interrupt the low priority task being carrying out, and preferentially be performed. Computer system will be carried out task scheduling to the task in task list, be finally completed user's application according to dispatching algorithm.Scheduling The principle that algorithm follows is carried out according to the rule in the 6th part f, g.The scheduling of the kind equipment employs relatively flexible movement and set Standby scheduling rule, realize the differentiation executive mode of different stage priority.

Part is planned in service, in order to improve the processing time for occurring multitask or hot job under intelligent space, is improved The reasonability of service procedure, part is planned in service procedure, service planning is carried out using driving parameter mechanism.This method shortens The time that service planning uses, it in case of emergency can quickly react, improve the efficiency of service in intelligent space And service quality.Specific planning mode forms planning process according to the step in the 7th part b.

Brief description of the drawings

Fig. 1 system framework figures of the present invention；

Fig. 2 PC controls interface；

Fig. 3 Kinect outside drawings；

Fig. 4 palm of the hand three-dimensional position explanation figures；

Fig. 5 control areas division figure；

Palm of the hand moving range figure when Fig. 6 is distance Kinect depth transducers center different depth；

Fig. 7 is that the palm of the hand two-dimensional signal that will deviate from Kinect depth transducers center is corrected to the palm of the hand of center The correction principle figure of two-dimensional signal；

Fig. 8 inertial navigations module uses illustration；

Fig. 9 mobile phone control program schematic diagrames；

Figure 10 dispatcher software service charts；

Figure 11 service driving schematic diagrames；

Figure 12 planning algorithm frame diagrams；

Figure 13 flow layout results.

Embodiment

Just the implementation of the present invention is described in detail below in conjunction with the accompanying drawings, but not limited to this.

Embodiment 1,

A kind of management and planning system based on WIFI wireless networks multiclass control terminal, including：

(1) hardware system based on WIFI is built under intelligent space, the hardware system based on WIFI includes intelligence Equipment, intelligent terminal and server, described smart machine, intelligent terminal respectively with the server wireless connection；

(2) terminal access server will be controlled, the control terminal includes：Remote handset control, server end voice control System, the gesture control based on Kinect and inertial navigation module and the control of server end body；The server end body controls： The smart machine in intelligent space, intelligent terminal are directly controlled by way of button at the management software interface of server end System；

(3) design server end is to all smart machines, the optimizing scheduling algorithm and mission planning algorithm of intelligent terminal, with Realize control and service ubiquitous under intelligent space；

The specific steps of hardware system of the structure based on WIFI comprise the following steps in the step (1):

A, the smart machine in intelligent space and intelligent terminal are matched using WIFI wireless network modules, made described By wireless network by remote auto control, described automatically controls including the smart machine, intelligence for smart machine, intelligent terminal Energy terminal is controlled by control terminal control or being serviced device end body；The smart machine and intelligent terminal are to be controlled by wireless network Electric light, TV, air-conditioning and curtain of network etc.；The WIFI wireless network modules use STM32 to pass through string for bottom control core Mouth communicates with RM04 modules, and the networking mode i.e. control core for taking many-one to broadcast corresponds to multiple servers and used Transmission Control Protocol carries out data transmission, and improves the stability of system in programming using Socket non-blocking modes, and the server includes Mobile phone, computer, improve the compatibility to control device；

B, the hardware of data transfer includes router and RM04 modules in step a,

The router effect is that the WIFI wireless networks of local are forwarded into public network so as to a public network as system Interface and cradle head camera is accessed by network by wds patterns；Dynamic Host Configuration Protocol server is by identifying RM04 modules in the router Then MAC Address distributes corresponding IP address of internal network, then internal address is forwarded on public network, the RM04 modules play by The effect of bottom control core couple in router, the RM04 modules are configured to network interface card pattern, are serviced in a local network as TCP Device is present；

C, the circuit of bottom control core produces different pulse signals to difference by STM32 single-chip microcomputers control relay Smart machine and intelligent terminal are controlled, and the circuit of the bottom control core also includes control interface for extending a variety of sensings Device；

D, in the communication of upper strata, each smart machine and intelligent terminal pass through the distribution of router with obtaining unique IP Location, server are controlled by IP address to smart machine and intelligent terminal；

In bottom communication, WIFI wireless network modules are communicated by serial mode and the STM32 single-chip microcomputers；Institute State STM32 single-chip microcomputers controls the switch of smart machine and intelligent terminal to run by way of control relay；

In communication process, each smart machine and intelligent terminal use with server by using TCP/IP communication agreement Socket programming modes realize the transmission of data or order；

E, according to step a, b, c, d and WIFI wireless networks and cable network are based on, centered on router, by Intelligent air Between in can be connected into each smart machine, intelligent terminal, server and the sensing equipment that can be communicated with server of network It is connected in a LAN, and each smart machine and intelligent terminal have the function of link internet；Make whole system Form tasks carrying part, device control unit point and server and data storage section；

It is as follows that Kinect gesture controls part described in the step (2) includes step：

F, Kinect is accessed into computer system；

G, by starting tracking of the Kinect to the palm of the hand, the positional information of the palm of the hand, and the depth information of the palm of the hand are obtained, The position of palm of the hand distance Kinect depth transducers center 1000-3000mm depth, enters to the two dimensional surface region where the palm of the hand The division of row functional area, laser pen region, area of isolation, control area are divided into, and define the palm of the hand in each region The instruction of the corresponding operation of diverse location, and make following provisions：Gauged distance be immediately ahead of the Kinect at 1 meter away from From；Standard flat is the plane where the four corner of palm of the hand movement at gauged distance；Standard area is the palm of the hand in standard Image range during planar movement on Kinect, and it is defined as rectangular area；The center of brandishing is that user stretches in front When going out the right hand, palm of the hand position residing in scope is entirely brandished, under different depth information；Reference area is palm of the hand movement Center of the scope on Kinect images, i.e. people caused by the center in Kinect directions on image perpendicular to waving Region；

H, computer is instructed according to gesture control, and the corresponding behaviour to smart machine and intelligent terminal is realized by network communication Make；The effect of contactless property operation is realized, this method avoids the research to gesture path, simplifies identification difficulty, and carry High discrimination；

It is as follows that inertial navigation module (AHRS) control section described in the step (2) includes step：

I, inertial navigation module (AHRS) is accessed into computer by communication；

J, merged by the Data Data of accelerometer and gyroscope magnetometer in inertial navigation module, the control of analysis and Control person Posture, inertial navigation module gather the signal of three axis MEMS gyro, and using quaternary number posture expression formula, integration tries to achieve gyroscope Attitude angle, while 3 axis MEMS accelerometer and the signal of three axle magnetometer are gathered, using gravitational field and geomagnetic field in geography Direction cosines conversion between coordinate system and body axis system carries out absolute angle resolving, obtains absolute pose angle, then basis The change frequency of absolute pose angle signal changes filtering parameter in real time, and the attitude angle obtained twice is carried out to be based on expanded type karr The data fusion of graceful filtering, the attitude angle and fusion acceleration of final output accurate stable, wherein, accelerometer and gyroscope Fusion principle repaiied with the value of accelerometer amendment gyroscope, when dynamic using the value of gyroscope when be mainly static The value of positive acceleration meter, the 3-axis acceleration (Ax, Ay, Az) exported according to the three axis accelerometer on movable body, is asked for respectively Angle of pitch pitch=tan^-1(- Ay ,-Az), inclination angleMerge accelerationMagnetometer is mainly to carry out magnetometer using accelerometer with the principle that merges of accelerometer Slope compensation, read the three-axle magnetic field intensity of magnetometer outputAsk for the magnetometer after slope compensation Output

WhereinExported for the magnetometer after slope compensation,For three axle magnetic of magnetometer output Field intensity, roll are inclination angle, and pitch is the angle of pitch；

Exported according to the magnetometer after the slope compensation calculated, ask for yaw angleObtain not After the angle value and acceleration of disconnected renewal, increase the reliability of data by the way of sliding window, related in the inertial navigation module And two kinds of control modes：Point to by control smart machine and intelligent terminal fire finger control and upward refer to fire finger control it is specific Pattern；When final control signal determines, by the way of threshold value：Judged according to the output valve of accelerometer, if the numerical value Then it is judged as that control instruction is sent more than threshold value.

K, effector position is carried out by distributed vision, the i.e. binocular with monopod video camera or multi-vision visual system Positioning, the effector are the sharp user that is possessed of control power to smart machine, intelligent terminal in intelligent space；According to inertial navigation mould The position of the angle information of block and current effector draw the angle that people faces, and judge what effector to be controlled by the angle Smart machine and intelligent terminal, after control signal is sent, controlled device is determined according to this angle information to be controlled；

It is as follows that Voice command part described in the step (2) includes step：

L, wireless microphone receiving terminal is accessed into computer；

M, user inputs voice command by wireless microphone, and voice command is carried out by the speech recognition part of computer Command recognition and parsing, and matched with the control command in voice command storehouse, if the match is successful, by controlling accordingly System order sends equipment control instruction, realizes the Voice command to equipment；

The control of remote handset described in the step (2) comprises the following steps：

N, server end uses Windows+Apache+SQLServer+PHP web application platform；Service uses B/S (Browser-Server) framework, effector are passed through clear using traditional PC, mobile device and any equipment for being equipped with browser Device of looking at accesses the server, has good cross-platform characteristic；

O, whole service application is divided into：Presentation layer (UI), Business Logic (BLL), data access layer (DAL)；Institute The responsible operation to database of data access layer is stated, data, services are provided for Business Logic；Business Logic concern business rule The designs relevant with business demand such as formulation then, the realization of operation flow；Presentation layer is used for display data and reception user is defeated The data entered, provide the user the interface of interactive operation；

P, response type Web designs (Responsive Web design)：The design and development of the page should be according to user's row For and facility environment (system platform, screen size, screen orientation etc.) carry out accordingly respond and adjust, no matter user Using traditional PC or smart mobile phone, Pad, the page should be able to all automatically switch resolution ratio, dimension of picture and associated script work( Can etc., to adapt to distinct device；I.e.：The page can adaptive user facility environment；Websocket is using independent based on TCP Offer full-duplex communication channel agreement, realize bidirectional real-time；

Step (3) the design server end includes step to the optimizing scheduling algorithm of all smart machines, intelligent terminal It is as follows：

All it has been connected in same LAN by various modes based on all smart machines, intelligent terminal, In scheduling portion, scheduled smart machine, intelligent terminal are divided into mobile device and fixed equipment, the tune on fixed equipment Degree, require to be controlled by using different control modes according to the timely position of user and control, the kind equipment is set Equal priority is set to, using requiring in dispatching algorithm, by the way of controlling；And the scheduling of mobile device is as described below：

P, each mobile device is linked into computer system by wireless network；Robot in intelligent space belongs to shifting Dynamic equipment；

Q, user is by client access server, and client can provide subscriber identity information to server, and to service Device sends user task application；The client is the software that user is controlled to mobile device, and it can be in server sheet Body can also be on the pc that can be communicated with server；

I, after server receives user's application, application is parsed, when obtaining user identity, the application of application task Between, the subtask that includes, the priority of task, after being parsed, task is loaded into task list according to task priority；

S, normal priority task, will be performed according to loading sequence；Higher priority task will be performed preferentially；Highest Priority tasks must perform in advance, and can interrupt the low priority task being carrying out, and preferentially be performed；

T, server will be carried out task scheduling to the task in task list, be finally completed user Shen according to dispatching algorithm Please；

U, the principle that dispatching algorithm follows：

I. user's equality, performance is both ways：When during the task of All factors being equal, preference will be give in list level, according to the application time successively Perform, it is unrelated with task object；Second, limit priority task only when currently without idle mobile devices, just forces to terminate just In the low priority task of execution, otherwise preferentially task is completed using idle mobile devices；

II. the energy most saves, if showing that task includes multiple subtasks, transfers to that these can be fully completed as far as possible The mobile device of subtask is completed, and avoids completing using multiple mobile devices；

V, dispatching algorithm flow：

Step1：A mission bit stream is obtained from task list；

Step2：The precedence information of the task is checked, if limit priority task, then Step3 is performed, otherwise performs Step6；

Step3：Check whether the mobile device for being not carried out limit priority task be present, if so, then moving these Equipment is divided into four classes, is respectively can complete all subtasks and idle mobile device, can complete part subtask and idle Mobile device, all subtasks and busy mobile device can be completed, part subtask can be completed and busy movement is set It is standby；Otherwise Step8 is performed；

Step4：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether to deposit In the second class mobile device, exist, can detect the second class mobile device complete whole subtasks, if can, moved by the second class Dynamic equipment performs the task, otherwise detects whether the 3rd class mobile device be present, exists then to be performed by the 3rd class mobile device and appoints Business, if, by the second class mobile device executable portion subtask, remaining subtask transfers to the 4th class mobile device to complete without if；If Without the second class mobile device, then when the 3rd class mobile device be present, there is the 3rd class mobile device to complete task, without the 3rd Class mobile device then completes task by the 4th class mobile device；

Step5：Step8 is performed after the completion of task；

Step6：Available free mobile device is checked whether, mobile device is divided into two classes if having, can respectively complete institute There is the mobile device of subtask and free time, the mobile device of part subtask and free time can be completed；

Step7：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether to deposit In the second class mobile device, task is performed by the second class mobile device if having；

Step8：If whole subtasks in the task have been performed both by, the task is deleted, if having only carried out part Subtask, then the subtask that executed is complete in the task is deleted, but the task still retains, if the task does not have mobile device can To perform, then the task is temporarily skipped；

Step9：Step1 is performed, obtains a new task, this process is repeated, until the task in list is all held OK；

Step (3) the design server end is to all smart machines, the mission planning algorithm of intelligent terminal, including step It is as follows：

W, each service addition Service name, such as RobotMove (robot movement) are given, driving parameter is added for the service As (E, F, G) (position, angle and velocity information etc.), and the service execution complete later output parameter (I, J, K) (figure Picture, target location and targeted attitude information etc.)；The working condition of also this service is respectively yes, no, work, system according to Its working condition selects to service：When the parameter that server receives meets at least one service, the service just by Driving, when the service execution complete can produce other parameter, can equally drive corresponding service, in this way formed planning stream Journey, in order to ensure the accuracy of flow and feasibility, algorithm part is designed as follows：

X, the specific steps in planning algorithm

Step1：Prepare parameter set, services set, active service collection and Backup collection, and final service list five is deposited Storage unit；

Step2：Whether cycle detection parameter set has parameter input to go to Step3 if parameter input, if without if Simultaneously cycle detection is waited in Step2；

Step3：According to the input parameter of parameter set, services set is scanned for, finding being capable of powered service deposit First layer, the index INDEX related to the number of plies add 1, go to Step4；

Step4：Duplicate checking is carried out to this layer of service just obtained, if the output of a service is defeated included in another In going out, this service is deleted from final service list, is placed in active service concentration；Parameter for driving this invalid service Deleted from parameter set and be placed in Backup concentration；To least recently used all parameters to prevent from repeating driving, will also delete simultaneously It is placed in Backup concentration；Go to Step5；

Step5：Detect whether this layer of service has output, if then going to Step2, if not going to Step6；

Step6：Calculate the wastage in bulk or weight of each service procedure, optimum selecting final flowsheet；

Step7：Terminate planning process, start to perform, and transfer to dispatch layer to run.

Fig. 1 is the system framework schematic diagram of the present invention.As shown in figure 1, first by intelligent space startup of server, with getting ready Complete network service, the hardware are computer, WIFI terminal, mobile hand-held device and controlled device, software Each software module that part is realized mainly under windows environment by MFC programmings and network programming, including equipment control are calculated Method module, Condition Detection module, equipment scheduling algoritic module, network server program, service planning implementation module.Wherein Hardware device has：Service robot, distributed vision, Temperature Humidity Sensor, air-conditioning, light units, monopod video camera, TV and Electrically driven curtain equipment etc..

The said equipment can be improved by WIFI control modules, be reached by the purpose of wireless network control.It mainly by RM04 modules, single-chip computer control system, relay circuit three parts composition.RM04 module main functions be play single-chip microcomputer with Function served as bridge between server.ICP/IP protocol is converted into serial port protocol so as to realize the communication of server and single-chip microcomputer. The wifi network of router is directly accessed using server mode of operation.Pass through serial ports and single chip communication, single-chip microcomputer control again Relay circuit realizes the control to electrical equipment.

After completion equipment end is built, programming apparatus control module, realizes the simple control of equipment in the server, such as schemes Shown in 2：The figure is the pc control procedure of computer, mainly realizes the aobvious of control to bottom all devices and monopod video camera Show, the movement of monopod video camera, focus operations can be realized by the program, and can flexibly realize the addition of equipment.The program To write be in the WifiControl classes based on bottom, control program is packaged into bottom class, improves the portable of program Property, it is also convenient for the use of user.

After equipment end deploys, the design at end is then controlled, first using gesture control mode to equipment It is controlled.

In gesture control, the gesture of people is detected using Kinect device first and control command is sent by server. By starting tracking of the Kinect to the palm of the hand, equipment is as shown in Figure 3.Application program can be obtained by OpenNI api function Positional information of the palm of the hand in picture in its entirety, and the depth information of the palm of the hand.As shown in Figure 4：A, b represent palm of the hand position in two dimension Position in image.C represents depth information of the palm of the hand apart from depth transducer.This is by call back function Hand_Update () The three-dimensional data of return.

First in the two dimensional surface that depth information is 1000mm, whole region is divided into two large divisions by the present invention：Meeting Schema control region and device control area domain.Conference model control area mainly utilizes the position of hand in space, utilizes hand Change the depth information of the palm of the hand in posture short time for pushing away forward to complete；Control area is then to be according to the position judgment of the palm of the hand No control light air-conditioning opens or closes.According to the division in above-mentioned two big regions, make following provisions：Gauged distance is to be located at Distance immediately ahead of Kinect at 1 meter；Standard flat is the putting down where the four corner of palm of the hand movement at gauged distance Face；Standard area is image range of the palm of the hand when standard flat moves on Kinect, it is specified that being rectangular area；Control centre When being that user stretches out the right hand in front, palm of the hand position residing in scope is entirely brandished can be under different depth information； Reference area is center of the palm of the hand moving range on Kinect images, i.e., people is perpendicular to the center in Kinect directions institute Control area on caused image.

Control area is divided into region as shown in Figure 5：Picture in its entirety is divided into positioned at the conference model control zone at center Domain and control area two parts of surrounding.In order to reach good control effect, user is controlled under normal place as far as possible System.If not being in gauged distance, because the moving range of the palm of the hand is fixed, but distance is more remote, and the palm of the hand is on picture Moving range can reduce, as shown in Figure 6.In order to solve the above problems, according to the depth information of palm of the hand position, using to measuring number According to analysis two-dimensional signal of the palm of the hand in different depth position is uniformly transformed into standard area by least square method.Pass through It is identical with the control effect in standard area that depth information conversion can ensure that speaker in any depth makes gesture.Conversion Design sketch it is as shown in Figure 7.

Another control mode is to use inertial navigation module, (has the binocular or more of monopod video camera by distributed vision Mesh vision system) effector position is positioned, data are merged with AHRS PDR and angle information using Kalman Algorithm Draw the position and direction of effector.Pass through the data fusion of accelerometer in AHRS and gyroscope and magnetometer, analysis and Control The control posture of person, it is related to two kinds of control modes in of the invention：Sensing controlled device is fired finger control and referred to upward and fires Refer to control control AD HOC.Control schematic diagram is as shown in Figure 8.

After gesture control is arranged, the present invention also combines network and mobile phone control, is used in Network Control Segment Windows+Apache+SQLServer+PHP web application platform.Service uses B/S (Browser-Server) frame Structure, traditional PC, mobile device and any equipment for being equipped with browser can be used by the browser access service in client, thus has There is good cross-platform characteristic；When service needs to improve or upgraded, it is only necessary to server program is updated, and client is without dimension Shield, greatly reduces software maintenance cost.The software uses B/S software architectures, and it is a kind of network structure mould after WEB rises Formula, web browser are the most important application software of client.This pattern has unified client, the core that systemic-function is realized Center portion point is focused on server, simplifies the exploitation, maintenance and use of system.Therefore, the service uses B/S (Browser- Server) framework, server end use Windows+Apache+SQLServer+PHP web application platform, browser End is the browser on user equipment.So, as long as customer equipment is equipped with browser in itself, you can the service is accessed, thus There is extraordinary professional platform independence.Meanwhile if user's request changes, then need only to the software maintenance of server end, it is not necessary to User is notified to update client software, therefore the maintenance cost of software is very low.

Whole service application is divided into：Presentation layer (UI), Business Logic (BLL), data access layer (DAL).Data Access layer is responsible for the operation to database, and data, services are provided for Business Logic；Business Logic pays close attention to the system of business rule The designs relevant with business demand such as fixed, operation flow realization；Presentation layer is used for display data and receives the number of user's input According to providing the user the interface of interactive operation.

The key technology for the use that the part uses has：Response type design based on Bootstrap frameworks：The design of the page With exploitation should progress be corresponding rings according to user behavior and facility environment (system platform, screen size, screen orientation etc.) Should and it adjust.Specific practice mode is made up of many-side, including elastic mesh and layout, picture, CSS media query Use.No matter user is used traditional PC or smart mobile phone, Pad, and the page should be able to all automatically switch resolution ratio, figure Chip size and associated script function etc., to adapt to distinct device；In other words, the page can adaptive user facility environment；Base In WebSocket bidirectional real-time：When intelligent space equipment state changes, real-time informing intelligent space is serviced Device, server are also required to this notice being synchronized to each service client.Websocket is that the independent offer based on TCP is complete The agreement of duplex communication channel.It and HTTP only link are that its Handshake Protocol is by HTTP server as upgrade request Explain and perform.WebSocket agreements make it possible the real-time, interactive of browser and server, are easy to the wound of real-time WEB service Build.WebSocket provides one kind without browse request for server, you can the standard mode of transmission content to browser, And connection is allowed to transmit message back and forth during opening.

In addition, its communication is completed by TCP80 ports, this is advantageous to prevent non-WEB Internet connections using fire wall Client network environment uses.WebSocket agreements by Google Chrome, InternetExplorer, FireFox, Most of major browsers such as Safari, Opera are supported.

Being controlled based on network as shown in figure 9, being write using Android language, has stronger mobile device portability, By network access router corresponding interface, control of the corresponding order realization to household electrical appliances is transmitted.The introducing of mobile phone, which adds, is The interactive mode of system, make interactive mode more hommization, while realize the remote control of system.

In addition branch configures voice accepting device in intelligent space, gathers user profile, is inputted by wireless microphone Voice command, voice command carry out command recognition and parsing, and and voice command by the speech recognition part of computer system Control command in storehouse is matched, wherein the side being combined using the speech recognition library of Microsoft with autonomous voice match file Formula, further increase recognition efficiency and the degree of accuracy of Chinese.If the match is successful, sent and set by corresponding control command Standby control instruction, realizes the Voice command to equipment.

It is comprehensive in server end design equipment scheduling and service planning algorithm after equipment end and control terminal design are completed Close all control commands and equipment control mode.According to the completion for assigning completion control task of control command.Adjusted in equipment In degree, as shown in Figure 10：All schedulable equipment are uniformly included in server first, classified according to device type.Close In the scheduling of fixed equipment, required, to be controlled, the kind equipment to be arranged into equal according to the timely position of user and control Priority, using requiring in dispatching algorithm, by the way of controlling.Scheduling for mobile device, by each movable equipment (referring mainly to service robot) is linked into computer system by wireless network.User accesses computer system by client, Client can provide subscriber identity information to computer system, and send user task application to computer system.Department of computer science After system receives user's application, application is parsed, obtain the user identity of application task, the application time, the subtask included, Priority of task etc., after being parsed, task is loaded into task list according to task priority.Normal precedence is in charge of a grade Business, will be performed according to loading sequence；Higher priority task will be performed preferentially；Limit priority task must perform in advance, And the low priority task being carrying out can be interrupted, preferentially performed.Computer system will be arranged task according to dispatching algorithm Task in table carries out task scheduling, is finally completed user's application.The principle that dispatching algorithm follows is according in the 6th part f, g Rule is carried out.The scheduling of the kind equipment employs relatively flexible mobile device scheduling rule, realizes different stage priority Differentiation executive mode.

Realize what the requirement automatic sensing to user was independently serviced during in order to be run in intelligent space system Purpose.The automatic planning module of service is with the addition of in server software, the data gathered according to sensor device, task is produced and drives Dynamic information, drives respective service, ultimately forms service procedure and preferentially perform.Each service addition is given using driving parameter mechanism Service name, such as RobotMove, as shown in figure 11：Driving parameter is added for the service such as (E, F, G), and the service execution Complete later output parameter (I, J, K).The working condition of also this service is respectively yes, no, work, and system is according to it Working condition select to service.When the parameter that server receives can meet at least one service, the service is just Driven, when the service execution complete can produce other parameter, corresponding service can be equally driven, according in the 7th part b It is as shown in figure 12 that step forms planning process.Final execution service procedure is as shown in figure 13.

According to the design of hardware components and the combination of each module of server software, the present invention realizes the energy in intelligent space It is enough that neatly various smart machines are scheduled, complete multiple control modes in intelligent space and merge.Detect at any time simultaneously The intention of user, analyzes user's request, and autonomous provides the user service.Intellectual space technique is set further to be stepped to practical Enter.

Claims (2)

1. A kind of 1. method of work of management and planning system based on WIFI wireless networks multiclass control terminal, it is characterised in that The system includes：

   (1) under intelligent space build the hardware system based on WIFI, the hardware system based on WIFI include smart machine, Intelligent terminal and server, described smart machine, intelligent terminal respectively with the server wireless connection；

   (2) terminal access server will be controlled, the control terminal includes：Remote handset control, server end Voice command, base Controlled in the gesture control and server end body of Kinect and inertial navigation module；The server end body controls：Servicing The management software interface at device end is directly controlled the smart machine in intelligent space, intelligent terminal by way of button；

   (3) design server end is to all smart machines, the optimizing scheduling algorithm and mission planning algorithm of intelligent terminal；

   The specific steps of hardware system of the structure based on WIFI comprise the following steps in the step (1):

   A, the smart machine in intelligent space and intelligent terminal are matched using WIFI wireless network modules, makes the intelligence Equipment, intelligent terminal are whole including the smart machine, intelligence by remote auto control, described automatically controlling by wireless network End is controlled by control terminal control or being serviced device end body；The WIFI wireless network modules use STM32 as bottom control Core, communicated by serial ports with RM04 modules, the networking mode i.e. control core for taking many-one to broadcast corresponds to multiple clothes Business device is simultaneously carried out data transmission using Transmission Control Protocol, improves the stability of system in programming using Socket non-blocking modes, described Server includes mobile phone, computer；

   B, the hardware of data transfer includes router and RM04 modules in step a,

   The router effect is that the WIFI wireless networks of local are forwarded into public network so as to a public network interface as system Cradle head camera is accessed into network with by wds patterns；Dynamic Host Configuration Protocol server is by identifying the MAC of RM04 modules in the router Then corresponding IP address of internal network is distributed in location, then internal address is forwarded on public network, and the RM04 modules are played bottom The effect of control core couple in router, the RM04 modules are configured to network interface card pattern, deposited in a local network as TCP server ；

   C, the circuit of bottom control core produces different pulse signals to different intelligent by STM32 single-chip microcomputers control relay Equipment and intelligent terminal are controlled, and the circuit of the bottom control core also includes control interface for extension multiple sensors；

   D, in being communicated on upper strata, each smart machine and intelligent terminal obtain unique IP address, clothes by the distribution of router Business device is controlled by IP address to smart machine and intelligent terminal；

   In bottom communication, WIFI wireless network modules are communicated by serial mode and the STM32 single-chip microcomputers；It is described STM32 single-chip microcomputers control the switch of smart machine and intelligent terminal to run by way of control relay；

   In communication process, each smart machine and intelligent terminal use with server by using TCP/IP communication agreement Socket programming modes realize the transmission of data or order；

   E, according to step a, b, c, d and WIFI wireless networks and cable network are based on, centered on router, by intelligent space Each smart machine of network, intelligent terminal, server and the sensing equipment that can be communicated with server can be connected into be connected to In one LAN, and each smart machine and intelligent terminal have the function of link internet；Form whole system Tasks carrying part, device control unit point and server and data storage section；

   It is as follows that Kinect gesture controls part described in the step (2) includes step：

   F, Kinect is accessed into computer system；

   G, by starting tracking of the Kinect to the palm of the hand, the positional information of the palm of the hand, and the depth information of the palm of the hand are obtained, in the palm of the hand The position of distance Kinect depth transducers center 1000-3000mm depth, work(is carried out to the two dimensional surface region where the palm of the hand The division in energy region, is divided into laser pen region, area of isolation, control area, and it is different in each region to define the palm of the hand The instruction of the corresponding operation in position, and make following provisions：Gauged distance is the distance at 1 meter immediately ahead of Kinect；Mark Directrix plane is the plane where the four corner of palm of the hand movement at gauged distance；Standard area is the palm of the hand in standard flat Image range when mobile on Kinect, and it is defined as rectangular area；The center of brandishing is that user stretches out the right side in front During hand, palm of the hand position residing in scope is entirely brandished, under different depth information；Reference area is palm of the hand moving range Center on Kinect images, i.e. people are perpendicular to the area of waving caused by the center in Kinect directions on image Domain；

   H, computer instructs according to gesture control, is realized by network communication to smart machine and the corresponding operating of intelligent terminal；

   It is as follows that inertial navigation module (AHRS) control section described in the step (2) includes step：

   I, inertial navigation module (AHRS) is accessed into computer by communication；

   J, merged by the Data Data of accelerometer and gyroscope magnetometer in inertial navigation module, the control appearance of analysis and Control person State, inertial navigation module gather the signal of three axis MEMS gyro, and using quaternary number posture expression formula, integration tries to achieve gyroscope appearance State angle, while 3 axis MEMS accelerometer and the signal of three axle magnetometer are gathered, sat using gravitational field and geomagnetic field in geography Direction cosines conversion between mark system and body axis system carries out absolute angle resolving, absolute pose angle is obtained, then according to absolutely Filtering parameter is changed in real time to the change frequency of attitude angle signal, the attitude angle obtained twice is carried out to be based on expanded type Kalman The data fusion of filtering, the attitude angle of final output accurate stable and fusion acceleration, wherein, accelerometer and gyroscope Principle is merged when be mainly static with the value of accelerometer amendment gyroscope, the value amendment of gyroscope is utilized when dynamic The value of accelerometer, the 3-axis acceleration (Ax, Ay, Az) exported according to the three axis accelerometer on movable body, asks for bowing respectively Elevation angle pitch=tan^-1(- Ay ,-Az), inclination angleMerge accelerationMagnetometer is mainly to carry out magnetometer using accelerometer with the principle that merges of accelerometer Slope compensation, read the three-axle magnetic field intensity of magnetometer outputAsk for the magnetometer after slope compensation Output

   <mrow> <msubsup> <mi>M</mi> <mi>x</mi> <mi>n</mi> </msubsup> <mo>=</mo> <msubsup> <mi>M</mi> <mi>x</mi> <mi>b</mi> </msubsup> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mrow> <mo>(</mo> <mi>r</mi> <mi>o</mi> <mi>l</mi> <mi>l</mi> <mo>)</mo> </mrow> <mo>+</mo> <msubsup> <mi>M</mi> <mi>y</mi> <mi>b</mi> </msubsup> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mrow> <mo>(</mo> <mi>p</mi> <mi>i</mi> <mi>t</mi> <mi>c</mi> <mi>h</mi> <mo>)</mo> </mrow> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mrow> <mo>(</mo> <mi>r</mi> <mi>o</mi> <mi>l</mi> <mi>l</mi> <mo>)</mo> </mrow> <mo>-</mo> <msubsup> <mi>M</mi> <mi>z</mi> <mi>b</mi> </msubsup> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mrow> <mo>(</mo> <mi>p</mi> <mi>i</mi> <mi>t</mi> <mi>c</mi> <mi>h</mi> <mo>)</mo> </mrow> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mrow> <mo>(</mo> <mi>r</mi> <mi>o</mi> <mi>l</mi> <mi>l</mi> <mo>)</mo> </mrow> </mrow>

   <mrow> <msubsup> <mi>M</mi> <mi>y</mi> <mi>n</mi> </msubsup> <mo>=</mo> <msubsup> <mi>M</mi> <mi>y</mi> <mi>b</mi> </msubsup> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mrow> <mo>(</mo> <mi>p</mi> <mi>i</mi> <mi>t</mi> <mi>c</mi> <mi>h</mi> <mo>)</mo> </mrow> <mo>+</mo> <msubsup> <mi>M</mi> <mi>z</mi> <mi>b</mi> </msubsup> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mrow> <mo>(</mo> <mi>p</mi> <mi>i</mi> <mi>t</mi> <mi>c</mi> <mi>h</mi> <mo>)</mo> </mrow> </mrow>

   WhereinExported for the magnetometer after slope compensation,The three-axle magnetic field intensity exported for magnetometer, Roll is inclination angle, and pitch is the angle of pitch；

   Exported according to the magnetometer after the slope compensation calculated, ask for yaw angleObtain constantly more After new angle value and acceleration, two kinds of control modes are related in the inertial navigation module：Point to by control smart machine and intelligence Terminal, which is fired finger control and referred to upward, fires finger control AD HOC；When final control signal determines, using the side of threshold value Formula：Judged according to the output valve of accelerometer, be judged as that control instruction is sent if the numerical value exceedes threshold value；

   K, effector position is positioned by distributed vision, the effector is to smart machine, intelligence in intelligent space Can terminal be possessed of control power profit user；Show that people faces according to the position of the angle information of inertial navigation module and current effector Angle, effector's smart machine and intelligent terminal to be controlled is judged by the angle, after control signal is sent, according to This angle information determines controlled device to be controlled；

   It is as follows that Voice command part described in the step (2) includes step：

   Wireless microphone receiving terminal is accessed into computer；

   M, user inputs voice command by wireless microphone, and voice command carries out order by the speech recognition part of computer Identification and parsing, and matched with the control command in voice command storehouse, if the match is successful, by controlling life accordingly Order sends equipment control instruction, realizes the Voice command to equipment；

   The control of remote handset described in the step (2) comprises the following steps：

   N, server end uses Windows+Apache+SQLServer+PHP web application platform；Service uses B/S (Browser-Server) framework, effector is using traditional PC, mobile device and any equipment for being equipped with browser by browsing Device accesses the server；

   O, whole service application is divided into：Presentation layer (UI), Business Logic (BLL), data access layer (DAL)；The number It is responsible for the operation to database according to access layer, data, services is provided for Business Logic；Business Logic concern business rule The designs relevant with business demand such as formulation, the realization of operation flow；Presentation layer is used for display data and receives user's input Data, provide the user the interface of interactive operation；

   P, response type Web is designed：The design and development of the page should be rung accordingly according to user behavior and facility environment Should and it adjust；

   It is as follows that step (3) the design server end includes step to the optimizing scheduling algorithm of all smart machines, intelligent terminal：

   All it has been connected in same LAN, has been dispatched by various modes based on all smart machines, intelligent terminal In part, scheduled smart machine, intelligent terminal are divided into mobile device and fixed equipment, on the scheduling of fixed equipment, root Require to be controlled by using different control modes according to the timely position and control of user, the kind equipment is arranged to phase Isopreference level, using requiring in dispatching algorithm, by the way of controlling；And the scheduling of mobile device is as described below：

   P, each mobile device is linked into computer system by wireless network；

   Q, for user by client access server, client can provide subscriber identity information to server, and be sent out to server Send user task application；The client is the software that user is controlled to mobile device, and it can be in ontology server Can be on the pc that can be communicated with server；

   I, after server receives user's application, application is parsed, obtains user identity, application time, the bag of application task The subtask that contains, the priority of task, after being parsed, task is loaded into task list according to task priority；

   S, normal priority task, will be performed according to loading sequence；Higher priority task will be performed preferentially；Highest is preferential It is in charge of a grade to perform in advance, and the low priority task being carrying out can be interrupted, preferentially performed；

   T, server will be carried out task scheduling to the task in task list, be finally completed user's application according to dispatching algorithm；

   U, the principle that dispatching algorithm follows：

   I. user's equality, performance is both ways：First, during the task of All factors being equal, preference will be give in list level, held successively according to the application time OK, it is unrelated with task object；Second, limit priority task only when currently without idle mobile devices, just forces to terminate The low priority task of execution, otherwise preferentially complete task using idle mobile devices；

   II. the energy most saves, if showing that task includes multiple subtasks, transfers to that this little can be fully completed as far as possible The mobile device of business is completed, and avoids completing using multiple mobile devices；

   V, dispatching algorithm flow：

   Step1：A mission bit stream is obtained from task list；

   Step2：The precedence information of the task is checked, if limit priority task, then Step3 is performed, otherwise performs Step6；

   Step3：Check whether the mobile device for being not carried out limit priority task be present, if so, then by these mobile devices It is divided into four classes, is respectively the movement that can be completed the mobile device of all subtasks and free time, part subtask and free time can be completed Equipment, all subtasks and busy mobile device can be completed, part subtask and busy mobile device can be completed；It is no Then perform Step8；

   Step4：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether there is the Two class mobile devices, are present, and can detect the second class mobile device complete whole subtasks, if can, set by the movement of the second class It is standby to perform the task, otherwise detect whether the 3rd class mobile device be present, exist and task is then performed by the 3rd class mobile device, if Without then by the second class mobile device executable portion subtask, remaining subtask transfers to the 4th class mobile device to complete；If no Second class mobile device, then when the 3rd class mobile device be present, there is the 3rd class mobile device to complete task, being moved without the 3rd class Dynamic equipment then completes task by the 4th class mobile device；

   Step5：Step8 is performed after the completion of task；

   Step6：Available free mobile device is checked whether, mobile device is divided into two classes if having, is respectively that can complete all sons The mobile device of task and free time, the mobile device that part subtask and free time can be completed；

   Step7：If first kind mobile device be present, the execution task nearest away from user is selected；Otherwise check whether there is the Two class mobile devices, task is performed by the second class mobile device if having；

   Step8：If whole subtasks in the task have been performed both by, the task is deleted, is appointed if having only carried out part Business, then delete the subtask that executed is complete in the task, but the task still retains, if the task does not have mobile device to hold OK, then the task is temporarily skipped；

   Step9：Step1 is performed, obtains a new task, repeats this process, until the task in list all performs.
2. 2. the work of a kind of management and planning system based on WIFI wireless networks multiclass control terminal according to claim 1 Make method, it is characterised in that mission planning of step (3) the design server end to all smart machines, intelligent terminal is calculated Method, including step are as follows：

   W, each service addition Service name is given；The working condition of also this service is respectively yes, no, work, and system is according to it Working condition select to service：When the parameter that server receives meets at least one service, the service is just driven It is dynamic, when the service execution completes that other parameter can be produced, corresponding service can be equally driven, forms planning stream in this way Journey, in order to ensure the accuracy of flow and feasibility, algorithm part is designed as follows：

   X, the specific steps in planning algorithm

   Step1：Prepare parameter set, services set, active service collection and Backup collection, and five storage lists of final service list Member；

   Step2：Cycle detection parameter set whether have parameter input if parameter input goes to Step3, if not Step2 is waited and cycle detection；

   Step3：According to the input parameter of parameter set, services set is scanned for, finding being capable of powered service deposit first Layer, the index INDEX related to the number of plies add 1, go to Step4；

   Step4：Duplicate checking is carried out to this layer of service just obtained, if the output of a service is included in another output In, this service is deleted from final service list, is placed in active service concentration；For drive the parameter of this invalid service also from Deleted in parameter set and be placed in Backup concentration；To least recently used all parameters to prevent from repeating driving, juxtaposition will be also deleted Concentrated in Backup；Go to Step5；

   Step5：Detect whether this layer of service has output, if then going to Step2, if not going to Step6；

   Step6：Calculate the wastage in bulk or weight of each service procedure, optimum selecting final flowsheet；

   Step7：Terminate planning process, start to perform, and transfer to dispatch layer to run.