Summary of the invention
The invention aims to solve the problems of the prior art, a kind of plane based on characteristic pattern identification is provided
Gestural control system and control method.
The present invention is achieved by the following technical solutions, and the present invention proposes a kind of plane hand based on characteristic pattern identification
Gesture control system, the system comprises master controller, operation platform module, data acquisition module, data processing module, data are logical
Believe module and controlled plant module;
The operation of other each modules of the main controller controls simultaneously monitors each module operating status in real time;Operation platform module
For the manipulation region of user's Freehandhand-drawing characteristic pattern or in advance with the body surface region of characteristic pattern, user can be in operating platform
Gesture motion and system interaction are made in marked region in module;Data acquisition module captures operating space by camera in real time
Interior gesture motion, and image data is passed to data processing module;Data processing module is extracted from incoming image data
Hand-characteristic identifies gesture command, and incoming data communication module;Data communication module receives gesture command, and sets to controlled
Standby module sends corresponding function instruction;Controlled plant module receive capabilities instruct and execute corresponding function, while returning to controlled set
Standby operating status.
Further, the controlled plant module includes one or more controlled plants.
The present invention also proposes a kind of control method of plane gestural control system based on characteristic pattern identification,
Step 1, initial networked control systems;
Step 2, detection Freehandhand-drawing index point;
Step 3, according to index point and other relational graphs are drawn, mark positioning operation area, and identify selected function;
Step 4 detects user gesture in functional areas, when detecting gesture, executes step 5, no to then follow the steps 6;
Step 5 sends control instruction according to user gesture;
If step 6, operating space figure change, step 8 is executed, it is no to then follow the steps 7;
Step 7, iteration execute step 2 to step 6, until executing step 10 at the end of control;
If step 8, operating space figure are incomplete or disappear, step 10 is executed, it is no to then follow the steps 9;
Step 9, iteration execute step 2 to step 8, until control terminates;
Step 10, finishing control.
Further, the step 2 specifically:
Step 2.1, according to image-forming condition automatic adjusument imaging parameters;
Step 2.2 captures image using new imaging parameters;
Step 2.3 carries out color segmentation to image using the marker color of typing in initialization procedure;
Step 2.4 carries out the detection of HOG feature to the result of color segmentation, obtains each index point position and size.
Further, the step 3 specifically:
Step 3.1, according to index point testing result, step 3.2 is executed when only one index point, when there is multiple marks
Step 3.5 is executed when point, when not having index point, current operation zone state is labeled as " invalid operation area ", executes step
3.9;
If there is additional mark around step 3.2, index point, step 3.3 is executed, otherwise by index point and its proximity
Field mark is " comprising unique identification point without labeling operation area ", and executes step 3.9;
Step 3.3 identifies marked content by OCR method;
If step 3.4, marked content meet default template requirement, by index point and mark adjacent domain labeled as " packet
The operating space of point containing unique identification and mark " is otherwise labeled as " invalid operation area ", executes step 3.9;
If step 3.5, index point arrangement mode meet default template requirement, step 3.6 is executed, otherwise by each index point
Surrounding zone marker is " invalid operation area ", executes step 3.9;
If there is additional mark around step 3.6, index point, step 3.7 is executed, each index point is otherwise surrounded into region
Labeled as " comprising multiple index points without labeling operation area ", step 3.9 is executed;
Step 3.7 identifies marked content by OCR method;
If step 3.8, marked content meet default template requirement, each index point and mark are surrounded into zone marker and are
" operating space comprising multiple index points and mark " otherwise will be labeled as " invalid operation area ", and execute step 3.9;
Step 3.9 analyzes operating space function according to operating space type, mark vertex type and marked content.
It is further, described that user gesture is detected in functional areas specifically:
Step 4.1, capture image and read operation area parameter;
Step 4.2, space and the temporal aspect for extracting finger form, position and track;
Step 4.3, the feature obtained to extraction are analyzed, and gesture command is obtained.
Further, the operating space is multiple operating spaces, and user is not by drawing multiple operating spaces to same equipment or not
It is controlled with equipment.
Further, the multiple operating space is controlled by more people, and when there is multiple users, each user draws operation respectively
Area controls same or distinct device.
Further, when controlling same equipment simultaneously using multiple operating spaces, to prevent instruction conflict, control system will
It is subject to the instruction first recognized;It is controlled in subsequent 30s after identifying and executing from the instruction of a certain operating space
Equipment is only in response to the instruction from the operating space.
Further, the user can augment and/or modify on the basis of existing operating space.
Beneficial effects of the present invention:
1, user can be by Freehandhand-drawing characteristic pattern with defining operation area, and controls corresponding equipment, is compared with the traditional method behaviour
Make to be restricted on regional space less, interactive experience is more free, flexible, accurate and efficient interactive experience;
2, fading ink pen can be used to draw operating space temporarily to use for user, ink volatilization not trace after a period of time
Mark, suitable for any required scene for temporarily building human-computer interaction interface;
3, the generating mode of characteristic pattern can be used but be not limited to Freehandhand-drawing, and any surface with characteristic pattern can be made
For operating space;
4, characteristic pattern can be customized according to user demand, using upper more humanized;
5, laser or projection device of the present invention independent of fixed position, hardware installation mode are more flexible;
6, algorithm operation quantity can also be effectively reduced using characteristic pattern auxiliary gesture identification, reduced to the hard of computing platform
Part requirement.
Specific embodiment
Technical solution in the embodiment of the present invention that following will be combined with the drawings in the embodiments of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based on this
Embodiment in invention, every other reality obtained by those of ordinary skill in the art without making creative efforts
Example is applied, shall fall within the protection scope of the present invention.
As shown in Figure 1, the present invention proposes a kind of plane gestural control system based on characteristic pattern identification, the system packet
Include master controller, operation platform module, data acquisition module, data processing module, data communication module and controlled plant module;
The operation of other each modules of the main controller controls simultaneously monitors each module operating status in real time;Operation platform module
For the manipulation region of user's Freehandhand-drawing characteristic pattern or in advance with the body surface region of characteristic pattern, user can be in operating platform
Gesture motion and system interaction are made in marked region in module;Data acquisition module captures operating space by camera in real time
Interior gesture motion, and image data is passed to data processing module;Data processing module is extracted from incoming image data
Hand-characteristic identifies gesture command, and incoming data communication module;Data communication module receives gesture command, and sets to controlled
Standby module sends corresponding function instruction;Controlled plant module receive capabilities instruct and execute corresponding function, while returning to controlled set
Standby operating status.The controlled plant module includes one or more controlled plants.
In conjunction with Fig. 2, the present invention also proposes a kind of control method of plane gestural control system based on characteristic pattern identification,
Step 1, initial networked control systems;
Step 2, detection Freehandhand-drawing index point;
Step 3, according to index point and other relational graphs are drawn, mark positioning operation area, and identify selected function;
Step 4 detects user gesture in functional areas, when detecting gesture, executes step 5, no to then follow the steps 6;
Step 5 sends control instruction according to user gesture;
If step 6, operating space figure change, step 8 is executed, it is no to then follow the steps 7;
Step 7, iteration execute step 2 to step 6, until executing step 10 at the end of control;
If step 8, operating space figure are incomplete or disappear, step 10 is executed, it is no to then follow the steps 9;
Step 9, iteration execute step 2 to step 8, until control terminates;
Step 10, finishing control.
The step 1 specifically:
Step 1.1 reads into system or inputs following parameters:
Function definition;
Indicia patterns;
Operating space shape and size;
Marker color;
Reference skin tones;
Manpower template;
Face template;
White balance;
Exposure;
Contrast;
Camera internal reference;
Other relevant parameters;
Step 1.2, according to parameter configuring system.
In conjunction with Fig. 3, the step 2 specifically:
Step 2.1, according to the image-forming conditions automatic adjusument imaging parameters such as illumination;
Step 2.2 captures image using new imaging parameters;
Step 2.3 carries out color segmentation to image using the marker color of typing in initialization procedure;
Step 2.4 carries out the detection of HOG feature to the result of color segmentation, obtains each index point position and size.
In conjunction with Fig. 4, the step 3 specifically:
Step 3.1, according to index point testing result, step 3.2 is executed when only one index point, when there is multiple marks
Step 3.5 is executed when point, when not having index point, current operation zone state is labeled as " invalid operation area ", executes step
3.9;
If there is additional mark around step 3.2, index point, step 3.3 is executed, otherwise by index point and its proximity
Field mark is " comprising unique identification point without labeling operation area ", and executes step 3.9;
Step 3.3 identifies marked content by OCR method;
If step 3.4, marked content meet default template requirement, by index point and mark adjacent domain labeled as " packet
The operating space of point containing unique identification and mark " is otherwise labeled as " invalid operation area ", executes step 3.9;
If step 3.5, index point arrangement mode meet default template requirement, step 3.6 is executed, otherwise by each index point
Surrounding zone marker is " invalid operation area ", executes step 3.9;
If there is additional mark around step 3.6, index point, step 3.7 is executed, each index point is otherwise surrounded into region
Labeled as " comprising multiple index points without labeling operation area ", step 3.9 is executed;
Step 3.7 identifies marked content by OCR method;
If step 3.8, marked content meet default template requirement, each index point and mark are surrounded into zone marker and are
" operating space comprising multiple index points and mark " otherwise will be labeled as " invalid operation area ", and execute step 3.9;
Step 3.9 analyzes operating space function according to operating space type, mark vertex type and marked content.
It is described that user gesture is detected in functional areas specifically:
Step 4.1, capture image and read operation area parameter;
Step 4.2, space and the temporal aspect for extracting finger form, position and track;
Step 4.3, the feature obtained to extraction are analyzed, and gesture command is obtained.
The present invention uses camera to capture and identifies predefined characteristic pattern to realize the control to terminal device.It uses
Person can draw given pattern in any selected location, such as draw control area using paper and be pasted on selected location, can also be with
Interim control area, ink self-discoloration after a period of time, nothing are drawn in positions such as metope, desktops using self-discoloration pen
It needs to clear up.The generating mode of characteristic pattern can be used but be not limited to Freehandhand-drawing, and any surface with characteristic pattern can be used as
Operating space corresponds to the organic component that hardware device is also used as system.
Signal sending end and receiving end are kept completely separate by control system of the present invention, capture side images using camera
Information, user can manipulate zone map in arbitrary plane Freehandhand-drawing, and then equipment control is carried out in the region drawn.The present invention
The method and system proposed can be solved effectively in current plane gestural control system, and what user's operation region was generally limited asks
Topic, provides more free, flexible, accurate and efficient interactive experience for user.Meanwhile the present invention is independent of fixed position
Laser or projection device can also effectively reduce algorithm operation quantity using characteristic pattern auxiliary gesture identification, reduce to calculating
The hardware requirement of platform.
Intelligent home control system based on Freehandhand-drawing control panel
Below by taking the application of smart home field as an example, illustrate embodiment of the present invention:
Hardware device
For system using monocular flake full-view camera as capture device, fading ink pen is controlled to set as freehand tool
Standby includes TV, automatically controlled floor lamp, Electric controlled window curtain and air-conditioning.Indoor arrangement is as shown in Figure 5.
The control of single operation area
Comprising unique identification point without labeling operation area
Below by taking floor lamp controls as an example, illustrate the control method without labeling operation area comprising unique identification point.
Triangle is defined as to the control index point of floor lamp in systems first, when user draws triangle in selected location
After shape pattern, drawing is identified as landing lamp switch by system, as shown in Fig. 6 (a).When user clicks triangle with finger
I.e. controllable floor lamp on or off, as shown in Fig. 6 (b).
In addition, also can define other figures as index point, mode of operation is essentially identical.In addition to clicking, when control has
When having the floor lamp of brightness control function, finger also can be used to slide control lamplight brightness on index point.
Operating space comprising unique identification point and mark
Below by taking TV controls as an example, illustrate the control method of the operating space comprising unique identification point and mark.
Defining alphabetical " TV " is that TV controls index point, and user draws " TV " mark, and mark such as Fig. 7 are made around it
(a) shown in, system will identify that " P- " is " switching to a upper channel " in figure, and " P+ " is " switching to succeeding channel ", and " V- " is " sound
Amount reduces ", " V+ " is " volume increase ".The executable corresponding function when user is clicked with finger to be marked, such as click " V+ "
It can be increased television sound volume, as shown in Fig. 7 (b).
When marked content is unintelligible, as shown in Fig. 7 (c), system can not identify marked content, which is considered as invalid operation
Area.
In addition to above-mentioned control mode, index point and mark be can be customized by users.
Comprising multiple index points without labeling operation area
Below by taking TV controls as an example, illustrate the control method without labeling operation area comprising multiple index points.
Defining alphabetical " TV " is that TV controls index point, and operating area is rectangle, and user draws two along oblique line directions
" TV " mark will be then recognized by the system with the rectangle that drawn mark is diagonal line vertex as operating space, as shown in Fig. 8 (a), wherein
" TV " mark be user's Freehandhand-drawing, dotted line be system identification region.It will be executed when user draws assignment graph in operating space
Corresponding function is " switching to succeeding channel " as drawn broken line to the right in Fig. 8 (b), and drawing upward broken line in Fig. 8 (c) is
" volume increase ".
Since defining operation region is rectangle, arrangement mode is that the index point of rectangle can be identified successfully, such as Fig. 8
(d) shown in.But arrangement mode is not that the index point of rectangle is then considered as invalid operation area, such as Fig. 8 (e) and (f).
If changing system definition, such as defining operation region is triangle, then Fig. 8 (e) will successfully be identified as operating space,
Other arrangement modes will be considered as invalid.In addition, index point and drawn graphics command can be customized by users.
Operating space comprising multiple index points and mark
The control mode of operating space comprising multiple index points and mark be comprising multiple index points without labeling operation
Supplement on the basis of area.Operating space function can be made to be more clear intuitively by additional mark, and can be customized by users.Figure
In operating space shown in 9, compared with Fig. 8 (a), due to being labelled with " P+ ", " P- ", " V+ " and " V- ", user can direct point
It hits mark and executes function, no longer need to draw broken line gesture, so as to which broken line gesture is mapped to other function, such as play on or off
Close TV.
The control of multioperation area
User can control same equipment or distinct device by drawing multiple operating spaces.
Single control multioperation area
User can draw same operation area in different location, control same equipment, can also draw according to actual needs
It makes multiple and different operating spaces and forms a complete home control panel, as shown in Figure 10, wherein triangle represents floor lamp, point
It hits " ON " control to turn on light, " OFF " is to turn off the light;" TV " represents TV, related marking Function above content by the agency of mistake;Rectangular generation
Table Electric controlled window curtain clicks " OPEN " and opens curtain, and " CLOSE " closes curtain;Asterisk represents air-conditioning, clicks " T+ " elevated temperature,
" T- " reduces temperature.
All index points and mark can be customized by users, as defined in the range of, index point and mark put in order
Its function is not influenced.
More people control multioperation area
When there is multiple users, each user can draw operating space respectively and control same or distinct device, as shown in figure 11,
Wherein (a) is the operating space that user A is drawn, and includes two control functions of floor lamp and TV;(b) operating space is drawn by user B,
Including two control functions of TV and Electric controlled window curtain.At this point, user A and B can control TV.
When controlling same equipment simultaneously using multioperation area, to prevent instruction conflict, system is by the finger first to recognize
Subject to order.After identifying and executing from the instruction of a certain operating space, in subsequent 30s, controlled plant is only in response to coming from
The instruction of the operating space.
Operating space supplement and modification
Operating space supplement
User can augment on the basis of existing operating space.Figure 12 (a) is the TV operation area drawn, Yong Huke
Increase index point or mark on the basis of existing with extended operation area, as shown in Figure 12 (b), system will identify that right area is frequency
Road control area will jump to corresponding channel when user writes out channel number with finger in the area, such as Figure 12 (c) institute
Show.
The supplement index point and mark that system is identified can be customized by the user.In use, supplementary content must accord with
Definition is closed, otherwise will be regarded as invalid operation area.
Operating space modification
User can modify on the basis of existing operating space.For example, in Figure 12 (b), when user passes through supplement mark
Out behind channel control area, repeated with left side " P+ " and " P- " two channel switching functions, at this time user can cross out " P+ " and
" P- " is drawn " V+ " and " V- ", and as shown in figure 13, then system will remove original two channel switching functions, increases volume tune
Save function.
Above to it is provided by the present invention it is a kind of based on characteristic pattern identification plane gestural control system and control method,
It is described in detail, used herein a specific example illustrates the principle and implementation of the invention, the above reality
The explanation for applying example is merely used to help understand method and its core concept of the invention;Meanwhile for the general technology of this field
Personnel, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion this theory
Bright book content should not be construed as limiting the invention.