CN104460991A - Gesture interaction control system based on digital household equipment - Google Patents
Gesture interaction control system based on digital household equipment Download PDFInfo
- Publication number
- CN104460991A CN104460991A CN201410664444.5A CN201410664444A CN104460991A CN 104460991 A CN104460991 A CN 104460991A CN 201410664444 A CN201410664444 A CN 201410664444A CN 104460991 A CN104460991 A CN 104460991A
- Authority
- CN
- China
- Prior art keywords
- module
- gesture
- identification
- tracking
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Abstract
The invention discloses a gesture interaction control system based on digital household equipment. The system comprises a coordinate calibration module, a gesture identification module, a gesture tracking module, a network transmission module and a system control module, wherein the coordinate calibration module is used for capturing finger coordinates in correspondence with an output position on a screen, the gesture identification module is used for processing video frames after the video frames are obtained by a camera, identifying hands and gestures and feeding the video frames back to the gesture tracking module, the gesture tracking module is used for conducting movement tracking on the gestures, the network transmission module is used for sending gesture tracking content to the system control module, and the system control module is used for controlling interaction between the functional modules. The system is low in cost, high in identification rate and good in control effect. A digital household environmental system has a few restrictions on people, and operability is high. Meanwhile, compared with single hand gesture identification, left and right hand identification expands the application range.
Description
Technical field
The present invention relates to digital home technical field, be specifically related to a kind of gesture interaction control system based on digital family equipment.
Background technology
Digital family system can define like this: utilize based on computer technology and network technology, by home environment, housed device is undertaken communicating and exchanges data by different mutual contact modes with domestic consumer, realize home environment, housed device and domestic consumer/interconnect, the convenient life making the people in family greatly can must enjoy interconnected household electrical appliances and Smart Home to bring, meet them in information, communication, the demand of interchange and amusement aspect, and then the system of the quality of raising life staying idle at home.The robotization of household electric appliances environment, intellectuality and networking are pursued by digital home, and the study hotspot of Contemporary Digital household is household internal networking, namely how to realize the interconnected of equipment room and completes seamless exchanges data [1].But from higher level, family's inner electronic equipment interconnected, is still in layers of physical devices, be far from reaching the intelligent requirement of " people-oriented ".Therefore, realize man-machine harmonious in digital household environment, natural interaction process, to the development of digital home, there is important impetus.Along with the development of Robotics, intellect service robot starts progressively to incorporate in the life of the mankind, and the development of human-computer interaction technology is the indispensable important component part of intellect service robot research.The main direction of studying of man-machine interaction mainly contains expression, sound and Gesture Recognition.Wherein gesture be a kind of from however man-machine communication's pattern intuitively, than ocular and clear more of expressing one's feelings in the expression of information, quantity of information is abundanter.In man-machine interaction, the identification of the gesture of view-based access control model realizes the indispensable gordian technique of man-machine interaction of new generation.The gesture identification of view-based access control model is by image information, allows robot obtain the gesture attitude information of people, classifies to different gesture informations.For addressing this problem, construct digital household environment herein, and placed robot as Man Machine Interface.Employing both hands control, and propose the gestures detection identifying schemes based on Gentle AdaBoost algorithm, by the sorter that the different gesture of self-built sample training is corresponding, and complete gesture interaction Control System Design, have higher accuracy and antijamming capability.
From digital home's industry, user interface at present based on touch-control is also extremely important emerging mutual market, application most at present is also all applied on the product of the small screen, and is mostly to design as non-productive operation, but not as exporting completely and the equipment of output set as iPad.Such as current many washing machines, some white families such as refrigerator all start to adopt touch control operation on human-computer interaction interface, and some manipulates based on the contact of true sense of touch, and some is then that the contact based on plane of similar apple manipulates.Use finger, or even any position of health is carried out with direct contact of plane.
Current voice-based user interface speech recognition technology is developing always, and current each large digital industry giant is in the related service based on voice recognition that research and development are respective " because the both hands of people can greatly free by voice operating from operation, thus it is very good interactive form under particular context " and simultaneously in television industries, because the geometry level of television content increases, user is also given in the very big expansion of this data the " channel how selecting fast oneself will watch in all more than 200 live channels of justing think that brings great difficulty in content search inquiry in fact, use traditional telepilot just as looking for a needle in a haystack, even then use its outstanding user interface of TV set-top box of apple also still cannot address this problem, because using its telepilot to carry out input operation is on TV the thing " phonetic entry is best selection in this situation " of extremely making us going mad
Fixed design and too heavy design make it lack movability; The increase of operations area too increases actuating quantity simultaneously, will increase sense tired out; Touch-control based on plane reduces the concavo-convex sense of touch of contact real world, without actual physical feedback; The clean of surface is a problem; Owing to being can not adjust sense of fatigue by increasing people based on the direction of desktop concept screen.
And speech recognition technology still can run into larger difficulty in the application, the problem of such as accent, with regard to China's Mainland, various dialect not lower hundred kinds, mandarin also has with a mixed accent, and many dialects are not the problem of voice but grammer and vocabulary change all completely, in this case the versatility based on voice operating will be very low, the another one problem words that to be exactly voice itself exist as a kind of interactive form it really can not with other voice simultaneously and deposit, " all these problems also voice-based interface alternation reason that can not popularize in a large number just that too noisy ambient sound or other voice all will affect its interactive quality.
Summary of the invention
The bimanual input control method that the object of the invention is view-based access control model can complete ordinary electrical home appliance controlling functions, system being low cost effectively in constructed digital household environment, and discrimination is high, and control effects is good.Less-restrictive, the operability of digital household environment system on human are stronger.Meanwhile, use left and right both hands identification herein, relative to singlehanded gesture identification, extend range of application.
The invention provides a kind of gesture interaction control system based on digital family equipment, described system comprises:
Coordinate scaling module, flutters for the outgoing position corresponded to above screen and catches finger coordinate;
Gesture recognition module, for after camera obtains frame of video, processes described frame of video, identifies and sells and gesture, and frame of video is fed back to gesture tracking module;
Gesture tracking module, for carrying out motion tracking to gesture;
Network transmission module, for sending to system control module by gesture tracking content;
System control module, mutual for what control between whole functional module.
Described coordinate scaling module is used for the known coordinate corresponding point correction program to input, and realizes parameter matrix.
The present invention has following beneficial effect, system being low cost, and discrimination is high, and control effects is good.Less-restrictive, the operability of digital household environment system on human are stronger.Meanwhile, use left and right both hands identification herein, relative to singlehanded gesture identification, extend range of application.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the gesture interaction Control system architecture schematic diagram based on digital family equipment in the embodiment of the present invention;
Fig. 2 is the process flow diagram that the coordinate in the embodiment of the present invention is demarcated;
Fig. 3 is the process flow diagram of the gesture identification in the embodiment of the present invention;
Fig. 4 is training in the embodiment of the present invention and matching process process flow diagram.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making other embodiments all obtained under creative work prerequisite, belong to the scope of protection of the invention.
Gesture interaction system based on computer vision is made up of five modules, comprises coordinate and demarcates, gesture identification, gesture tracking, and Internet Transmission and system control module composition, shown in the Structure and Process graph of a relation 1 of system.
Coordinate demarcating module is an independently module, corresponding to it is the finger coordinate that outgoing position above screen and camera catch, if camera and screen position relation do not become, so it does not just need to re-start computing, and if change that it will occur to change to adapt to module variations accordingly.Gesture recognition module is gesture tracking module service, after program obtains frame of video by camera, first by this frame of video of gesture recognition module process, identification is sold and gesture, and the positional information of hand is passed to gesture tracking module, motion tracking can be carried out to gesture at ensuing frame of video Program with regard to only needing gesture tracking module, and not need to re-start identification.Possibly there will be and follow the tracks of unsuccessfully, now can feed back to gesture recognition module, turn back to so again and go, by again identifying the position of gesture in said step above.Therefore these two modules have a process fed back mutually.
Coordinate demarcating module is an independently module, carrying out needing to demarcate the parameter of camera alternately, makes can obtain coordinate figure more accurately in identification below.It is an indispensable module in whole system that coordinate is demarcated.In computer program, coordinate demarcation can need the device of some outsides to reach the object of demarcation, such as chessboard table, and the flow process that coordinate is demarcated as shown in Figure 2.
In gesture recognition module, because this whole system identifies based on finger finger at present, thus this visual interactive employs complexion model, and adds Gauss model to improve recognition correct rate.In common complexion model, because the impact being subject to environmental factor is very large, discrimination is not very high, often has a lot of noise point in image after recognition.Gesture recognition module is the mutual data input basis of whole finger, if to go wrong the gesture tracking module that so will have influence on below in identification, the flow process of gesture identification as shown in Figure 3.
Fig. 4 shows training in the embodiment of the present invention and matching process process flow diagram, and people are think that image is made up of pixel to the understanding of image, but in pattern-recognition, the detection and Identification of machine to image have generally all been come by feature.Image all comprises oneself exclusive feature, by the detection to feature, just can complete the detection to image.Use feature to carry out modeling to images of gestures, can calculated amount be reduced, can elevator system training and detection speed.Four kinds of different AdaBoost algorithms have been carried out performance test and have been compared, finally drawn Gentle AdaBoost algorithm not only train and obtain classifier performance being optimum, and training speed is also the fastest.The present invention adopts Gentle AdaBoost algorithm as gestures detection matching algorithm.
The arthmetic statement of the training strong classifier of Gentle AdaBoost algorithm is as follows:
1. prepare a large amount of training sample (x
1, y
1), (x
2, y
2) ..., (x
n, y
n), wherein y
i=1 is expressed as positive sample (gesture, y
i=-1 is expressed as negative sample, n be just, sample comprehensive.
2. initialization sample weights omega
i=D (i), positive sample can be set to
negative sample can be set to
here p, q represent the quantity of positive and negative samples respectively, i.e. p+q=n.
3. couple t=1 ..., T:
1) normalization sample weights:
2) each feature, training place Weak Classifier h
t∈ [-1,1], and calculate the weighting (q of this feature in all samples
t,j) square error ε
t:
Wherein Weak Classifier h
tx () is defined as:
Wherein f
tx () is eigenwert.θ
jfor the threshold value by training the Weak Classifier obtained, α
1and α
2be the classification results of Weak Classifier, they are the arbitrary real numbers be positioned between [-1,1], only ideally right-on at classification results, | α
1|=1 and | α
2|=1, namely 1 represents positive sample, and-1 represents negative sample.
3) choose there is minimal error rate ε
tweak Classifier h
t
4) weights omega of each sample is upgraded
t+1, j=ω
t, jexp [-y
ih
t(x
i)].
4. last, strong classifier is then:
Wherein
represent the threshold value of strong classifier.
Generally, tracking module is after identification module, after the positional information obtaining interaction gesture, follows the tracks of to reduce the time of identification thus improve mutual efficiency to gesture.Thus gesture tracking module is one of most important module in whole system, if a gesture interaction system is unstable, tracking module fast, brings very large impact so will to whole interactive system, or even cause using.
The present invention has following beneficial effect, system being low cost, and discrimination is high, and control effects is good.Less-restrictive, the operability of digital household environment system on human are stronger.Meanwhile, use left and right both hands identification herein, relative to singlehanded gesture identification, extend range of application.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is that the hardware that can carry out instruction relevant by program has come, this program can be stored in a computer-readable recording medium, storage medium can comprise: ROM (read-only memory) (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or CD etc.
Above the gesture interaction control system based on digital family equipment that the embodiment of the present invention provides is described in detail, apply specific case herein to set forth principle of the present invention and embodiment, the explanation of above embodiment just understands method of the present invention and core concept thereof for helping; Meanwhile, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (2)
1. based on a gesture interaction control system for digital family equipment, it is characterized in that, described system comprises:
Coordinate scaling module, flutters for the outgoing position corresponded to above screen and catches finger coordinate;
Gesture recognition module, for after camera obtains frame of video, processes described frame of video, identifies and sells and gesture, and frame of video is fed back to gesture tracking module;
Gesture tracking module, for carrying out motion tracking to gesture;
Network transmission module, for sending to system control module by gesture tracking content;
System control module, mutual for what control between whole functional module.
2. as claimed in claim 1 based on the gesture interaction control system of digital family equipment, it is characterized in that, described coordinate scaling module is used for the known coordinate corresponding point correction program to input, and realizes parameter matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410664444.5A CN104460991A (en) | 2014-11-19 | 2014-11-19 | Gesture interaction control system based on digital household equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410664444.5A CN104460991A (en) | 2014-11-19 | 2014-11-19 | Gesture interaction control system based on digital household equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104460991A true CN104460991A (en) | 2015-03-25 |
Family
ID=52907183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410664444.5A Pending CN104460991A (en) | 2014-11-19 | 2014-11-19 | Gesture interaction control system based on digital household equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104460991A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105373037A (en) * | 2015-11-02 | 2016-03-02 | 渤海大学 | Kitchen ventilator control system based on gesture recognition |
| CN105536205A (en) * | 2015-12-08 | 2016-05-04 | 天津大学 | Upper limb training system based on monocular video human body action sensing |
| CN114167978A (en) * | 2021-11-11 | 2022-03-11 | 广州大学 | Human-computer interaction system carried on construction robot |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101256673A (en) * | 2008-03-18 | 2008-09-03 | 中国计量学院 | Method for tracing arm motion in real time video tracking system |
| CN101951474A (en) * | 2010-10-12 | 2011-01-19 | 冠捷显示科技（厦门）有限公司 | Television technology based on gesture control |
| CN102142055A (en) * | 2011-04-07 | 2011-08-03 | 上海大学 | True three-dimensional design method based on augmented reality interactive technology |
| CN102200830A (en) * | 2010-03-25 | 2011-09-28 | 夏普株式会社 | Non-contact control system and control method based on static gesture recognition |
| CN102339125A (en) * | 2010-07-23 | 2012-02-01 | 夏普株式会社 | Information equipment and control method and system thereof |
-
2014
- 2014-11-19 CN CN201410664444.5A patent/CN104460991A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101256673A (en) * | 2008-03-18 | 2008-09-03 | 中国计量学院 | Method for tracing arm motion in real time video tracking system |
| CN102200830A (en) * | 2010-03-25 | 2011-09-28 | 夏普株式会社 | Non-contact control system and control method based on static gesture recognition |
| CN102339125A (en) * | 2010-07-23 | 2012-02-01 | 夏普株式会社 | Information equipment and control method and system thereof |
| CN101951474A (en) * | 2010-10-12 | 2011-01-19 | 冠捷显示科技（厦门）有限公司 | Television technology based on gesture control |
| CN102142055A (en) * | 2011-04-07 | 2011-08-03 | 上海大学 | True three-dimensional design method based on augmented reality interactive technology |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105373037A (en) * | 2015-11-02 | 2016-03-02 | 渤海大学 | Kitchen ventilator control system based on gesture recognition |
| CN105536205A (en) * | 2015-12-08 | 2016-05-04 | 天津大学 | Upper limb training system based on monocular video human body action sensing |
| CN114167978A (en) * | 2021-11-11 | 2022-03-11 | 广州大学 | Human-computer interaction system carried on construction robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Matuszek et al. | Learning from unscripted deictic gesture and language for human-robot interactions | |
| WO2020062392A1 (en) | Signal processing device, signal processing method and related product | |
| WO2018177379A1 (en) | Gesture recognition, gesture control and neural network training methods and apparatuses, and electronic device | |
| Cai et al. | Audio-textual emotion recognition based on improved neural networks | |
| CN111385462A (en) | Signal processing device, signal processing method and related product | |
| Zampogiannis et al. | Learning the spatial semantics of manipulation actions through preposition grounding | |
| CN107832432A (en) | A kind of search result ordering method, device, server and storage medium | |
| Deng et al. | MVF-Net: A multi-view fusion network for event-based object classification | |
| CN104777911A (en) | Intelligent interaction method based on holographic technique | |
| CN108073851B (en) | Grabbing gesture recognition method and device and electronic equipment | |
| CN110796018A (en) | Hand motion recognition method based on depth image and color image | |
| CN110968235B (en) | Signal processing device and related product | |
| CN111665941A (en) | Multi-mode semantic fusion man-machine interaction system and method for virtual experiment | |
| Caramiaux et al. | Beyond recognition: using gesture variation for continuous interaction | |
| CN104460991A (en) | Gesture interaction control system based on digital household equipment | |
| CN102819751A (en) | Man-machine interaction method and device based on action recognition | |
| Zhu et al. | Learning generalizable manipulation policies with object-centric 3d representations | |
| Ding et al. | Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations | |
| CN109584864A (en) | Image processing apparatus and method | |
| Liu et al. | Real-time robotic mirrored behavior of facial expressions and head motions based on lightweight networks | |
| CN108228285A (en) | A kind of human-computer interaction instruction identification method multi-modal end to end | |
| He et al. | Developmental word acquisition and grammar learning by humanoid robots through a self-organizing incremental neural network | |
| Zhong et al. | Bridging the Gap between Robotic Applications and Computational Intelligence in Domestic Robotics | |
| CN115542759A (en) | Intelligent household control method, system and storage medium based on multi-mode recognition | |
| CN109785843A (en) | Image processing apparatus and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150325 |