CN102509079A - Real-time gesture tracking method and tracking system - Google Patents
Real-time gesture tracking method and tracking system Download PDFInfo
- Publication number
- CN102509079A CN102509079A CN2011103476957A CN201110347695A CN102509079A CN 102509079 A CN102509079 A CN 102509079A CN 2011103476957 A CN2011103476957 A CN 2011103476957A CN 201110347695 A CN201110347695 A CN 201110347695A CN 102509079 A CN102509079 A CN 102509079A
- Authority
- CN
- China
- Prior art keywords
- real
- image
- target
- information
- output terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a real-time gesture tracking method and a real-time gesture tracking system. An image sensor acquires image information, a corresponding processing unit finishes inter-frame image information extraction, an unchanged background and a moving object are counted in real time, the influence of the background on tracking is reduced, the background is re-separated by utilizing skin color information and a face detection technology, and a target is accurately tracked by utilizing a quick Kanade-Lucas-Tomasi (KLT) tracking algorithm, thereby implementing the hybrid real-time gesture tracking system based on skin color and movement information.
Description
Technical field
The present invention relates to the video motion view data and handle and field of machine vision, the particularly a kind of skin color of hand and movable information of gesture of utilizing mixes the method for realization gesture tracking and the system of this method of realization.
Background technology
Real-time gesture track algorithm is the hot issue of Flame Image Process and field of machine vision research always; Also be the difficult point problem simultaneously; Although proposed a series of track algorithm; And obtained effect preferably, Chinese patent publication number CN102053702A just discloses a kind of dynamic gesture control system and method.But all there is certain defective in these trackings; Only be used for colour of skin information like the camshift method, caused the accurate tracking existing problems of gesture, thereby influenced the development of human-computer interaction technology; Trace it to its cause is that people's gesture has diversity and complicacy; Each one residing environmental difference is a lot of in addition, therefore, how to realize in complicated background that the accurate tracking of hand target is most important.
About the gesture track algorithm, mainly contain optical flow method, based on the tracking of colour of skin information and camshifft method etc.Light stream is the instantaneous velocity of the pixel motion of space motion object on the observation imaging surface; The research of light stream is to utilize the time domain of the pixel intensity data in the image sequence to change and correlativity is confirmed " motion " of location of pixels separately, promptly studies the relation of object structures in gradation of image variation and the scene in time and motion thereof.Gray scale instantaneous rate of change on the two dimensional image plane preferred coordinate point is defined as light stream vector.Optical flow field (optical flow field) is meant the apparent motion of gradation of image pattern.It is a two-dimensional vector field, and the information that it comprises promptly is the transient motion velocity information of each picture point, and the purpose of research optical flow field is exactly the sports ground that can not directly obtain for approximate treatment from sequence image.When people's eyes are observed moving object; The scene of object forms a series of continually varying images on the retina of human eye; These a series of continually varying information are " flowing through " retina (being the plane of delineation) constantly, like a kind of " stream " of light, so be referred to as light stream (optical flow).The variation of image has been expressed in light stream, because it has comprised the information of target travel, therefore can the person of being observed be used for confirming the motion conditions of target.
At present the gesture track algorithm mainly is to be divided into three types, and one type of gesture track algorithm that is based on colour of skin information has three defectives based on the gesture detecting method of features of skin colors, the one, and illumination is very big to colour of skin influence, causes gestures detection in some cases, complete failure; The 2nd, in real process, people's face etc. also can get in the detection system as target; The 3rd, when having the background relevant with the colour of skin in the background, background also can be considered to target; Another kind of also is that the modified of colour of skin information is the camshift method, owing to utilized the meanshift algorithm, its tracking performance will have raising than the first kind, but when the colour of skin was disturbed, tracking accuracy had obviously decline; Optical flow method based on optical flow field has good effect to target following, and than robust, the variation of background is less to its influence simultaneously to illumination variation; Therefore become the first-selection of target following, in practical application, owing to reasons such as blocking property, multiple light courcess, the transparency and noises; Make the gray scale conservation assumed condition of optical flow field fundamental equation not satisfy; Can not solve correct optical flow field, most optical flow computation method is quite complicated simultaneously, and calculated amount is huge; Can not satisfy real-time requirement, therefore generally precision and real-time not required to be adopted than higher supervisory system.
Summary of the invention
The present invention obtains image information through imageing sensor; Utilize corresponding processing unit to accomplish the inter frame image information extraction; The background that real-time statistics is constant and the object of motion reduce the influence of background to following the tracks of, and utilize colour of skin information to realize separating again background with human face detection tech; Utilize KLT track algorithm fast to accomplish the accurate tracking of target then, thereby realize mixing the real-time gesture tracker based on the colour of skin and movable information.
The present invention for realizing the technical scheme that its purpose adopts is: a kind of real-time gesture tracking may further comprise the steps
Steps A, utilize image sensor cell to obtain the chromatic image of target;
Step B, be target, image is handled to improve picture quality;
Distinguish target and background in step C, the every two field picture from image;
Step D, in target, reject people's face, remainder is gesture in the target;
Step e, utilize optical flow method to accomplish gesture to follow the tracks of, and the confidence level of system is assessed.
Further, in the above-mentioned real-time gesture tracking: in the described steps A, imageing sensor is camera or other web camera.
Further, in the above-mentioned real-time gesture tracking: in the described steps A, described target is meant portrait.
Further, in the above-mentioned real-time gesture tracking: among the described step B: image processing comprised vision signal removed make an uproar and amplify.
Further, in the above-mentioned real-time gesture tracking: among the described step C: utilize the colour of skin information of priori to accomplish separating of background and portrait target.
Further, in the above-mentioned real-time gesture tracking: among the described step C: comprise step by step following:
C01, each two field picture in the image after handling among the step B is transformed into the HSV space;
C02, in the HSV space, utilize priori colour of skin information, confirm the area of skin color in each two field picture, and non-area of skin color rejected.
Further, in the above-mentioned real-time gesture tracking: in the described step e: and to the reliability of system assess consider following some:
The condition of features of skin colors information;
Non-face;
Comprise movable information;
The zone of former frame tracing area and present frame will have similarity.
The system that the present invention also provides a kind of real-time gesture to follow the tracks of comprises: show image and graphical interfaces image display cell, obtain image sensing unit, the graphics processing unit of visual information; Movable information statistic unit, people's face detecting unit, colour of skin extraction unit, KLT tracking cell and comprehensive integration unit;
The output of described image sensing unit connects described graphics processing unit;
The input end of described movable information statistic unit links to each other with the output terminal of described graphics processing unit: utilize inter-frame information to accomplish the statistics of movable information, remove motionless background;
The input end of described people's face detecting unit links to each other with the output terminal of described graphics processing unit, utilizes human face detection tech, realizes the extraction of people's face;
The input end of described colour of skin extraction unit links to each other with the output terminal of described graphics processing unit, utilizes features of skin colors information, removes background;
The input end of described KLT tracking cell links to each other with the output terminal of the output terminal of described graphics processing unit, described movable information statistic unit and the output terminal of described people's face detecting unit respectively, realizes the tracking of target;
The input end of described comprehensive integration unit links to each other with the output terminal with described KLT tracking cell of the output terminal of the output terminal of the output terminal of described graphics processing unit, described movable information statistic unit, described people's face detecting unit respectively, and its output terminal links to each other with image and graphical interfaces image display cell; Utilize features of skin colors information, people's face to detect and movable information, tracking target is carried out reliability assessment.
The present invention is utilized in image sensing and the processing unit on the display device; Through comprehensive integration to movable information statistics, colour of skin extractive technique, KLT tracking technique; Accomplish the real-time follow-up of gesture, thereby realize mixing the design of real-time gesture tracker based on colour of skin information and movable information.
Description of drawings
Fig. 1 is a system chart of the present invention.
Fig. 2 is a process flow diagram of the present invention.
Fig. 3 is that the colour of skin proposes flow process.
Fig. 4 is a KLT tracking target process flow diagram.
Fig. 5 is an embodiment of the invention process flow diagram.
Embodiment
Embodiment 1; Present embodiment obtains image information through imageing sensor, utilizes corresponding processing unit to accomplish inter frame image information extraction, the background that real-time statistics is constant and the object of motion; Reduce the influence of background to following the tracks of; The colour of skin information of utilizing priori and human face detection tech realize the separating again of background, utilizes fast the KLT track algorithm to accomplish the accurate tracking of target then, thereby realize mixing the real-time gesture tracker based on the colour of skin and movable information.
Present embodiment is a kind of real-time gesture tracker, and its block diagram is as shown in Figure 1, main composition comprise:
Image display cell: be responsible for show image and graphical interfaces, generally can be described as a display, such as the display of televisor or computing machine.
Normal image sensing unit: be responsible for obtaining visual information,, can obtain the image of target such as video camera or web camera etc.
Graphics processing unit: be responsible for each two field picture in the top image is handled; Mainly be exactly filtering and amplification; Filtering can filtering noise, amplifies useful signal is increased the image of being made up of a frame two field picture of image sensing unit picked-up; Each army tent image denoising and target are strengthened, for next step the effective extraction and the characteristic information extraction of target provides safeguard.
Movable information statistic unit: utilize inter-frame information to accomplish the statistics of movable information; Remove motionless background; Because the image of object between each frame that does not move is constant, therefore there is information in two frames, one difference with regard to the object that has only motion, and other can be near 0; The size of statistics moving image prevents the influence of noise to it.
People's face detecting unit: utilize human face detection tech, realize the extraction of people's face, reduce of the influence of people's face to tracker.Present embodiment utilizes the human face detection tech of Opencv, accomplishes the detection of people's face, confirms the position of people's face in a frame, so that remove the influence of people's face.
Colour of skin extraction unit: utilize features of skin colors information; Be responsible for the removal of part background; Here can only remove and the not similar background information of the colour of skin, the background similar with the colour of skin such as people's face can not be removed, and remove after needing people's face detection to come out again; What just can stay at last is gesture, and also the dermatoid background of some type can not be removed.
The KLT tracking cell: realize the tracking of target, process flow diagram is as shown in Figure 4.
Comprehensive integration unit: utilize features of skin colors information, people's face to detect and movable information, tracking target is carried out reliability assessment, thereby guarantee the reliability of tracking.Here assessment mainly is that it must satisfy four conditions: the one, and the condition of features of skin colors information, the 2nd, non-face, the 3rd, comprise movable information, the 4th, the zone of former frame tracing area and present frame will have similarity.
The key step of present embodiment is as shown in Figure 2, comprising:
1, image sensor cell obtains visual information;
2, graphics processing unit is a target to improve picture quality, accomplishes the noise reduction and the enhancing of image, and the purpose that strengthens here is outstanding target; Mainly carry out filtering and amplification.
3, the movable information statistic unit is a purpose to extract target, thereby distinguishes target and background; Here mainly be that the target of motion is added up.
4, people's face detecting unit, the extraction of responsible official's face reduces the influence of people's face to following the tracks of; Present embodiment utilizes the human face detection tech of Opencv, accomplishes the detection of people's face, confirms the position of people's face in a frame, so that remove the influence of people's face.
5, colour of skin extraction unit utilizes the colour of skin information of priori to accomplish the separation of target context, and process flow diagram is as shown in Figure 3, and coloured image is transformed in the HSV space, utilizes the colour of skin information of priori to confirm the zone of the colour of skin, the zone of rejecting the non-colour of skin.
6, the KLT tracking cell utilizes optical flow method, accomplishes the tracking of gesture target; Here introduced by < < a kind of feature point tracking method research that combines based on SIFT and KLT>> like < < aerospace journal>> 2011 the 7th interim articles for optical flow method.
7, comprehensive integration unit utilizes people's face to detect information, colour of skin information and movable information, accomplishes the tracker reliability assessment.
8, user interface elements is used for display image treatment effect and operation-interface.
A preferred embodiment step of the present invention is as shown in Figure 5:
1, display device unit explicit user graphical interfaces;
2, image sensing cell obtains the user and belongs to ambient image information;
3, pre-service such as the noise reduction of graphics processing unit realization image and enhancing;
4, the movable information statistic unit is realized moving and is separated with quiet target;
5, people's face detecting unit mainly is to utilize the AdaBoost sorter to accomplish people's face to detect, and reduces the influence of people's face to gesture.Because people's face is also known by meeting and thought staff, to detect in order to reduce its influence, after having detected, be that the part of people's face is removed;
7, colour of skin extraction unit mainly is to utilize the colour of skin information completion background of priori and separating of target, reduces the influence of its moving object to following the tracks of;
8, the KLT tracking cell mainly is through utilizing characteristic information to realize the accurate tracking of target;
9, comprehensive integration unit detects information, colour of skin information and movable information through relying on people's face, accomplishes the tracker reliability assessment, thereby guarantees the reliability of tracker.
Claims (8)
1. a real-time gesture tracking is characterized in that: may further comprise the steps
Steps A, utilize image sensor cell to obtain the chromatic image of target;
Step B, be target, image is handled to improve picture quality;
Distinguish target and background in step C, the every two field picture from image;
Step D, in target, reject people's face, remainder is gesture in the target;
Step e, utilize optical flow method to accomplish gesture to follow the tracks of, and the reliability of system is assessed.
2. real-time gesture tracking according to claim 1 is characterized in that: in the described steps A, imageing sensor is camera or other web camera.
3. real-time gesture tracking according to claim 1 is characterized in that: in the described steps A, described target is meant portrait.
4. real-time gesture tracking according to claim 1 is characterized in that: among the described step B: image processing is comprised vision signal removed make an uproar and amplify.
5. real-time gesture tracking according to claim 1 is characterized in that: among the described step C: utilize the colour of skin information completion background of priori and separating of portrait target.
6. real-time gesture tracking according to claim 1 is characterized in that: among the described step C: comprise step by step following:
C01, each two field picture in the image after handling among the step B is transformed into the HSV space;
C02, in the HSV space, utilize priori colour of skin information, confirm the area of skin color in each two field picture, and non-area of skin color rejected.
7. real-time gesture tracking according to claim 1 is characterized in that: in the described step e: and to the reliability of system assess consider following some:
The condition of features of skin colors information;
Non-face;
Comprise movable information;
The zone of former frame tracing area and present frame will have similarity.
8. the system that follows the tracks of of a real-time gesture comprises: show image and graphical interfaces image display cell, obtain image sensing unit, the graphics processing unit of visual information; It is characterized in that also comprising: movable information statistic unit, people's face detecting unit, colour of skin extraction unit, KLT tracking cell and comprehensive integration unit;
The output of described image sensing unit connects described graphics processing unit;
The input end of described movable information statistic unit links to each other with the output terminal of described graphics processing unit: utilize inter-frame information to accomplish the statistics of movable information, remove motionless background;
The input end of described people's face detecting unit links to each other with the output terminal of described graphics processing unit, utilizes human face detection tech, realizes the extraction of people's face;
The input end of described colour of skin extraction unit links to each other with the output terminal of described graphics processing unit, utilizes features of skin colors information, removes background;
The input end of described KLT tracking cell links to each other with the output terminal of the output terminal of described graphics processing unit, described movable information statistic unit and the output terminal of described people's face detecting unit respectively, realizes the tracking of target;
The input end of described comprehensive integration unit links to each other with the output terminal with described KLT tracking cell of the output terminal of the output terminal of the output terminal of described graphics processing unit, described movable information statistic unit, described people's face detecting unit respectively, and its output terminal links to each other with image and graphical interfaces image display cell;
Utilize features of skin colors information, people's face to detect and movable information, tracking target is carried out reliability assessment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103476957A CN102509079A (en) | 2011-11-04 | 2011-11-04 | Real-time gesture tracking method and tracking system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103476957A CN102509079A (en) | 2011-11-04 | 2011-11-04 | Real-time gesture tracking method and tracking system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102509079A true CN102509079A (en) | 2012-06-20 |
Family
ID=46221160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011103476957A Pending CN102509079A (en) | 2011-11-04 | 2011-11-04 | Real-time gesture tracking method and tracking system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102509079A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679130A (en) * | 2012-09-24 | 2014-03-26 | 富士通株式会社 | Hand tracking method, hand tracking apparatus and gesture recognition system |
| CN103839036A (en) * | 2012-11-22 | 2014-06-04 | 富士通株式会社 | Hand tracking method and device |
| CN107846555A (en) * | 2017-11-06 | 2018-03-27 | 深圳慧源创新科技有限公司 | Automatic shooting method, device, user terminal and computer-readable storage medium based on gesture identification |
| CN108132713A (en) * | 2012-12-19 | 2018-06-08 | 原相科技股份有限公司 | Switching device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101332362A (en) * | 2008-08-05 | 2008-12-31 | 北京中星微电子有限公司 | Interactive delight system based on human posture recognition and implement method thereof |
| CN102053702A (en) * | 2010-10-26 | 2011-05-11 | 南京航空航天大学 | Dynamic gesture control system and method |
| CN102081918A (en) * | 2010-09-28 | 2011-06-01 | 北京大学深圳研究生院 | Video image display control method and video image display device |
-
2011
- 2011-11-04 CN CN2011103476957A patent/CN102509079A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101332362A (en) * | 2008-08-05 | 2008-12-31 | 北京中星微电子有限公司 | Interactive delight system based on human posture recognition and implement method thereof |
| CN102081918A (en) * | 2010-09-28 | 2011-06-01 | 北京大学深圳研究生院 | Video image display control method and video image display device |
| CN102053702A (en) * | 2010-10-26 | 2011-05-11 | 南京航空航天大学 | Dynamic gesture control system and method |
Non-Patent Citations (3)
| Title |
|---|
| 刘玉: "一种基于SIFT和KLT相结合的特征点跟踪方法研究", 《宇航学报》 * |
| 曹昕燕: "基于肤色和运动检测技术的单目视觉手势分割", 《湖南大学学报（自然科学版）》 * |
| 郭怡文: "基于肤色检测和背景差分的人手定位和跟踪", 《测控技术》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679130A (en) * | 2012-09-24 | 2014-03-26 | 富士通株式会社 | Hand tracking method, hand tracking apparatus and gesture recognition system |
| CN103679130B (en) * | 2012-09-24 | 2018-04-13 | 富士通株式会社 | Hand method for tracing, hand tracing equipment and gesture recognition system |
| CN103839036A (en) * | 2012-11-22 | 2014-06-04 | 富士通株式会社 | Hand tracking method and device |
| CN103839036B (en) * | 2012-11-22 | 2017-09-29 | 富士通株式会社 | Hand method for tracing and device |
| CN108132713A (en) * | 2012-12-19 | 2018-06-08 | 原相科技股份有限公司 | Switching device |
| CN108132713B (en) * | 2012-12-19 | 2021-06-18 | 原相科技股份有限公司 | Switching device |
| CN107846555A (en) * | 2017-11-06 | 2018-03-27 | 深圳慧源创新科技有限公司 | Automatic shooting method, device, user terminal and computer-readable storage medium based on gesture identification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jain et al. | Real-time upper-body human pose estimation using a depth camera | |
| CN109598242B (en) | Living body detection method | |
| US20160154469A1 (en) | Mid-air gesture input method and apparatus | |
| US9256324B2 (en) | Interactive operation method of electronic apparatus | |
| CN101793562A (en) | Face detection and tracking algorithm of infrared thermal image sequence | |
| CN104821010A (en) | Binocular-vision-based real-time extraction method and system for three-dimensional hand information | |
| CN103218605A (en) | Quick eye locating method based on integral projection and edge detection | |
| Zhao et al. | Early smoke detection of forest fire video using CS Adaboost algorithm | |
| CN102509079A (en) | Real-time gesture tracking method and tracking system | |
| JP2010057105A (en) | Three-dimensional object tracking method and system | |
| Raheja et al. | Hand gesture pointing location detection | |
| CN101719280B (en) | Method for detecting petty infrared target | |
| CN109658437A (en) | A kind of method and device of quick detection moving object | |
| CN104813341B (en) | Image processing system and image processing method | |
| Miller et al. | Person tracking in UAV video | |
| CN103473753A (en) | Target detection method based on multi-scale wavelet threshold denoising | |
| Shaker et al. | Real-time finger tracking for interaction | |
| Damle et al. | Human body skeleton detection and tracking | |
| JP2023047657A (en) | Noise cancelling device and program | |
| Hammer et al. | Motion segmentation and appearance change detection based 2D hand tracking | |
| KR100729265B1 (en) | A face detection method using difference image and color information | |
| Min et al. | Robust 3D human tracking based on Kinect | |
| Gobhinath et al. | Implementation of Real Time Static Hand Gestures Recognition for Sign Language | |
| CN103065324B (en) | A kind of moving target detecting method and device | |
| Kondo | An image sequence segmentation method using gradient orientation information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120620 |