CN103376890B - The gesture remote control system of view-based access control model - Google Patents
The gesture remote control system of view-based access control model Download PDFInfo
- Publication number
- CN103376890B CN103376890B CN201210121832.XA CN201210121832A CN103376890B CN 103376890 B CN103376890 B CN 103376890B CN 201210121832 A CN201210121832 A CN 201210121832A CN 103376890 B CN103376890 B CN 103376890B
- Authority
- CN
- China
- Prior art keywords
- hands
- scope
- image
- candidate
- gesture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses the gesture remote control system of a kind of view-based access control model, the gesture remote control system of this view-based access control model includes: image capture device, for capturing a series of images of object；Gesture identification equipment, for the gesture of a series of images identification object captured from image capture device and recognition result is sent to operational order triggers equipment；And operational order triggers equipment, for triggering scheduled operation order according to the recognition result come from the transmission of gesture identification equipment.Described gesture identification equipment includes: hands detection part, the hands of the image detection object for being captured from image capture device；Hands tracking unit, for when the hands of object being detected when hands detection part in an image, follows the tracks of the hands of object in ensuing image；Gesture identification parts, the hands of the object traced into for the hands of object detected according to hands detection part and hands tracking unit determine the hands of object motion and according to determined by the motion of hands of object identify the gesture of object.
Description
Technical field
The present invention relates to image procossing, pattern recognition and Object tracking field, and relate more specifically to
The gesture remote control system of view-based access control model.
Background technology
Along with life Computer and the numerous portable intelligent device current people becomes increasingly
Indispensable, people are by more natural and more efficient mutual between people and computer of hope.So
And, the man-machine interaction of traditional such as mouse/keyboard, remote controller, even touch screen etc
(HCI) ancillary equipment (such as, in bathroom or kitchen, is being driven under some specific conditions
In, etc.) it is inconvenient for a user, because local it is desirable that freely touch at these
HCI.Therefore, in recent years, the gesture remote control system as one of potential solution obtains more coming
The most concerns.
Substantially, gesture remote control system will be followed the tracks of hands and analyzed the expression of significant hands, if they
Be identified as the one in predefined gesture, then corresponding operational order will be triggered to perform
Predetermined operation.Owing in a lot of situations, gesture identification is the most complicated, so at gesture identification
In reason, many different instruments are employed to solve this problem, such as HMM
(Hidden Markov Models (HMM)), particle filter, finite state machine (FSM) and
Neutral net.Most of gesture recognition systems require high computation complexity；Additionally, some of them tool
Some is had to limit, for example, it is desired to extra equipment (if desired for wearing glove) or the instrument of precision
(collecting depth information if desired for infrared camera) or can only be in good illumination environment and simple background
Environment runs and (such as cannot distinguish between hands and the object of there is the colour of skin similar color, or can only identify quiet
State gesture, etc.).
Accordingly, it would be desirable to the computation complexity of a kind of real time remote control is low and can be good in complex environment
The gesture recognition system run.
Summary of the invention
According to an aspect of the present invention, the gesture remote control system of a kind of view-based access control model includes: image capturing
Equipment, described image capture device is for capturing a series of images of object；Gesture identification equipment, institute
State gesture identification equipment for a series of images identification object that captured from described image capture device
Recognition result is also sent to operational order triggering equipment by gesture；And operational order triggers equipment, institute
State operational order and trigger equipment for touching according to the recognition result come from the transmission of described gesture identification equipment
Sending out scheduled operation order, wherein, described gesture identification equipment includes: hands detection part, and described hands is examined
Survey the hands of the parts image detection object for being captured from described image capture device；Hands tracking portion
Part, described hands tracking unit is at hands object being detected when described hands detection part in an image
Time, ensuing image is followed the tracks of the hands of object；Gesture identification parts, described gesture identification parts
For the hands of object detected according to described hands detection part and described hands tracking unit trace into right
The hands of elephant determine the hands of object motion and according to determined by the motion of hands of object identify object
Gesture.
In one embodiment, described hands detection part will be by being captured by described image capture device
Image be transformed into gray level image, utilize cascade classifier based on local binary patterns from this gray scale
The hands of image detection object.
In one embodiment, described hands tracking unit follows the tracks of the hands of object by following process: sharp
It is used in previous image the scope of the hands detecting or tracing into and the broca scale picture of present image with previous
Difference image between the broca scale picture of image, carrys out the use in the broca scale picture of original definition present image
In the hunting zone following the tracks of hands；Perform template matching method to determine the hands as the present image traced into
Scope, wherein, described template matching method includes:
In first candidate's hands scopes multiple defined in hunting zone, those the first candidate hands scopes have with
The size that the size of To Template is identical, and, in second candidate's hands scope defined in difference image,
This candidate's hands scope has the size identical with the size of To Template, and wherein, described To Template is
The scope of the hands detected in previous image or trace into；
Following steps are performed until these multiple first candidates for the plurality of first candidate's hands scope circulation
Hands scope is all till following matching judgment processes, so that it is determined that go out the time mated most with To Template
Player's scope is as the scope of the hands traced in the broca scale picture of present image:
Calculate the meansigma methods of first candidate's hands scope and the absolute difference of each pixel of To Template
As the first error；
If this first error is more than the first predetermined threshold, then it represents that this candidate's hands scope not with
To Template mates, thus is excluded；
If this first error is less than the first predetermined threshold, then calculate the second error, the second error
It it is the value that obtained of the meansigma methods of the value of each pixel by the first error being deducted second candidate's hands scope
The value being multiplied by predetermined adjustment factor and obtain；
If this second error is less than the second predetermined threshold, it is determined that coupling, i.e. this first candidate hands
Scope is determined to be in the scope of the hands traced in the skin color range of present image, and the second error
Value as Second Threshold so that next first candidate's hands scope is carried out matching judgment.
In one embodiment, after the hunting zone determined in the broca scale picture of present image
And before performing template matching method, revise the search model of original definition with reference to described difference image
Enclose, and define the plurality of candidate's hands scope in hunting zone after the decrease and perform described template
Joining method to determine the scope of the hands of present image, wherein, described correction includes: searching original definition
Each limit of rope scope is gradually to inside contracting, and is more than the pixel of predetermined threshold when any side runs into pixel value
Time, this limit stops to inside contracting.
In one embodiment, in described template matching method, for each first candidate's hands scope
Determine that when mating with To Template, this coupling is verified, and described checking includes: calculate and be currently identified
The first candidate's hands scope mated most is each with the scope of hands detected in described hands detection equipment
The meansigma methods of the absolute difference of individual pixel is as the 3rd error；Judge that whether the 3rd error is more than the 3rd threshold
Value, if the 3rd error more than the 3rd predetermined threshold, then judges currently to be identified first mated most
Candidate's hands scope is not mated, thus gets rid of this first candidate hands scope.
In one embodiment, the broca scale picture used in the present invention can obtain as follows
To: the value of the R component in the RGB component of each pixel of the image captured is deducted G component
With the average of B component, to obtain a difference；And described difference is compared with predetermined threshold, if
Described difference is less than described predetermined threshold, then the value of the respective pixel of broca scale picture takes 0, and if
Described difference is bigger than described predetermined threshold, then the value of the respective pixel of broca scale picture takes described difference.
In one embodiment, described gesture identification parts according to described hands detection part and described hands with
Track parts are in the position of the hands of the object that every two field picture detected or traced into, from the hands of every two field picture
Hand position between adjacent two two field pictures that position calculation obtains is orientated and takes from adjacent two hand positions
Direction of displacement to calculated adjacent two two field pictures determines the motion of the hands of object and will be determined
The motion of hands of object right to identify with predefined gesture-hands movement trajectory map table carries out contrast
The gesture of elephant.
According to a further aspect in the invention, a kind of gesture remote control method of view-based access control model is right including capturing
The a series of images of elephant；Gesture from a series of images identification object captured；According to recognition result
Trigger scheduled operation order, wherein, identify that the gesture of object includes: from the image detection captured
The hands of object；When the hands of object being detected in an image, it is right to follow the tracks of in ensuing image
The hands of elephant；And the hands according to the object detected and the hands of object traced into determine the hands of object
Motion and according to determined by the motion of hands of object identify the gesture of object.
According to another aspect of the invention, a kind of side for RGB image being transformed into broca scale picture
Method, the method includes: the value of the R component of the RGB component of each pixel of RGB image deducted
G component and the average of B component, to obtain a difference；Described difference is compared with predetermined threshold, as
The most described difference is less than described predetermined threshold, then the value of the respective pixel of broca scale picture takes 0, and such as
The most described difference is bigger than described predetermined threshold, then the value of the respective pixel of broca scale picture takes described difference.
According to another aspect of the invention, a kind of for following the tracks of mesh calibration method, bag in image sequence
Include: utilize scope and the broca scale of present image of the target detecting in previous image or tracing into
Difference image between picture and the broca scale picture of previous image, carrys out the broca scale of original definition present image
The hunting zone for following the tracks of target in Xiang；Perform template matching method to determine as working as of tracing into
The scope of the target of front image, wherein, described template matching method includes:
At the first candidate target scopes multiple defined in described hunting zone, those the first candidate target models
Enclose and there is the size identical with the size of To Template, and, defined in described difference image second
Candidate target scope, this candidate target scope has the size identical with the size of To Template, its
In, described To Template is the scope of the target detecting in previous image or tracing into；
Following steps are performed until these multiple first times for the plurality of first candidate target scope circulation
Select target zone all till following matching judgment processes, mate most with To Template so that it is determined that go out
Candidate target scope as the scope of the target traced in the broca scale picture of present image:
Calculate the average of a first candidate target scope and the absolute difference of each pixel of To Template
Value is as the first error；
If this first error is more than the first predetermined threshold, then it represents that this candidate target scope is not
Mate with To Template, thus be excluded；
If this first error is less than the first predetermined threshold, then calculate the second error, the second error
It is that the meansigma methods of the value of each pixel by the first error deducts the second candidate target scope is obtained
The value that value is multiplied by predetermined adjustment factor and obtains；
If this second error is less than the second predetermined threshold, it is determined that coupling, i.e. this first candidate mesh
Mark scope is determined to be in the scope of the target traced in the skin color range of present image, and second
The value of error as Second Threshold so that next first candidate target scope is carried out matching judgment.
The gesture recognition system of the view-based access control model according to the present invention, has and skin even if being moved across at hands
In the case of some object of the color that color is similar, it is also possible to simply and efficiently from capture continuously
Frame image sequence detects hands exactly, follows the tracks of hands movement and identify gesture.
Accompanying drawing explanation
Fig. 1 shows the schematic structure of the gestural control system of the present invention；
Fig. 2 shows the flow chart of the gesture recognition process performed by gesture identification equipment of the present invention；
Fig. 3 shows the example of the gray level image after captured RGB image and conversion；
Fig. 4 shows the example when successfully be detected hands in gray level image after the conversion, wherein hands
Scope illustrated with rectangle frame；
Fig. 5 shows that the exemplary of broca scale picture utilized in the gesture recognition process of the present invention is shown
Figure；
Fig. 6 shows the difference diagram between the broca scale picture of continuous two images moved right for hands
Picture；
(a), (b) and (c) in Fig. 7 show defined in the gesture recognition process of the present invention for work as
The broca scale picture of front image is followed the tracks of the exemplary diagram of the process of the hunting zone of hands；
Fig. 8 show the hunting zone of original definition be corrected after the exemplary diagram of result；
Fig. 9 shows the template matching algorithm improved in the gesture recognition process of the present invention by utilization
Follow the tracks of the exemplary diagram of the process of the hands crossing over face；And
Figure 10 shows in the gesture identification equipment of the present invention for calculating hand position orientation, hands position
Put orientation to change and the exemplary diagram in the defined hands movement direction of hand position orientation.
Detailed description of the invention
Below, by preferred embodiments of the present invention will be described in detail with reference to the annexed drawings.Note, in this specification
With in accompanying drawing, it is denoted by the same reference numerals and there is the structural elements of substantially the same function and structure
Part, and omit the repeated description to these structural details.
The invention provides the gesture remote control system of a kind of efficient and healthy and strong view-based access control model, this system
Utilize common web camera work and in complex environment, show reliability, having only to relatively simultaneously
Few amount of calculation.
Fig. 1 shows the diagram of the schematic construction of the gestural control system of the present invention.Such as Fig. 1 institute
Showing, the gesture remote control system of the view-based access control model of the present invention mainly includes three parts: image capture device
101, for capturing a series of images of object；Gesture identification equipment 102, for setting from image capturing
Recognition result is also sent to operational order by the gesture of standby 101 some the row image recognition objects captured
Triggering equipment 103；And operational order triggers equipment 103, for according to setting from described gesture identification
Standby 102 send the recognition result come triggers scheduled operation order.
In this manual, web camera is used as image capture device, but the present invention is not limited to
This, any kind of known or following it will be appreciated that can capture video images acquisition equipment all
It is used as this image capture device.In the present invention, gesture identification equipment is gestural control system
Core, it identifies significant gesture and by the gesture motion identified and predefined hands
Gesture compares, if the gesture motion identified matches with one of predefined gesture, then identifies
Result is sent to operational order and triggers equipment to trigger predetermined operational order.This is described more fully below
The content of the gesture recognition process of invention.
Before describing the gesture recognition process of the present invention, first provide this gesture recognition process and supported
Predefined gesture, as shown in table 1.As known from Table 1, compared with static gesture, the present invention
Gesture identification method also support dynamic gesture.Such as, when hands " short stay ", hands keeps quiet
The shortest time (such as less than 1 second), when hands keeps " stop ", hands keeps static length
Reaching more than 1 second, " 1 second " is merely illustrative and this value arbitrarily can be set by user here.Additionally,
Such as, when hands motion (to the right, upwards, downwards) to the left, after hands movement track is short stay
Mobile (to the right, upwards, downwards) to the left, and when hands is when brandishing, hands movement track be to the left-to
Right-to the left-downwards (that is, →, ←, →, ←) mobile.Additionally, when hands turns clockwise, hands is transported
Dynamic track is
And when hands rotates counterclockwise, hands movement track is
Here, hands movement rail
Mark can be complete circle or more than half circle.If additionally, user does not make any significant
Gesture, then arbitrary hands movement track is recorded.Although this giving definition of gesture as above, but
Being that the present invention is not limited to this, user can the most at random be defined, and therefore, can be used for
The gesture identified can be more natural.
Table 1: definition of gesture
Then, the detailed process of the gesture recognition process of the present invention will be described.This gesture recognition process from
The image captured detects the hands of object, follows the tracks of the motion of hands, then according to the movement locus of hands with
And predefined gesture carrys out what kind of significant gesture command of recognition expression, detailed process such as figure
Shown in 2.
Fig. 2 shows the flow process of the gesture recognition process performed by gesture identification equipment of the present invention
Figure.As in figure 2 it is shown, the gesture recognition process of the present invention by three main processing stage constitute: inspection
Survey, follow the tracks of and identify.Specifically, a series of figures of the object for being captured by image capture device
Every two field picture in Xiang, performs following process.First, at S201, image capture device caught
The image of the object obtained is normalized, and being such as normalized is 160 pixel × 120 pixels, certainly this
Bright being not limited to this, the big I being normalized arbitrarily is set by user.At S202, according to relevant hands
The record that detection and hands are followed the tracks of determines whether detect in one's hands in previous frame image and whether trace into
Hands, if being not detected by hands in previous frame, then performs the process of relevant hands detection, if upper
One frame is followed the tracks of in one's hands, then performed the process that relevant hands is followed the tracks of.Such as, an embodiment
In, mark isTrack is set for indicating whether detecting in one's hands, when detecting in one's hands, and isTrack
Composed 1, and when being not detected by hands, isTrack is composed 0.It addition, in this embodiment, mark
Will isDectec is set for indicating whether following the tracks of in one's hands, and when following the tracks of in one's hands, isDectec is composed
1, and when not following the tracks of in one's hands, isDectec is composed 0.Therefore, in this embodiment, by inspection
Look into the value of mark isTrack and isDectec just can determine that whether detect in previous frame in one's hands and whether with
Track is in one's hands.
In previous frame, it is not detected by hands and follows the tracks of in the case of in one's hands, in S202 being i.e.
"No", performs the process of relevant hands detection.Specifically, as shown in Figure 2, in S203,
Present image is transformed into gray level image from RGB image, certainly, if the image captured is just
Be gray level image, then this step will be omitted.After Fig. 3 shows captured RGB image and conversion
The example of gray level image.In S204, utilize training in advance good based on LBP (local binary mould
Formula) cascade classifier gray level image after the conversion gamut in inspection side hands.Fig. 4 illustrates and works as
Successfully be detected example during hands in gray level image after the conversion, wherein the scope of hands is by with rectangle frame
Illustrate.As previously described, if be detected that hands, in S206, isDectec is composed 1, without
Detect in one's hands, then in S207, isDectec is composed 0, and these records are saved for next frame
Use when image processes.
When gray level image after the conversion is not detected by the case of hands, the gesture identification of the present invention
Method returns to S201 to process next frame image.On the other hand, when ash after the conversion
Degree image successfully be detected in the case of hands, in S208, current RGB image is transformed into the colour of skin
Image, this is easy to separate area of skin color with background, thus reduce next frame image is carried out hands with
The impact that track is caused due to background when processing.More specifically, used in the present invention by RGB
The method that image is transformed into broca scale picture is as follows: assume that the value of the pixel in broca scale picture is represented by s,
The R component of RGB component of each pixel in RGB image, G component, the value difference of B component
Represented by r, g, b, and set an interim intermediate variable Temp, then the value of s is fixed by equation below
Justice:
Temp=r-((g+b)/2))；
Temp=MAX (0, Temp)；
S=Temp ＞ 140？0:Temp；
That is, first, the value of the R component in the RGB component of each pixel of RGB image is deducted G
Component and the average value of B component, to obtain a difference；Then, by this difference and predetermined threshold ratio
Relatively, if this difference is less than predetermined threshold, then the value of the respective pixel of broca scale picture takes 0, and such as
Really this difference is bigger than predetermined threshold, then the value of the respective pixel of broca scale picture takes this difference.Such as, figure
5 show captured RGB image showing by the broca scale picture after converting with predetermined threshold 140
Example, wherein rectangle frame represents that hands is detected, as it has been described above, area of skin color separates with background.Although
There has been described and as above convert, from RGB image, the method obtaining broca scale picture, but user
Can also be as desired to use other colour of skin segregation methods to obtain broca scale picture.Additionally, currently scheme
The broca scale picture of picture is reserved for the difference image meter when next image carrying out hands tracking and processing
Calculate, and represent that the parameter of the scope of the hands detected also is reserved for carrying out next image
Template matching when hands tracking processes and error evaluation.
In a upper image, have been detected by hands or follow the tracks of in the case of in one's hands, in S202 being i.e.
"Yes", performs the process that relevant hands is followed the tracks of.The hands tracking of the present invention processes and uses the template improved
Joining method, this template matching method calculates faster and can tolerate some of hands hand in motor process
Change.Specifically, in S209, present image is transformed into broca scale picture from RGB image, becomes
Change method as described above.In S210, calculate present image broca scale picture with preserved before
Difference image between the broca scale picture of one image.In the calculating of difference image, only along setting about fortune
The difference in dynamic direction is retained, and the value in opposite direction with hands movement is dropped.Specifically, first,
The value of each pixel of the broca scale picture of calculating present image and the corresponding picture of the broca scale picture of previous image
The difference of the value of element.Then, this difference is compared with 0, if this difference is bigger than 0, then difference diagram
The value of the respective pixel of picture takes this difference, if this difference is less than 0, then and the respective pixel of difference image
Value take 0.Fig. 6 show the broca scale of continuous two images moved right for hands as between
Difference image.
Then, in S210, according to the hands detecting in previous image or tracing into and currently scheme
Difference image between broca scale picture and the broca scale picture of previous image of picture carrys out the colour of skin at present image
Image is followed the tracks of hands.Specifically, first, the model of the hands detecting in previous image or tracing into is utilized
Enclose the hunting zone for following the tracks of hands in the broca scale picture of original definition present image.Such as, exist
The scope of hands is by with in the example shown in rectangle frame, and hunting zone can define according to equation below:
Range.x=MAX (0, Target.x-15)；
Range.y=MAX (0, Target.y-15)；
Range.width=MIN (a-Range.x, Target.width+30)；
Range.height=MIN (b-Range.y, Target.height+30),
Wherein, a left side for the rectangle frame of the hands detected during Target.x, Target.y represent previous image or trace into
The horizontal stroke on the summit at upper angle, vertical coordinate, Target.width, Target.height represent inspection in previous image
The width of the rectangle frame of hands surveyed or trace into and height, and Range.x, Range.y represent current
The horizontal stroke on summit in the upper left corner of the hunting zone in the broca scale picture of image, vertical coordinate,
Range.width, Range.height represent the width of the hunting zone in the broca scale picture of present image
And highly, and a and b represents the horizontally and vertically pixel count of image, and " 15 " respectively,
The numerical value of " 30 " is set based on experience value and according to circumstances may be such that initial setting
Hunting zone is as close possible to other values of the hands scope followed the tracks of.
Next step perform the amount of calculation in template matching method in order to reduce and improve accuracy, as above
After described original definition hunting zone, utilize the broca scale picture of present image and the skin of previous image
Difference image between color image revises the hunting zone of original definition, will be in described difference image
Area-of-interest is as revised hunting zone.As shown in Figure 7, it is shown that in (a) previous image
The scope of the hands detected or trace into, the broca scale picture of (b) present image and the colour of skin of previous image
Difference image between image, and (c) is at the search model defined in the broca scale picture of present image
Enclosing, wherein, in figure, the label in the upper left corner exemplarily represents the frame number of image.Further, Fig. 8 shows
The diagram that the hunting zone of the example as hunting zone correction reduces, wherein, original definition are gone out
The four edges of hunting zone is by gradually to inside contracting, and when any bar limit runs into pixel value more than predetermined threshold
When being worth the pixel of (such as 5), then this limit stops to inside contracting.
After hunting zone defined in the broca scale picture of present image, in S212, perform mould
Plate matching method is to determine the scope of the hands as the present image traced into.Here template matching method refers to
Using the scope of hands that detects or trace in a upper image as To Template, by candidate's hands scope with
This template compares, if error is less than certain value, it is determined that coupling.In order to reduce when hands is crossed over
There is tracking error during some objects (such as, face, other hands) of the colour of skin, above-mentioned template
Join method and consider the movable information from difference image, particularly as follows:
In first candidate's hands scopes multiple defined in hunting zone, those the first candidate hands scopes have with
The size that the size of To Template is identical, and, in second candidate's hands scope defined in difference image,
This candidate's hands scope has the size identical with the size of To Template；
Following steps are performed until these multiple first candidates for the plurality of first candidate's hands scope circulation
Hands scope is all till following matching judgment processes, so that it is determined that go out the time mated most with To Template
Player's scope is as the scope of the hands traced in the broca scale picture of present image: calculate one first
The meansigma methods of the absolute difference of each pixel of candidate's hands scope and To Template is as the first error；If should
First error is more than the first predetermined threshold, then it represents that this candidate's hands scope is not mated with To Template,
Thus be excluded；If this first error less than the first predetermined threshold, then calculates the second error, second
Error is that the meansigma methods of the value of each pixel by the first error deducts second candidate's hands scope is obtained
Value be multiplied by predetermined adjustment factor；If this second error is less than the second predetermined threshold, it is determined that
Join, i.e. this first candidate hands scope is determined to be in the hands traced in the skin color range of present image
Scope, and the value of the second error as Second Threshold so that next first candidate's hands scope is entered
Row matching judgment.
As it has been described above, because this template matching method is by looking for step by step with reference to previous candidate's hands scope
Going out the hands scope mated most, therefore, this makes it possible to the change of the hand tolerating hands in motion continuously
Change.But, in continuous print tracking processes, this result also in the accumulation of matching error.Therefore, may be used
To limit the matching error of history by arranging extra verification process, this verification process includes: meter
Calculate and be currently identified the first candidate's hands scope mated most and hands detected in hands detection equipment
The meansigma methods of the absolute difference of each pixel of scope is as the 3rd error；Judge that the 3rd error is the biggest
In the 3rd threshold value, if the 3rd error more than the 3rd predetermined threshold, then judges currently to be identified
The first candidate's hands scope joined is not mated, thus gets rid of this first candidate hands scope.Fig. 9 shows
Go out the tracking result when hands is move across face, in order to be shown more clearly that effect, optionally shown
Go out the picture frame at interval, and wherein rectangle frame has represented the hands traced into.
Then, in S213, it is judged that whether successfully track in one's hands in S212.If successfully tracked
In one's hands, then in S214, indicate that isTrack is composed 1, otherwise indicates in S215 as mentioned above
IsTrack composed 0 and method flow forward to S203 with to present image perform hands detection process.
In S214, indicate that isTrack is composed in the case of 1, i.e. follow the tracks of in one's hands in present image
In the case of, at S216, the colour of skin of present image is preserved and processes for next frame image
Time difference image calculate, and would indicate that the hands scope traced in present image parameter preserve with
For for the template matching of next frame image is processed, and the position of hands that will trace in present image
The horizontal stroke put, vertical coordinate are respectively defined as in previous image the abscissa of the scope of the hands detecting or tracing into
With the half of width and and previous image in the vertical coordinate of the scope of hands that detects or trace into high
The sum of the half of degree, if (x y) represents the position of the hands traced in present image with HandPos
Coordinate, then it is represented as equation below:
HandPos (x, y)=(Target.x+Target.width/2, Target.y+Target.height/2)
Then, at S217, to all figures in a series of images captured by image capture device
After being carried out the process that hands detects or hands is followed the tracks of, the multiple hand positions passing through to be recorded can get
The movement locus of hands.By analyzing this movement locus and with reference to defined gesture-movement locus mapping table
May recognize that the gesture of object.Specifically, first, calculate in every two adjacent images and detect or follow the tracks of
Between the position of the hands arrived hand position orientation Orient, i.e. calculate by present image detection or with
Track to hands position and previous image in the straight line that constitutes of the position of hands detecting or trace into and level
The angle in direction
Then, direction of displacement DeltaOrient between two adjacent images is calculated.Specifically, will be current
Image is orientated Orient relative to the hand position of previous image and deducts latter image relative to present image
Hand position is orientated LastOrient and obtains difference.If the absolute value of this difference is more than 180, then exist
In the case of described difference is more than 0, described difference deducts 360 and obtains value as picture displacement direction
DeltaOrient；In the case of described difference is less than 0, described difference obtains value and makees plus 360
For picture displacement direction DeltaOrient.If the absolute value of this difference is not more than 180, the most described difference
Value is as picture displacement direction DeltaOrient.
Then, for 8 kinds of significant gestures (" arbitrarily " is used for recording real time kinematics track, and
And " short stay " is provided as the preliminary activities for up/down/left/right gesture, stop
Stay, up/down/left/right, wave as one group, they values based on Orient；CW/CC
As another group, they values based on DeltaOrient and the accumulation of a period of time and, especially, right
In the gesture waved, it includes the alternating movement of motion to the left and to the right), gesture identification is as follows:
Stop
In order to identify the gesture of stop, need that holding in one's hands is detected for continuous STAY_NUM frame quiet
Only, i.e. for continuous STAY_NUM frame, always have the position of the hands traced at present image with
The position of the hands detecting in previous image or tracing into is identical, and wherein STAY_NUM is fixed in advance
The frame number of justice.
Up/down/left/right
In order to identify the gesture of up/down/left/right, first, should detect that preliminary activities is " short
Suspend and stay ", then at ensuing DIREC_NUM frame, the value of Orient should be protected in one direction
Holding within the specific limits, wherein, DIREC_NUM is predefined frame number.As example, false
Fixed " short stay " action is to keep static for 3 frames.Figure 10 (b) show for up/down/
The value scope of the Orient in left/right direction.Such as, if 46≤Orient≤134, then it is assumed that be
Upward direction moves.
Wave:
In order to identify the gesture waved, need to detect 4 motor segments of continuous print: to the right-to the left-to the right-
To the left.In order to detect each motor segment, need Orient value should be maintained at accordingly for N continuous frame
Value in the range of (MIN_SEG_NUM≤N ＜=MAX_SEG_NUM), wherein
MIN_SEG_NUM and MAX_SEG_NUM is predefined frame number threshold value.
Turn clockwise:
In order to identify the gesture turned clockwise, need in some sequential frame images, DeltaOrient
Value just remain, the absolute value of DeltaOrient be maintained at certain limit (such as, more than 10 and
Scope less than or equal to 50) and DeltaOrient absolute value and reached predetermined threshold
CIRCLE_DEGREE。
Rotate counterclockwise:
In the judgement rotated counterclockwise, in addition to the value of DeltaOrient remains and bears, other are equal
Identical with the judgement turned clockwise.
After gesture described above is identified, all enumerators for corresponding gesture candidate will be by
Reset.Additionally, system will be " frozen " a few frame, i.e. nonrecognition gesture in these several frames.This machine
It is the wrong identification in order to avoid being not intended to gesture for some that system is introduced into.Such as, if the user desired that
Constantly making gesture to the right, hands can be retracted after gesture to the right completing one by naturally,
Make next one gesture to the right the most again.If not utilizing this kind " freezing " mechanism, the most this " shifting
Return " gesture being misidentified as to the left will be likely to.
So, the gesture of object is identified.
Then, in S219, it is exported to operational order as the gesture of recognition result and triggers and set
Standby.
Then, equipment is triggered in the operational order as the interface between gesture and corresponding operating order
In, send the gesture identification result come by checking predefined gesture according to from gesture identification equipment
Corresponding operational order is triggered with the mapping table of operational order.Gesture and the mapping table of operational order
As shown in Table 2 below.This table 2 gives and 8 kinds of corresponding looking into for windows picture of gesture
See the mapping between the operation of device software.Although given here be 8 kinds of gestures corresponding for
Mapping between the operation of windows Photo Viewer software, but the invention is not restricted to this, more
Mapping between gesture and the corresponding operating planted can be defined, and operation not only can relate to various types of
The operation of the software of type, it is possible to relating to the operation of difference in functionality to various electronic equipments, this is by user
As desired to definition.Such as, the present invention can be applied to gesture-mouse control.
| Gesture | Operational order |
| Short stay | Carriage return |
| To the left | If previous/picture amplifies, represent and be moved to the left |
| To the right | If the next one/picture amplifies, represent and move right |
| Upwards | If picture amplifies, represent and move up |
| Downwards | If picture amplifies, represent and move down |
| Wave | Exit |
| Turn clockwise | Amplify |
| Rotate counterclockwise | Reduce |
Table 2: gesture-operational order mapping table
The gesture recognition system of the view-based access control model according to the present invention, has and skin even if being moved across at hands
In the case of some object of the color that color is similar, it is also possible to simply and efficiently from capture continuously
Frame image sequence detects hands exactly, follows the tracks of hands movement and identify gesture.
Described above is each parts of the gesture identification remote control system of the view-based access control model according to the present invention and each
Corresponding operating in parts, however can the situation of the purport without departing from the present invention make multiple change and
Amendment, these change and amendment also falls within the scope of the present application.
Claims (9)
1. a gesture remote control system for view-based access control model, including:
Image capture device, described image capture device is for capturing a series of images of object；
Gesture identification equipment, described gesture identification equipment is for being captured from described image capture device
Recognition result is also sent to operational order and triggers equipment by the gesture of a series of images identification object；And
Operational order triggers equipment, and described operational order triggers equipment for according to from described gesture identification
Equipment sends the recognition result come and triggers scheduled operation order, wherein, described gesture identification equipment bag
Include:
Hands detection part, described hands detection part is for being captured from described image capture device
The hands of image detection object；
Hands tracking unit, described hands tracking unit is for when described hands detection part is at an image
In when the hands of object being detected, ensuing image is followed the tracks of the hands of object；
Gesture identification parts, described gesture identification parts are for detecting according to described hands detection part
To the hands of object and the hands of object that traces into of described hands tracking unit to determine the motion of the hands of object
And according to determined by the motion of hands of object identify the gesture of object,
Wherein, described hands tracking unit is by processing the hands following the tracks of object as follows:
Utilize scope and the broca scale picture of present image of the hands detecting in previous image or tracing into
And the difference image between the broca scale picture of previous image, carrys out the broca scale picture of original definition present image
In for following the tracks of the hunting zone of hands；
Perform template matching method to determine the scope of the hands as the present image traced into,
Wherein, described template matching method includes:
In first candidate's hands scopes multiple defined in hunting zone, those the first candidate hands scopes have with
The size that the size of To Template is identical, and, in second candidate's hands scope defined in difference image,
This candidate's hands scope has the size identical with the size of To Template, and wherein, described To Template is
The scope of the hands detected in previous image or trace into；
Following steps are performed until these multiple first candidates for the plurality of first candidate's hands scope circulation
Hands scope is all till following matching judgment processes, so that it is determined that go out the time mated most with To Template
Player's scope is as the scope of the hands traced in the broca scale picture of present image:
Calculate the meansigma methods of first candidate's hands scope and the absolute difference of each pixel of To Template
As the first error；
If this first error is more than the first predetermined threshold, then it represents that this candidate's hands scope not with
To Template mates, thus is excluded；
If this first error is less than the first predetermined threshold, then calculate the second error, the second error
It it is the value that obtained of the meansigma methods of the value of each pixel by the first error being deducted second candidate's hands scope
The value being multiplied by predetermined adjustment factor and obtain；
If this second error is less than the second predetermined threshold, it is determined that coupling, i.e. this first time
Player's scope is determined to be in the scope of the hands traced in the skin color range of present image, and second
The value of error as Second Threshold so that next first candidate's hands scope is carried out matching judgment.
The gesture remote control system of view-based access control model the most according to claim 1, wherein, described hands is examined
Survey parts to pass through the image captured by described image capture device is transformed into gray level image, utilize
Cascade classifier based on local binary patterns is from the hands of this gray level image detection object.
The gesture remote control system of view-based access control model the most according to claim 1, wherein, is determining
After hunting zone in the broca scale picture of present image and before performing template matching method, reference
Described difference image defines in revising the hunting zone of original definition, and hunting zone after the decrease
The plurality of candidate's hands scope also performs described template matching method to determine the scope of the hands of present image,
Wherein, described correction includes: by each limit of the hunting zone of original definition gradually to inside contracting, and works as
When any side runs into the pixel that pixel value is more than predetermined threshold, this limit stops to inside contracting.
The gesture remote control system of view-based access control model the most according to claim 1, wherein, at described mould
In plate matching method, determine for each first candidate's hands scope mate with To Template time, this coupling
Being verified, described checking includes:
Calculate and be currently identified that the first candidate's hands scope mated most is examined with in described hands detection equipment
The meansigma methods of the absolute difference of each pixel of the scope of the hands measured is as the 3rd error；
Judge whether the 3rd error is more than the 3rd threshold value, if the 3rd error is more than the 3rd predetermined threshold
Value, then judge currently to be identified that the first candidate's hands scope mated most is not mated, thus get rid of
This first candidate hands scope.
The gesture remote control system of view-based access control model the most according to any one of claim 1 to 4, its
In, broca scale seems to obtain as follows:
The value of the R component in the RGB component of each pixel of the image captured is deducted G component
With the average of B component, to obtain a difference；And
Described difference is compared with predetermined threshold, if described difference is less than described predetermined threshold, then skin
The value of the respective pixel of color image takes 0, and if described difference bigger than described predetermined threshold, then skin
The value of the respective pixel of color image takes described difference.
The gesture remote control system of view-based access control model the most according to claim 1, wherein, described gesture
Identification component is detected according to described hands detection part and described hands tracking unit at every two field picture or is followed the tracks of
To the position of hands of object, adjacent two two field pictures that obtain from the position calculation of the hands of every two field picture it
Between hand position orientation and the position of adjacent two two field pictures that obtains from adjacent two hand position orientation calculation
Move direction determine object hands motion and by determined by the motion of hands of object with predefined
Gesture-hands movement trajectory map table carries out contrasting the gesture identifying object.
7. a gesture remote control method for view-based access control model, including:
The a series of images of capture object；
Gesture from a series of images identification object captured；
Scheduled operation order is triggered according to recognition result,
Wherein, identify that the gesture of object includes:
Hands from the image detection object captured；
When the hands of object being detected in an image, ensuing image is followed the tracks of object
Hands；And
Hands according to the object detected and the hands of object traced into determine the motion of the hands of object also
The motion of the hands of object determined by according to identifies the gesture of object,
Described gesture remote control method also includes:
Utilize scope and the broca scale picture of present image of the hands detecting in previous image or tracing into
And the difference image between the broca scale picture of previous image, carrys out the broca scale picture of original definition present image
In for following the tracks of the hunting zone of hands；
Perform template matching method to determine the scope of the hands as the present image traced into,
Wherein, described template matching method includes:
In first candidate's hands scopes multiple defined in hunting zone, those the first candidate hands scopes have with
The size that the size of To Template is identical, and, in second candidate's hands scope defined in difference image,
This candidate's hands scope has the size identical with the size of To Template, and wherein, described To Template is
The scope of the hands detected in previous image or trace into；
Following steps are performed until these multiple first candidates for the plurality of first candidate's hands scope circulation
Hands scope is all till following matching judgment processes, so that it is determined that go out the time mated most with To Template
Player's scope is as the scope of the hands traced in the broca scale picture of present image:
Calculate the meansigma methods of first candidate's hands scope and the absolute difference of each pixel of To Template
As the first error；
If this first error is more than the first predetermined threshold, then it represents that this candidate's hands scope not with
To Template mates, thus is excluded；
If this first error is less than the first predetermined threshold, then calculate the second error, the second error
It it is the value that obtained of the meansigma methods of the value of each pixel by the first error being deducted second candidate's hands scope
The value being multiplied by predetermined adjustment factor and obtain；
If this second error is less than the second predetermined threshold, it is determined that coupling, i.e. this first candidate hands
Scope is determined to be in the scope of the hands traced in the skin color range of present image, and the second error
Value as Second Threshold so that next first candidate's hands scope is carried out matching judgment.
The gesture remote control method of view-based access control model the most according to claim 7, wherein, broca scale picture
Obtain as follows:
The value of the R component of the RGB component of each pixel of the image captured is deducted G component with
The average of B component, to obtain a difference；
Described difference is compared with predetermined threshold, if described difference is less than described predetermined threshold, then skin
The value of the respective pixel of color image takes 0, and if described difference bigger than described predetermined threshold, then skin
The value of the respective pixel of color image takes described difference.
9. for following the tracks of a mesh calibration method in image sequence, including:
Utilize scope and the broca scale of present image of the target detecting in previous image or tracing into
Difference image between picture and the broca scale picture of previous image, carrys out the broca scale of original definition present image
The hunting zone for following the tracks of target in Xiang；
Execution template matching method is to determine the scope of the target as the present image traced into, wherein,
Described template matching method includes:
At the first candidate target scopes multiple defined in described hunting zone, those the first candidate target models
Enclose and there is the size identical with the size of To Template, and, defined in described difference image second
Candidate target scope, this candidate target scope has the size identical with the size of To Template, its
In, described To Template is the scope of the target detecting in previous image or tracing into；
Following steps are performed until these multiple first times for the plurality of first candidate target scope circulation
Select target zone all till following matching judgment processes, mate most with To Template so that it is determined that go out
Candidate target scope as the scope of the target traced in the broca scale picture of present image:
Calculate the average of a first candidate target scope and the absolute difference of each pixel of To Template
Value is as the first error；
If this first error is more than the first predetermined threshold, then it represents that this candidate target scope is not
Mate with To Template, thus be excluded；
If this first error is less than the first predetermined threshold, then calculate the second error, the second error
It is that the meansigma methods of the value of each pixel by the first error deducts the second candidate target scope is obtained
The value that value is multiplied by predetermined adjustment factor and obtains；
If this second error is less than the second predetermined threshold, it is determined that coupling, i.e. this first candidate mesh
Mark scope is determined to be in the scope of the target traced in the skin color range of present image, and second
The value of error as Second Threshold so that next first candidate target scope is carried out matching judgment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210121832.XA CN103376890B (en) | 2012-04-16 | 2012-04-16 | The gesture remote control system of view-based access control model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210121832.XA CN103376890B (en) | 2012-04-16 | 2012-04-16 | The gesture remote control system of view-based access control model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103376890A CN103376890A (en) | 2013-10-30 |
| CN103376890B true CN103376890B (en) | 2016-08-31 |
Family
ID=49462114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210121832.XA Active CN103376890B (en) | 2012-04-16 | 2012-04-16 | The gesture remote control system of view-based access control model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103376890B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101534742B1 (en) * | 2013-12-10 | 2015-07-07 | 현대자동차 주식회사 | System and method for gesture recognition of vehicle |
| CN104050454B (en) * | 2014-06-24 | 2017-12-19 | 深圳先进技术研究院 | A kind of motion gesture track acquisition methods and system |
| CN104318218A (en) * | 2014-10-29 | 2015-01-28 | 百度在线网络技术（北京）有限公司 | Image recognition method and device |
| CN105807783A (en) * | 2014-12-30 | 2016-07-27 | 览意科技（上海）有限公司 | Flight camera |
| CN106325485B (en) * | 2015-06-30 | 2019-09-10 | 芋头科技（杭州）有限公司 | A kind of gestures detection recognition methods and system |
| CN106491071A (en) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | A kind of method for giving a test of one's eyesight and terminal |
| CN105223957B (en) * | 2015-09-24 | 2018-10-02 | 北京零零无限科技有限公司 | A kind of method and apparatus of gesture manipulation unmanned plane |
| CN105242614A (en) * | 2015-11-17 | 2016-01-13 | 广州新科佳都科技有限公司 | Platform screen door safety protection control method and system |
| CN106934333B (en) * | 2015-12-31 | 2021-07-20 | 芋头科技（杭州）有限公司 | Gesture recognition method and system |
| CN105657260A (en) * | 2015-12-31 | 2016-06-08 | 宇龙计算机通信科技(深圳)有限公司 | Shooting method and terminal |
| CN105677039B (en) * | 2016-02-16 | 2020-06-09 | 北京博研智通科技有限公司 | Method and device for detecting driving state based on gesture and wearable device |
| CN106022211B (en) * | 2016-05-04 | 2019-06-28 | 北京航空航天大学 | A method of utilizing gesture control multimedia equipment |
| CN106020448B (en) * | 2016-05-06 | 2019-03-29 | 深圳市国华识别科技开发有限公司 | Man-machine interaction method and system based on intelligent terminal |
| CN106094861B (en) * | 2016-06-02 | 2024-01-12 | 零度智控（北京）智能科技有限公司 | Unmanned aerial vehicle, unmanned aerial vehicle control method and unmanned aerial vehicle control device |
| CN106041912B (en) * | 2016-06-16 | 2018-06-22 | 深圳先进技术研究院 | Master-slave mode snake-shaped robot system and its position control method |
| CN106920251A (en) * | 2016-06-23 | 2017-07-04 | 阿里巴巴集团控股有限公司 | Staff detecting and tracking method and device |
| CN106295531A (en) * | 2016-08-01 | 2017-01-04 | 乐视控股（北京）有限公司 | A kind of gesture identification method and device and virtual reality terminal |
| CN108062773B (en) * | 2016-11-07 | 2021-05-28 | 深圳光启合众科技有限公司 | Image processing method and device and robot |
| CN106951871B (en) * | 2017-03-24 | 2020-07-28 | 北京地平线机器人技术研发有限公司 | Motion trajectory identification method and device of operation body and electronic equipment |
| CN109558000B (en) * | 2017-09-26 | 2021-01-22 | 京东方科技集团股份有限公司 | Man-machine interaction method and electronic equipment |
| CN110163055A (en) * | 2018-08-10 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Gesture identification method, device and computer equipment |
| CN110163068A (en) * | 2018-12-13 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Target object tracking, device, storage medium and computer equipment |
| CN110276292B (en) * | 2019-06-19 | 2021-09-10 | 上海商汤智能科技有限公司 | Intelligent vehicle motion control method and device, equipment and storage medium |
| CN110458095B (en) * | 2019-08-09 | 2022-11-18 | 厦门瑞为信息技术有限公司 | Effective gesture recognition method, control method and device and electronic equipment |
| CN110716648B (en) * | 2019-10-22 | 2021-08-24 | 上海商汤智能科技有限公司 | Gesture control method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Gesture identification method and system based on visual sense |
| CN102200830A (en) * | 2010-03-25 | 2011-09-28 | 夏普株式会社 | Non-contact control system and control method based on static gesture recognition |
| CN102339125A (en) * | 2010-07-23 | 2012-02-01 | 夏普株式会社 | Information equipment and control method and system thereof |
-
2012
- 2012-04-16 CN CN201210121832.XA patent/CN103376890B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102200830A (en) * | 2010-03-25 | 2011-09-28 | 夏普株式会社 | Non-contact control system and control method based on static gesture recognition |
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Gesture identification method and system based on visual sense |
| CN102339125A (en) * | 2010-07-23 | 2012-02-01 | 夏普株式会社 | Information equipment and control method and system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103376890A (en) | 2013-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103376890B (en) | The gesture remote control system of view-based access control model | |
| Ji et al. | Advances in view-invariant human motion analysis: A review | |
| Choi et al. | A general framework for tracking multiple people from a moving camera | |
| CN108717531B (en) | Human body posture estimation method based on Faster R-CNN | |
| Raheja et al. | Real-time robotic hand control using hand gestures | |
| Ibraheem et al. | Survey on various gesture recognition technologies and techniques | |
| CN107357427A (en) | A kind of gesture identification control method for virtual reality device | |
| CN106030610B (en) | The real-time 3D gesture recognition and tracking system of mobile device | |
| CN104850219A (en) | Equipment and method for estimating posture of human body attached with object | |
| CN102831439A (en) | Gesture tracking method and gesture tracking system | |
| CN103135753A (en) | Gesture input method and system | |
| CN105589553A (en) | Gesture control method and system for intelligent equipment | |
| CN104063677A (en) | Equipment used for estimating human body posture and method thereof | |
| JP6487642B2 (en) | A method of detecting a finger shape, a program thereof, a storage medium of the program, and a system for detecting a shape of a finger. | |
| JP2011186576A (en) | Operation recognition device | |
| Chen et al. | Vision-based skeleton motion phase to evaluate working behavior: case study of ladder climbing safety | |
| Viraktamath et al. | Comparison of YOLOv3 and SSD algorithms | |
| CN111767932B (en) | Action determination method and device, computer equipment and computer readable storage medium | |
| Jain et al. | Gestarlite: An on-device pointing finger based gestural interface for smartphones and video see-through head-mounts | |
| Gritai et al. | Matching trajectories of anatomical landmarks under viewpoint, anthropometric and temporal transforms | |
| CN102136139B (en) | Targeted attitude analytical equipment and targeted attitude analytical approach thereof | |
| Zhang et al. | Robot-person tracking in uniform appearance scenarios: A new dataset and challenges | |
| KR102227494B1 (en) | Apparatus and method for processing an user input using movement of an object | |
| Kim et al. | Visual multi-touch air interface for barehanded users by skeleton models of hand regions | |
| KR101171239B1 (en) | Non-touch data input and operating method using image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |