CN102436301B - Human-machine interaction method and system based on reference region and time domain information - Google Patents
Human-machine interaction method and system based on reference region and time domain information Download PDFInfo
- Publication number
- CN102436301B CN102436301B CN201110239672.4A CN201110239672A CN102436301B CN 102436301 B CN102436301 B CN 102436301B CN 201110239672 A CN201110239672 A CN 201110239672A CN 102436301 B CN102436301 B CN 102436301B
- Authority
- CN
- China
- Prior art keywords
- reference zone
- motion
- image
- human
- rgb color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a human-machine interaction method and system based on reference region and time domain information. The human-machine interaction method comprises the following steps of: acquiring a colored RGB (Red, Green and Blue) image of the reference region; acquiring a consecutive frame subtraction image; counting the motion characteristics of a two-dimensional space and the motion characteristics of the time domain; carrying out characteristic comparison on the calculated motion characteristic vector and an action type template in a preset action type template characteristic library; calculating the similarity of the motion characteristic vector and the motion characteristic template; judging the human body motion in the reference region according to the result obtained by the calculating similarity; and finally, outputting standard data of an action event for calling an outer-layer application program. According to the human-machine interaction method and system disclosed by the invention, the processing flow is simplified and the processing speed is increased. According to a self-adaptive region selection algorithm contained in the invention, the constraint for the human-machine interaction action design is reduced, the flexibility of the human body action is increased and more and more convenient human-machine interaction action types are contained.
Description
Technical field
The present invention relates to Multi-media image processing technical field, in particular a kind of man-machine interaction method based on reference zone and time-domain information and system.
Background technology
Man-machine interaction (Human-Computer Interaction, writes a Chinese character in simplified form HCI): be realize people and computing system to carry out friendship mutual.Along with the fast development of computer technology, the interacting activity of people and computing machine becomes an important component part of people's daily life gradually.The traditional human-computer interaction device such as mouse, keyboard has some limitations in the naturality used and friendly etc., and the human-computer interaction technology that therefore research meets interpersonal communication custom becomes current development trend.
Man-machine interaction mode in the past needs additional specific sensor, such as handle, telepilot, data glove.This mode brings many inconveniences to user operation.The man-machine interaction mode also had has been come by the detection and tracking of staff hand shape, and because the freedom of movement of staff is very large, change of shape is large, and therefore under the natural scene of complexity, verification and measurement ratio is not high.Such method makes the more time burden of mutual needs, is difficult to the demand meeting the application of many real time human-machine interactions.
Therefore, prior art has yet to be improved and developed.
Summary of the invention
The technical problem to be solved in the present invention is, for the above-mentioned defect of prior art, a kind of man-machine interaction method based on reference zone and time-domain information and system are provided, provide and a kind ofly contain more human action dirigibilities and real-time man-machine interaction method based on computer vision, by just having skipped the detection and tracking of staff hand shape based on the system of selection of reference zone and time-domain information process, thus simplify processing procedure, improve man-machine interaction efficiency.
The technical scheme that technical solution problem of the present invention adopts is as follows:
Based on a man-machine interaction method for reference zone and time-domain information, wherein, comprise step:
A, by camera collection human body image, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select in human face region side certain region with reference to region adaptivity, and obtain reference zone RGB color image;
The neighbor frame difference image of reference zone RGB color image in B, obtaining step A;
C, the neighbor frame difference image obtained according to step B calculate the motion feature of human body in two-dimensional space reference zone, by the motion feature of computing reference region RGB color image in certain time domain, and calculate temporal motion proper vector;
D, the motion feature calculated in step C vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculate motion feature vector and the similarity of type of action masterplate;
E, according to step D calculate similarity obtain result the human motion in reference zone is judged, last output action event criteria data, for outer application call;
Described steps A specifically comprises the steps:
A1, calculate the integrogram of the human body image arrived by camera collection, extract the class rectangular characteristic of this human body image, according to predetermined sorter feature database, the method running cascade cascade searches for human face region in this human body image;
A2, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select in human face region side certain region with reference to region adaptivity, the position of computing reference region RGB color image and size:
, wherein, P be comprise reference zone RGB color image central point horizontal ordinate, ordinate, width and height vector, T is mapping function, independent variable
represent the center horizontal ordinate of input human face region, ordinate, width and height respectively.
The described man-machine interaction method based on reference zone and time-domain information, wherein, the predetermined sorter feature database in described steps A 1 comprises the steps:
A11, calculate the integrogram of described human body image, extract the class rectangular characteristic of described human body image;
A12, screen effective feature according to Adaboost algorithm, form Weak Classifier;
A13, by combination multiple Weak Classifier, form strong classifier;
A14, the multiple strong classifier of cascade, form the sorter feature database of Face datection.
The described man-machine interaction method based on reference zone and time-domain information, wherein, described step B specifically comprises the steps:
B1, by described reference zone RGB color image by following formula:
Carry out gray processing process; Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB color image, and f (x, y) represents gray level image, and its value is between 0 ~ 255;
B2, pass through formula:
, wherein
,
be respectively the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this motion change value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.
The described man-machine interaction method based on reference zone and time-domain information, wherein, the statistics two-dimensional space motion feature in described step C specifically comprises:
C11, calculate motor point number according to motion change value described in neighbor frame difference image in step B2, computing formula is
, wherein W, H represent the wide and high of neighbor frame difference image respectively;
C12, calculate motion centroid position according to the motor point number of neighbor frame difference image in step C11, computing formula is
,
;
C13, setting time domain window N, the information of record N continuous frame reference zone RGB color image;
C14, according to step C12, the motion characteristic value of computing reference region RGB color image, computing formula is
, this v value to illustrate in reference zone RGB color image human action in motion state sometime; Then temporal motion proper vector is calculated:
, definition i=1 ~ N, i are natural number, then
represent the i-th frame motion characteristic value
.
The described man-machine interaction method based on reference zone and time-domain information, wherein, described step D comprises:
D1, predefine K type of action masterplate T
i, wherein i=1 ~ K, i are natural number;
D2, utilize absolute value distance motion feature vector V that calculation procedure C14 obtains and type of action masterplate
similarity
;
If D3
, and
<TH, TH are predetermined threshold value, then this type of action is attributed to kth class; If do not met, do not belong to any class.
Based on a man-machine interactive system for reference zone and time-domain information, wherein, comprising:
Reference zone image collection module, for passing through camera collection human body image, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select around face certain region, to obtain reference zone RGB color image with reference to region adaptivity;
Neighbor frame difference acquisition module, for obtaining the neighbor frame difference image of described reference zone RGB color image;
Statistics and computing module, for calculating the motion feature of human body in two-dimensional space reference zone according to the neighbor frame difference image obtained, by the motion feature of computing reference region RGB color image in certain time domain, and calculate temporal motion proper vector;
Characteristic Contrast module, for calculated motion feature vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculates the similarity of motion feature vector and type of action masterplate;
Output module, for judging the human motion in reference zone according to the result calculating similarity acquisition, last output action event criteria data, for outer application call;
Described reference zone image collection module comprises:
Face search unit, for calculating the integrogram of the human body image arrived by camera collection, extracts the class rectangular characteristic of this human body image, and according to predetermined sorter feature database, the method running cascade cascade searches for human face region in this human body image;
Reference zone computing unit, for being with reference to object with face, adopts the adaptively selected algorithm of reference zone to select around face certain region with reference to region adaptivity, the position of computing reference region RGB color image and size:
, wherein, P comprises reference zone RGB color image central point horizontal ordinate, ordinate, width, and the vector of height, T is mapping function, independent variable
represent the center horizontal ordinate of input human face region respectively, ordinate, width and height.
The described man-machine interactive system based on reference zone and time-domain information, wherein, described neighbor frame difference acquisition module comprises:
Gray proces unit, for described reference zone RGB color image is passed through following formula:
Carry out gray processing process; Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB color image, and f (x, y) represents gray level image, and its value is between 0 ~ 255;
Neighbor frame difference image acquisition unit, for passing through formula:
, wherein
,
for the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this motion change value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.
Man-machine interaction method based on reference zone and time-domain information provided by the present invention and system, a kind of method for testing motion made for common RGB image or depth image and motion recognition method, by just having skipped the detection and tracking of staff hand shape based on the system of selection of reference zone and time-domain information process, thus simplify processing procedure, improve man-machine interaction efficiency.
Man-machine interaction method based on reference zone and time-domain information provided by the invention and system.The method, has abandoned required staff hand-type detection and positioning in invention in the past and has followed the tracks of, simplified processing procedure, improve processing speed.The adaptive region selection algorithm that the present invention comprises, decreases the constraint to interactive action design, improves human action dirigibility, contain the type of more interactive actions more easily.
The man-machine interaction method based on reference zone and time-domain information of the present embodiment, owing to having abandoned staff hand shape detection and tracking, having which saved the processing time simultaneously, having decreased the erroneous judgement because causing when staff can't detect.Reduce time loss, the introducing of adaptive reference regional selection method, only need process reference zone and do not need process view picture two field picture, so also saved the time of many image procossing.Automatically set because this reference zone is relative position according to face, so after position of human body changes, reference zone is followed automatically.When the limb motion of people, very naturally drop in this reference zone.Such man-machine interaction is carried out very naturally easily, does not need human body to constrain in specific position.The more human action freedom and flexibility of such containing, promote the convenience of user interactions, improve man-machine interaction efficiency.
Accompanying drawing explanation
Fig. 1 is the man-machine interaction method process flow diagram that the present invention is based on reference zone and time-domain information.
Fig. 2 is the structural representation of step S100 camera collection human body image in the man-machine interaction method that the present invention is based on reference zone and time-domain information.
Fig. 3 is step S100 reference zone selection course schematic diagram in the man-machine interaction method that the present invention is based on reference zone and time-domain information.
Fig. 4 is that in the man-machine interaction method that the present invention is based on reference zone and time-domain information, reference zone confirms procedure chart.
Fig. 5 is the temporal motion feature calculation example to right translation in the man-machine interaction method step S300 that the present invention is based on reference zone and time-domain information.
Fig. 6 is the multiple reference zone selection course schematic diagram that the present invention is based in the man-machine interaction method step S100 of reference zone and time-domain information.
Fig. 7 is the theory diagram of the man-machine interactive system that the present invention is based on reference zone and time-domain information.
Embodiment
The present invention is based on the man-machine interaction method of reference zone and time-domain information and system to provide and a kind ofly contain more human action dirigibilities and real-time man-machine interaction method and system based on computer vision, for making the object of the invention, technical scheme clearly, clearly, also the present invention is elaborated referring to accompanying drawing in conjunction with the embodiments.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
The invention provides a kind of man-machine interaction method based on reference zone and time-domain information, as shown in Figure 1, mainly comprise step:
Step S100: by camera collection human body image is with reference to object with face, adopts the adaptively selected algorithm of reference zone to select in human face region side certain region with reference to region adaptivity, and obtains reference zone RGB color image.As shown in Figure 2, be the collection being carried out human body image by common camera 10 by camera collection human body image, common camera 10 comprises the USB camera on PC, the camera of handheld device, the camera on intelligent television.Common camera is commonly used, and can arbitrarily gather the natural scene comprising human motion, use cost is lower.
Reach a conclusion according to human cinology's principle and a large amount of field experiment, man-machine interaction 80% upper limks movements be very naturally complete at the shoulder near zone of human body, these actions comprise waves, to left, to right translation, upwards translation, downward translation, pressings etc., therefore, the selection of reference zone mainly can be placed on this region.Step S100 concrete methods of realizing is as follows:
S110, detection face, adopt Adaboost Face datection algorithm, by judging whether have face to exist in the image that common camera is absorbed based on the Adaboost Face datection algorithm of class rectangular characteristic.First calculate the human body image integrogram arrived by camera collection, extract the class rectangular characteristic of this human body image, according to the sorter feature database trained, the method running cascade cascade searches for human face region in this human body image.
Wherein sorter feature database training method comprises: A11, calculate the integrogram of the human body image arrived by camera collection, extracts the class rectangular characteristic of this human body image; A12, screen effective feature according to Adaboost algorithm, form Weak Classifier; A13, by combination multiple Weak Classifier, form strong classifier; The multiple strong classifier of A14 cascade, forms the sorter feature database of Face datection.As shown in Figure 3, region 50 shown in solid box is exactly the human face region automatically detected.
S120, be with reference to object with face, reference zone adaptively selected algorithm is adopted to select in human face region side certain region with reference to region adaptivity, as shown in Figure 3, reference zone is shown in dotted line frame 30, after determining reference zone 30, get final product position and the size of computing reference region RGB color image:
, wherein, P is the vector comprising the RGB color image central point horizontal ordinate of reference zone 30, ordinate, width and height, and T is mapping function, independent variable
represent the center horizontal ordinate of human face region 50, ordinate, width and height respectively.
" reference zone " in the embodiment of the present invention take human face region as reference, the region obtained by above reference zone adaptive algorithm.Reference zone is also within the effective angular field of view of camera simultaneously.Such as mapping function T is set as linear transformation function, then
,
.Wherein a, b, l, m are preset value, a represents the transversal displacement of reference zone relative to human face region center, b represents the vertical misalignment amount of reference zone relative to human face region center, m is the multiplication factor of reference zone relative to human face region width, and l is the multiplication factor of reference zone relative to human face region height.
As shown in Figure 4, solid box 50 is human face region, order represents the transversal displacement a=30cm of reference zone 30 relative to human face region 50 center, represent the vertical misalignment amount b=20cm of reference zone 30 relative to human face region 50 center, represent the multiplication factor m=2 of reference zone 30 relative to human face region 50 width, represent the multiplication factor l=2 of reference zone relative to human face region 50 height, shown in the 30(dotted rectangle of region) select certain region near human face region 50 (near such as left shoulder adaptively, preaxial, region above the waist, top area etc.).The introducing of reference zone is the process being different from the entire image frame adopted when in the past doing image procossing, and reference zone is much smaller compared with entire image, and such data processing amount reduces much relatively.
There is reference zone, as long as locking reference zone is observed, just can complete the identification of most common human action.So, the data volume of image procossing can significantly reduce, thus has saved the processing time.
Fig. 6 shows more reference zone example, and solid box 50 is human face region, and dotted line frame 31 is the first reference zone, and dotted line frame 32 is the second reference zone, and dotted line frame 33 is the 3rd reference zone.
First reference zone 31 is relative to the transversal displacement a=30cm of human face region 50 center, relative to the vertical misalignment amount b=30cm of human face region 50 center, relative to the multiplication factor m=2 of human face region 50 width, relative to the multiplication factor l=6 of human face region 50 height.
Second reference zone 32 is relative to the transversal displacement a=10cm of human face region 50 center, relative to the vertical misalignment amount b=20cm of human face region 50 center, relative to the multiplication factor m=6 of human face region 50 width, relative to the multiplication factor l=1.5 of human face region 50 height.
3rd reference zone 33 is relative to the transversal displacement a=30cm of human face region 50 center, relative to the vertical misalignment amount b=30cm of human face region 50 center, relative to the multiplication factor m=2 of human face region 50 width, relative to the multiplication factor l=6 of human face region 50 height.The visual concrete condition of example of above-mentioned each reference zone setting is oppositely arranged according to face location, is not limited to above-mentioned parameter.
After obtaining reference zone RGB color image, namely enter step S200.
Step S200: the neighbor frame difference image of reference zone RGB color image in obtaining step S100;
Step S200 specifically comprises: S210, to step S100 obtain reference zone RGB color image carry out gray processing process, gray processing process expression formula is:
Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB image.F (x, y) represents that gray level image is also luminance graph, and its value is between 0 ~ 255.
S220: pass through formula
, wherein
,
be respectively the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.
After obtaining the neighbor frame difference image of reference zone RGB color image, enter step S300.
Step S300: the motion feature calculating human body in two-dimensional space reference zone according to the neighbor frame difference image obtained in step S200, by the motion feature of computing reference region RGB color image in certain time domain, calculates temporal motion proper vector.
Specifically following steps are adopted during statistics two-dimensional space motion feature:
S311, calculate motor point number according to motion change value described in neighbor frame difference image in step S220: computing formula is
, wherein W, H represent the wide and high of neighbor frame difference image respectively.
S312, calculate motion centroid position according to the motor point number of neighbor frame difference image in step S311, computing formula is
,
.
S313, setting time domain window N, the information of record N continuous frame reference zone RGB color image.
S314, according to step S312, the motion characteristic value of computing reference region RGB color image, computing formula is
, this v value to illustrate in reference zone RGB color image human action in motion state sometime; Then temporal motion proper vector is calculated:
, definition i=1 ~ N, i are natural number, then
represent the i-th frame motion characteristic value
.
The time domain reference figure (moving right 7 frame pictures continuously for staff) of to be a width shown in Fig. 5 be reference zone image, in this figure, continuous 7 frames are respectively the first frame reference zone t1, the second frame reference zone t2, the 3rd frame reference zone t3, the 4th frame reference zone t4, the 5th frame reference zone t5, the 6th frame reference zone t6, the 7th frame reference zone t7, calculate the motion feature that this N continuous=7 frame is corresponding, i.e. V=(0,0,1,1,1,0,0), this proper vector indicates the continuous state of human action.
After calculating temporal motion proper vector, enter step S400.
Step S400: the motion feature calculated in step S300 vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculates the similarity of motion feature vector and type of action masterplate.
Step S400 specifically adopts following steps:
S410, predefine K type of action masterplate
, i is natural number (such as upwards translation, downward translation, to left, to right translation, waves, pressing), such as a large amount of the results shows, reasonable set N=7;
So
and
(
represent the motion barycenter of the i-th frame,
for setting threshold value) represent in image coordinate to right translation.
and
represent to left,
represent waving motion.
S420, utilize absolute value distance motion feature vector V that calculation procedure S314 obtains and type of action masterplate
similarity
, the absolute value distance of similarity selection here calculates.
If
, and
<TH, TH are predetermined threshold value, then this type of action is attributed to kth class; If do not met, do not belong to any class; The motion feature vector being about to calculate carries out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, judges whether to satisfy condition, if otherwise return step S100 acquisition reference zone RGB color image.If satisfied condition, then enter step S500 and input out corresponding type of action.
It is to the right translation that arrow shown in Fig. 3 just illustrates this type of action.
Step S500: the result according to calculating similarity acquisition in step S400 judges the human motion in reference zone, and last output action event criteria data, for calling of outer application program, with output action type.
The present invention is based on the man-machine interaction method of reference zone and time-domain information, may be used for robot and automatically control, internet browsing and guidance operation, the application demands such as game interactive.Such as Internet user is surfed the web by internet television, first human body be in be equipped with camera televisor before, in effective angular field of view of camera, will automatic search to human face region, then a by presetting, b, m and l value self-adaptation finds reference zone and locks, the movable information of record reference zone successive frame, determine the front and back change of certain time domain internal reference area image, when in reference zone, the motion of human body is identified, namely compare according to the type of action in the motion feature calculated and predetermined action template types feature database, determine the type of action exported, as shown in Figure 5, when action is to right translation, trigger the event of page turning to the right.
The man-machine interaction method that the present invention is based on reference zone and time-domain information has abandoned staff hand shape detection and tracking, while saving the processing time, decreases the erroneous judgement because causing when staff can't detect.Reduce time loss, the introducing of adaptive reference regional selection method, only need process reference zone and do not need to process whole two field picture, so also saved the time of many image procossing.And human face detection tech relative maturity is stablized, real-time, be widely used in industrial products.Automatically reference zone sets because this reference zone is relative position according to face, so after position of human body changes, can be followed automatically.When the limb motion of people, very naturally drop in this reference zone.Such man-machine interaction is carried out very naturally easily, does not need human body to constrain in specific position.The more human action freedom and flexibility of such containing, promote the convenience of user interactions, improve man-machine interaction efficiency.
Based on the embodiment of above-mentioned man-machine interaction method, present invention also offers a kind of man-machine interactive system based on reference zone and time-domain information, as shown in Figure 7, it comprises:
Reference zone image collection module 710, for passing through camera collection human body image, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select around face certain region, to obtain reference zone RGB color image with reference to region adaptivity; Specifically as described in above-mentioned steps S100.
Neighbor frame difference acquisition module 720, for obtaining the neighbor frame difference image of described reference zone RGB color image; As detailed above described in step S200.
Statistics and computing module 730, for calculating the motion feature of human body in two-dimensional space reference zone according to the neighbor frame difference image obtained, by the motion feature of computing reference region RGB color image in certain time domain, and calculate temporal motion proper vector; Specifically as shown in above-mentioned steps S300.
Characteristic Contrast module 740, for calculated motion feature vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculates the similarity of motion feature vector and type of action masterplate; Specifically as shown in above-mentioned steps S400.
Output module 750, for judging the human motion in reference zone according to the result calculating similarity acquisition, last output action event criteria data, for outer application call.Specifically as shown in above-mentioned steps S500.
Wherein, described reference zone image collection module comprises:
Face search unit, for calculating the integrogram of the human body image arrived by camera collection, extracts the class rectangular characteristic of this human body image, and according to predetermined sorter feature database, the method running cascade cascade searches for human face region in this human body image; Specifically as shown in above-mentioned S110.
Reference zone computing unit, for being with reference to object with face, adopts the adaptively selected algorithm of reference zone to select around face certain region with reference to region adaptivity, the position of computing reference region RGB color image and size:
, wherein, P comprises reference zone RGB color image central point horizontal ordinate, ordinate, width, and the vector of height, T is mapping function, independent variable
represent the center horizontal ordinate of input human face region respectively, ordinate, width and height, specifically as shown in above-mentioned S120.
And described neighbor frame difference acquisition module comprises:
Gray proces unit, for described reference zone RGB color image is passed through following formula:
Carry out gray processing process; Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB color image, and f (x, y) represents gray level image, and its value is between 0 ~ 255; Specifically as shown in above-mentioned S210.
Neighbor frame difference image acquisition unit, for passing through formula:
, wherein
,
for the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.Specifically as shown in above-mentioned S220.
A kind of man-machine interaction method based on reference zone and time-domain information provided by the present invention and system, make motion for common RGB image or depth image to detect and motion identification, by having skipped the detection and tracking of staff hand shape based on the system of selection of reference zone and time-domain information process, thus simplify processing procedure, improve man-machine interaction efficiency.The staff hand-type detection and positioning that the method has been abandoned to be needed in prior art is followed the tracks of, and simplifies processing procedure, improves processing speed.The adaptive region selection algorithm that the present invention comprises, decreases the constraint to interactive action design, improves human action dirigibility, contain the type of more interactive actions more easily.
Should be understood that, application of the present invention is not limited to above-mentioned citing, for those of ordinary skills, can be improved according to the above description or convert, and all these improve and convert the protection domain that all should belong to claims of the present invention.
Claims (7)
1., based on a man-machine interaction method for reference zone and time-domain information, it is characterized in that, comprise step:
A, by camera collection human body image, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select in human face region side certain region with reference to region adaptivity, and obtain reference zone RGB color image;
The neighbor frame difference image of reference zone RGB color image in B, obtaining step A;
C, the neighbor frame difference image obtained according to step B calculate the motion feature of human body in two-dimensional space reference zone, by the motion feature of computing reference region RGB color image in certain time domain, calculate temporal motion proper vector;
D, the motion feature calculated in step C vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculate motion feature vector and the similarity of type of action masterplate;
E, according to step D calculate similarity obtain result the human motion in reference zone is judged, last output action event criteria data, for outer application call;
Described steps A specifically comprises the steps:
A1, calculate the integrogram of the human body image arrived by camera collection, extract the class rectangular characteristic of this human body image, according to predetermined sorter feature database, the method running cascade cascade searches for human face region in this human body image;
A2, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select in human face region side certain region with reference to region adaptivity, the position of computing reference region RGB color image and size:
, wherein, P be comprise reference zone RGB color image central point horizontal ordinate, ordinate, width and height vector, T is mapping function, independent variable
represent the center horizontal ordinate of input human face region, ordinate, width and height respectively.
2., according to claim 1 based on the man-machine interaction method of reference zone and time-domain information, it is characterized in that, the predetermined sorter feature database in described steps A 1 comprises the steps:
A11, calculate the integrogram of described human body image, extract the class rectangular characteristic of described human body image;
A12, screen effective feature according to Adaboost algorithm, form Weak Classifier;
A13, by combination multiple Weak Classifier, form strong classifier;
A14, the multiple strong classifier of cascade, form the sorter feature database of Face datection.
3., according to claim 1 based on the man-machine interaction method of reference zone and time-domain information, it is characterized in that, described step B specifically comprises the steps:
B1, by described reference zone RGB color image by following formula:
Carry out gray processing process; Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB color image, and f (x, y) represents gray level image, and its value is between 0 ~ 255;
B2, pass through formula:
, wherein
,
be respectively the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this motion change value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.
4., according to claim 3 based on the man-machine interaction method of reference zone and time-domain information, it is characterized in that, the statistics two-dimensional space motion feature in described step C specifically comprises:
C11, calculate motor point number according to motion change value described in neighbor frame difference image in step B2, computing formula is
, wherein W, H represent the wide and high of neighbor frame difference image respectively;
C12, calculate motion centroid position according to the motor point number of neighbor frame difference image in step C11, computing formula is
,
;
C13, setting time domain window N, the information of record N continuous frame reference zone RGB color image;
C14, according to step C12, the motion characteristic value of computing reference region RGB color image, computing formula is
, this v value to illustrate in reference zone RGB color image human action in motion state sometime; Then temporal motion proper vector is calculated:
, definition i=1 ~ N, i are natural number, then
represent the i-th frame motion characteristic value
.
5., according to claim 4 based on the man-machine interaction method of reference zone and time-domain information, it is characterized in that, described step D comprises:
D1, predefine K type of action masterplate T
i, wherein i=1 ~ K, i are natural number;
D2, utilize absolute value distance motion feature vector V that calculation procedure C14 obtains and type of action masterplate
similarity
;
If D3
, and
<TH, TH are predetermined threshold value, then this type of action is attributed to kth class; If do not met, do not belong to any class.
6., based on a man-machine interactive system for reference zone and time-domain information, it is characterized in that, comprising:
Reference zone image collection module, for passing through camera collection human body image, be with reference to object with face, adopt the adaptively selected algorithm of reference zone to select around face certain region, to obtain reference zone RGB color image with reference to region adaptivity;
Neighbor frame difference acquisition module, for obtaining the neighbor frame difference image of described reference zone RGB color image;
Statistics and computing module, for calculating the motion feature of human body in two-dimensional space reference zone according to the neighbor frame difference image obtained, by the motion feature of computing reference region RGB color image in certain time domain, and calculate temporal motion proper vector;
Characteristic Contrast module, for calculated motion feature vector is carried out aspect ratio pair with the type of action masterplate in predetermined action type exemplary feature storehouse, calculates the similarity of motion feature vector and type of action masterplate;
Output module, for judging the human motion in reference zone according to the result calculating similarity acquisition, last output action event criteria data, for outer application call;
Described reference zone image collection module comprises:
Face search unit, for calculating the integrogram of the human body image arrived by camera collection, extracts the class rectangular characteristic of this human body image, and according to predetermined sorter feature database, the method running cascade cascade searches for human face region in this human body image;
Reference zone computing unit, for being with reference to object with face, adopts the adaptively selected algorithm of reference zone to select around face certain region with reference to region adaptivity, the position of computing reference region RGB color image and size:
, wherein, P comprises reference zone RGB color image central point horizontal ordinate, ordinate, width, and the vector of height, T is mapping function, independent variable
represent the center horizontal ordinate of input human face region respectively, ordinate, width and height.
7., according to claim 6 based on the man-machine interactive system of reference zone and time-domain information, it is characterized in that, described neighbor frame difference acquisition module comprises:
Gray proces unit, for described reference zone RGB color image is passed through following formula:
Carry out gray processing process; Wherein x, y are horizontal ordinate and the ordinate of any pixel in described reference zone RGB color image, and f (x, y) represents gray level image, and its value is between 0 ~ 255;
Neighbor frame difference image acquisition unit, for passing through formula:
, wherein
,
for the gray-scale value of adjacent front and back frame pixel,
for the change threshold of setting, calculate the motion change value of any pixel in neighbor frame difference image
, this motion change value be 1 pixel indicate motor point, be that the pixel of 0 represents do not have motor point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110239672.4A CN102436301B (en) | 2011-08-20 | 2011-08-20 | Human-machine interaction method and system based on reference region and time domain information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110239672.4A CN102436301B (en) | 2011-08-20 | 2011-08-20 | Human-machine interaction method and system based on reference region and time domain information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102436301A CN102436301A (en) | 2012-05-02 |
| CN102436301B true CN102436301B (en) | 2015-04-15 |
Family
ID=45984397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110239672.4A Expired - Fee Related CN102436301B (en) | 2011-08-20 | 2011-08-20 | Human-machine interaction method and system based on reference region and time domain information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102436301B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150002419A1 (en) * | 2013-06-26 | 2015-01-01 | Microsoft Corporation | Recognizing interactions with hot zones |
| CN103353935B (en) * | 2013-07-19 | 2016-06-08 | 电子科技大学 | A kind of 3D dynamic gesture identification method for intelligent domestic system |
| DE102014224898A1 (en) * | 2014-12-04 | 2016-06-09 | Robert Bosch Gmbh | Method for operating an input device, input device |
| CN104991684A (en) | 2015-07-23 | 2015-10-21 | 京东方科技集团股份有限公司 | Touch control device and working method therefor |
| CN107330424B (en) * | 2017-08-03 | 2020-10-16 | 重庆邮电大学 | Interaction area and interaction time period identification method, storage device and mobile terminal |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010019024A2 (en) * | 2008-08-13 | 2010-02-18 | Mimos Berhad | Method and system for tracking and tagging objects |
| CN101777114A (en) * | 2009-01-08 | 2010-07-14 | 北京中星微电子有限公司 | Intelligent analysis system and intelligent analysis method for video monitoring, and system and method for detecting and tracking head and shoulder |
| CN101989326A (en) * | 2009-07-31 | 2011-03-23 | 三星电子株式会社 | Human posture recognition method and device |
| CN102081918A (en) * | 2010-09-28 | 2011-06-01 | 北京大学深圳研究生院 | Video image display control method and video image display device |
-
2011
- 2011-08-20 CN CN201110239672.4A patent/CN102436301B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010019024A2 (en) * | 2008-08-13 | 2010-02-18 | Mimos Berhad | Method and system for tracking and tagging objects |
| CN101777114A (en) * | 2009-01-08 | 2010-07-14 | 北京中星微电子有限公司 | Intelligent analysis system and intelligent analysis method for video monitoring, and system and method for detecting and tracking head and shoulder |
| CN101989326A (en) * | 2009-07-31 | 2011-03-23 | 三星电子株式会社 | Human posture recognition method and device |
| CN102081918A (en) * | 2010-09-28 | 2011-06-01 | 北京大学深圳研究生院 | Video image display control method and video image display device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102436301A (en) | 2012-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106598227B (en) | Gesture identification method based on Leap Motion and Kinect | |
| Ke et al. | Computer vision for human–machine interaction | |
| Chen et al. | Repetitive assembly action recognition based on object detection and pose estimation | |
| Zhao et al. | Combing rgb and depth map features for human activity recognition | |
| US7308112B2 (en) | Sign based human-machine interaction | |
| CN103530613B (en) | Target person hand gesture interaction method based on monocular video sequence | |
| CN103376890B (en) | The gesture remote control system of view-based access control model | |
| CN103353935A (en) | 3D dynamic gesture identification method for intelligent home system | |
| Voronin et al. | Action recognition for the robotics and manufacturing automation using 3-D binary micro-block difference | |
| Wu et al. | Robust fingertip detection in a complex environment | |
| CN102436301B (en) | Human-machine interaction method and system based on reference region and time domain information | |
| US8698092B2 (en) | Method and apparatus for motion recognition | |
| CN105536205A (en) | Upper limb training system based on monocular video human body action sensing | |
| CN105107200B (en) | Face Changing system and method based on real-time deep body feeling interaction and augmented reality | |
| CN103135753A (en) | Gesture input method and system | |
| CN109800676A (en) | Gesture identification method and system based on depth information | |
| CN105589553A (en) | Gesture control method and system for intelligent equipment | |
| TW201331891A (en) | Activity recognition method | |
| CN108197534A (en) | A kind of head part's attitude detecting method, electronic equipment and storage medium | |
| CN106502390B (en) | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognition | |
| CN108983980A (en) | A kind of mobile robot basic exercise gestural control method | |
| CN105929947B (en) | Man-machine interaction method based on scene situation perception | |
| Wang et al. | Gesture recognition by using kinect skeleton tracking system | |
| Niranjani et al. | System application control based on Hand gesture using Deep learning | |
| Osipov et al. | Real-time static custom gestures recognition based on skeleton hand |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150415 Termination date: 20200820 |