CN101794384A - Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry - Google Patents
Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry Download PDFInfo
- Publication number
- CN101794384A CN101794384A CN 201010122916 CN201010122916A CN101794384A CN 101794384 A CN101794384 A CN 101794384A CN 201010122916 CN201010122916 CN 201010122916 CN 201010122916 A CN201010122916 A CN 201010122916A CN 101794384 A CN101794384 A CN 101794384A
- Authority
- CN
- China
- Prior art keywords
- group
- image
- action
- motion
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry. The method includes the following steps: shooting action is acquired into database in advance and is grouped by class, each group constructs a motion diagram, all the actions are rendered into two-dimensional image under multi-angle and then key characteristic is extracted, and image characteristic value of each gesture is calculated. Accurate outline extraction is carried out on shooting picture sequence shot in operation when a person is shooting, characteristic value of skeleton map is calculated, the group in which the gesture most similar to the characteristic value thereof is located is found in database and is taken as a hit group, the group with the most hit in all the skeleton hit of the shooting action is found, then the node where the gesture the most similar to the characteristic value of each skeleton map on the motion diagram is located is found, the points are analyzed and repaired into a continuous section, and the continuous section is taken as action identification result. The invention can fast and accurately identify shooting action only by utilizing an image capturing device.
Description
Technical field
The present invention relates to a kind ofly extract act of shooting recognition methods with the grouping motion inquiry, be specifically related to a kind of contour images and the motion diagram formed of people's act of shooting data method of carrying out people's act of shooting identification by the people based on human body contour outline figure.
Background technology
The recognition technology of human motion always is the research focus of computer vision field, its research purpose is exactly to allow machine can identify people's action, comprise figure, gesture etc., it also is a kind of brand-new people and machine alternant way, this mutual not by gauge point etc. attached to the hardware device on the person, the only motion by people self.Potential application is widely arranged in the application of industry, in these external video display, these digital entertainment industries of playing huge prospect is arranged also even be applied.
Main in existing ripe Motion Recognition technology what rely on is some extraordinary hardware devices, and as motion capture device, but it need come recorder's three dimensional local information and discern motion by the binding mark point on the person, and this influences people's interactive experience very much.The Motion Recognition of unmarked point is a kind of good substitute mode, it does not need to the outer hardware device of user's plus, this method mainly relies on video camera to obtain image again two dimensional image analysis to be estimated three-dimensional motion, recover three-dimensional nature from two dimension is the problem of separating more, so can not reach hundred-percent accuracy of identification, (as interactive entertainment) but in some applications, its accuracy of identification can meet the demands.
The present invention obtains high-quality character contour by people's the shade and the interference of environment in the rejecting image, again by the organic figure of histokinesis, on motion diagram, use the search strategy of multistage expansion and Local Search, realized the basketball movement identification of unmarked point faster.
Summary of the invention
The objective of the invention is to overcome the deficiencies in the prior art, a kind of act of shooting recognition methods of inquiring about with grouping motion of extracting based on human body contour outline figure is provided.
Extracting the act of shooting recognition methods of inquiring about with grouping motion based on human body contour outline figure comprises the steps:
The various acts of shooting of 1) will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into motion diagram;
2) for the attitude on the motion diagram of each group, it is played up two dimensional image under a plurality of visual angles, use the method for machine learning that each group image on each visual angle is extracted the characteristics of image vector;
3) with step 2) in the two dimensional image at each visual angle of each attitude be spliced into a figure, and with step 2) in proper vector calculate the eigenwert of this stitching image, be called the eigenwert of this attitude;
4) take in the scene of basketball action recognition system picture under each visual angle when nobody, be called background;
5) when carrying out action recognition, take the picture of the whole act of shooting sequence under each visual angle of people, be called foreground picture;
6) profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called profile diagram;
7) profile diagram with each visual angle is stitched together, with step 2) in the proper vector that extracts calculate eigenwert, on described each group of step 1), seek and the most similar attitude of his eigenwert, and will be called the group of hitting of this two field picture with the group at the nearest attitude of its eigenwert place.
8) find out act of shooting every two field picture hit group, vote in and hit maximum groups, be called the group of hitting of whole action;
9) find every two field picture to hit the point at the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly the recognition result that the people shoots and moves.
Described various acts of shooting of will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into the motion diagram step comprises:
1) will catch the three-dimensional motion of getting off with motion capture equipment and be divided into four groups, first group comprises forward left hand, forward both hands, left side body left hand, second group comprises forward right hand one hand, forward both hands, the right side body right hand, the 3rd group comprises dorsad both hands, left hand, left side body left hand dorsad, the 4th group comprises dorsad both hands, the right hand, the right side body right hand dorsad, and group forming criterion is the action nature transition in each group;
2) each frame with the everything in every group all intercepts, and each frame is called an attitude;
3) three-dimensional distance between any two attitudes of calculating in every group, represent these attitudes and the distance between them with a motion diagram, node on the motion diagram is an attitude, limit on the motion diagram is the three-dimensional framework distance between the represented attitude of its two node connecting, for each group all makes up a motion diagram.
The described profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called the profile diagram step and comprises:
1), calculates the difference I of input picture and background image for the image that is filmed by video camera of an input
Differ, (x, color value y) are (R to certain point on the definition background image
Background(x, y), G
Background(x, y), B
Background(x, y)), the color value of corresponding point is (R on the input picture
Input(x, y), G
Input(x, y), B
Input(x, y)), I
DifferBe a gray-scale map, it has represented the difference of input picture and background image corresponding point, I
DifferThe formula that calculates be s (x, y)=max (| R
Backgroud(x, y)-R
Backgroud(x, y) |, | G
Backgroud(x, y)-G
Backgroud(x, y) |, | B
Backgroud(x, y)-B
Backgroud(x, y) |); I
Differ(x, y)=s (x, y)/3+s (x-1, y)/6+s (x+1, y)/6+s (x, y-1)/6+s (x, y+1)/6;
2) earlier with a big threshold value with I
DifferBinaryzation, prospect is a white, background is a black, and then expand, will obtain the agglomerate of some rough human bodies, and little noise is by elimination, at first find that maximum in these agglomerates agglomerate, setting a distance then is threshold value, and the agglomerate of the main agglomerate of those distances less than this threshold value is adsorbed onto on this main agglomerate, and the rectangular area R that finds the prospect in this image at last is the zone that human body occurs;
3) earlier the image inside the bounding box is carried out binaryzation with a little threshold value, this will obtain a profile diagram clearly;
4) carry out binaryzation in the bottom of this bounding box with a big threshold value, this will reject people's shadow;
5) again the image that extracts is carried out simple smoothing and noise-reducing process at last, and be placed on the picture centre after human body contour outline is scaled to unified size, obtain profile diagram.
The described point that finds every two field picture to hit the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly that people's recognition result step of moving of shooting comprises:
1) the contour images sequence will be mapped to some points hitting on the motion diagram of group, at first those obviously be departed from the point of colony and remove, and naming a person for a particular job on the action diagram forms some tracts;
2) remove to expand the node that lacks forward and backward from each section beginning, for the node that will infer, with the most contiguous its node that frame was mapped to is the center, with the setpoint distance is a zone of radius, Qu Yu each point hereto, calculate its two dimensional image and the hamming distance of profile diagram on eigenwert that is frame number with this supposition node number, find that point just to think the node of this frame coupling apart from minimum;
3) find out the formed all sequences section of node on the motion diagram, for every section sequence, with the described expanding method of previous step, infer all nodes of whole action sequence, these nodes are formed new sequence, for each new sequence of infer, calculate it each node two dimensional image and the eigenwert of the corresponding frame of original contour figure the hamming distance and, that sequence of picking out this and minimum at last is exactly the result that we match;
4) the result action sequence is carried out interpolation, suitably increase frame number and link up to improve the level and smooth of action.
The carrier of end user's of the present invention profile diagram information This move is as the input of system, by action diagram various acts of shooting of gathering are in advance organically organized as the database support, with the characteristics of image that searches out by various simple machine learning methods as matching tool, the action recognition of three-dimensional being transformed into the images match problem of two dimension. the present invention mainly improves the accuracy rate that improves identification to the quality of key factor-profile diagram of influence net result and the tissue and the inquiry of motion diagram. in profile extraction to the people, at first by the big threshold filter of preceding background differential chart and morphological operation being obtained the bounding box position of the part of human body on the image, filter out the noise jamming in most of zone of expense human region on the image, in the human body bounding box, obtain meticulous human body contour outline then with less threshold filter, remove people's shade in the bottom of human region with the big threshold filter of another one at last, obtain the higher human body contour outline image of quality. in the figure of histokinesis, the various acts of shooting that at first will gather in advance are according to the right-hand man, big class such as single both hands is divided into groups, the decision of making the final vote is then mated on which group, at first the mistake on the big class can not appear in safety action identification, after obtaining mating frame on the original action diagram, at first on motion diagram, the coupling frame of obviously makeing mistakes is detected and repairs, overcome the error of a part to produce because of machine learning, then to a plurality of disconnected continuously deductions after motion diagram is gone forward respectively, the final optimum solution of seeking, matching result is optimized, and when on motion diagram, inferring another with a point, only in a subrange of motion diagram, carry out, can avoid the span between net result frame and the frame excessive, guarantee the continuity and the flatness of the sequence of result's motion.
Description of drawings
Fig. 1 (a) is the scene of gathering act of shooting with motion capture equipment
Fig. 1 (b) is the action that collects with motion capture equipment;
Fig. 2 is to grouping of gathering action and the motion diagram of constructing in every group;
The Background that Fig. 3 gathers when being system's operation;
The picture that has the people (foreground picture) that Fig. 4 gathers when being system's operation in real time;
Fig. 5 is the process that people's profile extracts, and four width of cloth figure are followed successively by according to the direction of arrow: original difference figure, judge bounding box synoptic diagram, little threshold filter meticulousr profile diagram, remove the profile diagram of shade;
Fig. 6 is the synoptic diagram of on the group node that matches for the first time being repaired hitting;
Fig. 7 is the several node continuous segment synoptic diagram that find after to first matched node reparation;
Fig. 8 (a) expands the sequence node (action) of generation to the 1st section continuum among Fig. 7
Fig. 8 (b) expands the sequence node (action) of generation to the 2nd section continuum among Fig. 7.
Embodiment
Extracting the act of shooting recognition methods of inquiring about with grouping motion based on human body contour outline figure comprises the steps:
The various acts of shooting of 1) will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into motion diagram;
2) for the attitude on the motion diagram of each group, it is played up two dimensional image under a plurality of visual angles, use the method for machine learning that each group image on each visual angle is extracted the characteristics of image vector;
3) with step 2) in the two dimensional image at each visual angle of each attitude be spliced into a figure, and with step 2) in proper vector calculate the eigenwert of this stitching image, be called the eigenwert of this attitude;
4) take in the scene of basketball action recognition system picture under each visual angle when nobody, be called background;
5) when carrying out action recognition, take the picture of the whole act of shooting sequence under each visual angle of people, be called foreground picture;
6) profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called profile diagram;
7) profile diagram with each visual angle is stitched together, with step 2) in the proper vector that extracts calculate eigenwert, on described each group of step 1), seek and the most similar attitude of his eigenwert, and will be called the group of hitting of this two field picture with the group at the nearest attitude of its eigenwert place.
8) find out act of shooting every two field picture hit group, vote in and hit maximum groups, be called the group of hitting of whole action;
9) find every two field picture to hit the point at the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly the recognition result that the people shoots and moves.
Described various acts of shooting of will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into the motion diagram step comprises:
1) will catch the three-dimensional motion of getting off with motion capture equipment and be divided into four groups, first group comprises forward left hand, forward both hands, left side body left hand, second group comprises forward right hand one hand, forward both hands, the right side body right hand, the 3rd group comprises dorsad both hands, left hand, left side body left hand dorsad, the 4th group comprises dorsad both hands, the right hand, the right side body right hand dorsad, and group forming criterion is the action nature transition in each group;
2) each frame with the everything in every group all intercepts, and each frame is called an attitude;
3) three-dimensional distance between any two attitudes of calculating in every group, represent these attitudes and the distance between them with a motion diagram, node on the motion diagram is an attitude, limit on the motion diagram is the three-dimensional framework distance between the represented attitude of its two node connecting, for each group all makes up a motion diagram.
The described profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called the profile diagram step and comprises:
1), calculates the difference I of input picture and background image for the image that is filmed by video camera of an input
Differ, (x, color value y) are (R to certain point on the definition background image
Background(x, y), G
Background(x, y), B
Background(x, y)), the color value of corresponding point is (R on the input picture
Input(x, y), G
Input(x, y), B
Input(x, y)), I
DifferBe a gray-scale map, it has represented the difference of input picture and background image corresponding point, I
DifferThe formula that calculates be s (x, y)=max (| R
Backgroud(x, y)-R
Backgroud(x, y) |, | G
Backgroud(x, y)-G
Backgroud(x, y) |, | B
Backgroud(x, y)-B
Backgroud(x, y) |); I
Differ(x, y)=s (x, y)/3+s (x-1, y)/6+s (x+1, y)/6+s (x, y-1)/6+s (x, y+1)/6;
2) earlier with a big threshold value with I
DifferBinaryzation, prospect is a white, background is a black, and then expand, will obtain the agglomerate of some rough human bodies, and little noise is by elimination, at first find that maximum in these agglomerates agglomerate, setting a distance then is threshold value, and the agglomerate of the main agglomerate of those distances less than this threshold value is adsorbed onto on this main agglomerate, and the rectangular area R that finds the prospect in this image at last is the zone that human body occurs;
3) earlier the image inside the bounding box is carried out binaryzation with a little threshold value, this will obtain a profile diagram clearly;
4) carry out binaryzation in the bottom of this bounding box with a big threshold value, this will reject people's shadow;
5) again the image that extracts is carried out simple smoothing and noise-reducing process at last, and be placed on the picture centre after human body contour outline is scaled to unified size, obtain profile diagram.
The described point that finds every two field picture to hit the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly that people's recognition result step of moving of shooting comprises:
1) the contour images sequence will be mapped to some points hitting on the motion diagram of group, at first those obviously be departed from the point of colony and remove, and naming a person for a particular job on the action diagram forms some tracts;
2) remove to expand the node that lacks forward and backward from each section beginning, for the node that will infer, with the most contiguous its node that frame was mapped to is the center, with the setpoint distance is a zone of radius, Qu Yu each point hereto, calculate its two dimensional image and the hamming distance of profile diagram on eigenwert that is frame number with this supposition node number, find that point just to think the node of this frame coupling apart from minimum;
3) find out the formed all sequences section of node on the motion diagram, for every section sequence, with the described expanding method of previous step, infer all nodes of whole action sequence, these nodes are formed new sequence, for each new sequence of infer, calculate it each node two dimensional image and the eigenwert of the corresponding frame of original contour figure the hamming distance and, that sequence of picking out this and minimum at last is exactly the result that we match;
4) the result action sequence is carried out interpolation, suitably increase frame number and link up to improve the level and smooth of action.
Embodiment
Extract with the act of shooting recognition methods step of grouping motion inquiry as follows based on human body contour outline figure:
1) at first gathers various acts of shooting as Fig. 1 a and Fig. 1 b with motion capture device, then database is put in these actions, mode (right-hand man of action and single both hands) according to Fig. 2 is divided into four groups with these actions then, and each action in the database necessarily belongs in wherein one group.In each group, make up a motion diagram at last, the mode that makes up motion diagram is: the attitude that the everything in will organizing is taken into a frame frame apart, be expressed as a node on the figure, calculate the three-dimensional framework distance between any two attitudes, be expressed as connecting two internodal lines;
2) attitude that each node in each group is represented is played up two dimensional image, as the image on the node among Fig. 2, use the method for machine learning that these all images are trained, extract the proper vector of image, the proper vector of image can be selected various general image features, select for use the Lis Hartel of image to levy in this example, the Lis Hartel that extracts 150 images is at last levied;
3) use 150 Lis Hartels that extract to levy the eigenwert of the image that calculates the everything node of graph, be stored on this node;
When 4) bringing into operation, at first take the picture of having no talent in the scene, save as background, the Background during as Fig. 3 under one of them visual angle in our system;
5) keep scene and camera position constant, take picture when the people enters after scene is made act of shooting, this is a foreground picture, is wherein a frame as Fig. 4;
6) profile that under each visual angle each frame foreground picture is carried out following process extracts: the process of extraction such as Fig. 5, foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and obtains profile diagram;
7) for a series of movements sequence of taking, he is made up of a lot of frames, each two field picture is stitched together the profile diagram at each visual angle forms new image, levies with the Lis Hartel of the 2nd step extraction stitching image is calculated, and each two field picture has an eigenwert like this.For each two field picture, find out that node and this gap the most close in each action group respectively with its eigenwert, find out that group of gap minimum in all groups, write down in this group number, be called the group of hitting of this frame;
8) find the group of hitting of every two field picture according to the method in the 7th step, add up each group, select that maximum group of number of times, be called the group of hitting of whole action sequence as the number of times that hits group;
9) for each two field picture, hitting in the group of action, all find that node the most similar to its eigenwert, the matched node that is called this frame, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly the recognition result that the people shoots and moves.
The step that the 6th step is as described in Figure 5 extracted people's profile diagram is:
1) at first calculates the original differences figure I of foreground image and background image
Differ, (x, color value y) are (R to certain point on the definition background image
Background(x, y), G
Background(x, y), B
Background(x, y)), the color value of corresponding point is (R on the input picture
Input(x, y), G
Input(x, y), B
Input(x, y)), then I
DifferAccount form be s (x, y)=max (| R
Backgroud(x, y)-R
Backgroud(x, y) |, | G
Backgroud(x, y)-G
Backgroud(x, y) |, | B
Backgroud(x, y)-B
Backgroud(x, y) |); I
Differ(x, y)=s (x, y)/3+s (x-1, y)/6+s (x+1, y)/6+s (x, y-1)/6+s (x, y+1)/6;
2) determine the residing approximate location of people in the image then, the bounding box position of just calculating the people, earlier with a bigger threshold value with I
DifferBinaryzation, prospect is a white, background is a black, and then expand, will obtain the agglomerate of some more rough human bodies, and little noise is by elimination, at first find that maximum in these agglomerates agglomerate, think that he is the major part of the person, setting a distance then is threshold value, is that as seen all the other are invisible with the main agglomerate of those distances less than the agglomerate of this threshold value, these agglomerates that find so just are considered to people's health, and the minimum bounding box of finding out these agglomerates at last is exactly the bounding box of human body;
3) in bounding box with a less threshold value to I
DifferCarry out filtering, and outside bounding box, all be changed to black, will obtain a comparatively meticulous human body contour outline figure like this, but a slice shadow region may appear in the underfooting part of human body usually;
4) last below the human body bounding box about 1/3 place with a bigger threshold value to I
DifferFiltering will well filter out shade, obtain better people's profile diagram at last;
The step of described the 9th step according to the synthetic last final motion result of original match point is:
1) from the 8th step, can find every two field picture at a match point that hits on the group, these points are as hypographous point among Fig. 6, some continues together these points on motion diagram, some may depart from most of points, those points that obviously depart from colony at first obviously depart from these colony's point and remove as putting 5 points that are considered to matching error;
2) because removed some points, so these points will be repaired, for example the solid black point that becomes among Fig. 6 is repaired in the position of the 5th frame, the method of repairing is to find out some consecutive point at the point of the 6th frame or the some certain limit on every side of the 4th frame earlier, which is the most similar for the eigenwert of calculating the 5th frame profile diagram then and the eigenwert of these consecutive point, and that the most similar node is just thought the matched node of the 5th new frame.；
3) through the erroneous point reparation, remaining will be a section sequence of fracture, each section sequence is formed a continuum, as continuum among Fig. 71 and continuum 2, this method thinks that last that synthetic action sequence comprises one of them continuum certainly, so this step will be that whole action sequence is inferred on the basis with each continuum, this is called multistage and expands.Be based on the deduction that continuum 1 is made as Fig. 8 a, Fig. 8 b is based on the deduction that continuum 2 is made.The algorithm of inferring is to go forward and backward to expand from each section beginning, and establishing the node that will infer is f, the node that on behalf of the action of i frame, it mated, and establishing the father node that is used to expand it is f
Parent(the joint known point of normally the most contiguous i frame) then finds action diagram with f
ParentNode is the center, be a zone of radius at a certain distance, Qu Yu each point hereto, calculate it profile diagram and the profile diagram of i frame between hamming distance on eigenwert, find that point just to think the attitude node that the i frame should mate apart from minimum, the foundation of inferring these points be front and back two frames of an action sequence should be in same regional area on the action diagram, this is called Local Search.At last for each inferred results, calculate it each node two dimensional image and the eigenwert of the corresponding frame of original contour figure the hamming distance and, pick out at last and minimum that, the action sequence of this sequence representative is exactly the result that we match;
4) the result action sequence is carried out interpolation, suitably increase frame number and link up to improve the level and smooth of action.
This method has been implemented as a concrete act of shooting recognition system, under Windows, use C Plus Plus to write realization, the discrimination that mainly being of the performance situation of native system moved, we test system to the identification situation of the motion of different people, test result such as following table:
The personnel that take one's test are two widely different people A and B of figure, everyone respectively does 46 acts of shooting, add up to 92 actions. wherein comprised left hand, the right hand, both hands, lean to one side, with the act of shooting of partly carrying on the back body, three statisticss have been listed according to the different levels that coupling is required, wherein first result is the action after mating and the consistance of the basic attitude of test data, require the right-hand man of test data, single both hands, intuitive motion characteristics such as whether squat down are mated fully, second statistics is the discrimination of concrete shooting hand-type, the position that requires differentiation toss that test data can be good and pitching and sell, the 3rd added up test data towards accuracy rate, requirement can identify forward, lean to one side and dorsad.
Test result shows that native system is all right to the identification of act of shooting, and native system can be used in the interactive entertainment the discrimination of action.
Claims (4)
1. an act of shooting recognition methods of inquiring about based on human body contour outline figure extraction and grouping motion is characterized in that comprising the steps:
The various acts of shooting of 1) will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into motion diagram;
2) for the attitude on the motion diagram of each group, it is played up two dimensional image under a plurality of visual angles, use the method for machine learning that each group image on each visual angle is extracted the characteristics of image vector;
3) with step 2) in the two dimensional image at each visual angle of each attitude be spliced into a figure, and with step 2) in proper vector calculate the eigenwert of this stitching image, be called the eigenwert of this attitude;
4) take in the scene of basketball action recognition system picture under each visual angle when nobody, be called background;
5) when carrying out action recognition, take the picture of the whole act of shooting sequence under each visual angle of people, be called foreground picture;
6) profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called profile diagram;
7) profile diagram with each visual angle is stitched together, with step 2) in the proper vector that extracts calculate eigenwert, on described each group of step 1), seek and the most similar attitude of his eigenwert, and will be called the group of hitting of this two field picture with the group at the nearest attitude of its eigenwert place.
8) find out act of shooting every two field picture hit group, vote in and hit maximum groups, be called the group of hitting of whole action;
9) find every two field picture to hit the point at the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly the recognition result that the people shoots and moves.
2. a kind of act of shooting recognition methods of inquiring about with grouping motion of extracting based on human body contour outline figure as claimed in claim 1, it is characterized in that, described various acts of shooting of will be in advance gathering with motion capture device according to towards with right-hand man's big class grouping, then the action in each group is disassembled independently 3 d pose, the attitude in each group is built into the motion diagram step comprises:
1) will catch the three-dimensional motion of getting off with motion capture equipment and be divided into four groups, first group comprises forward left hand, forward both hands, left side body left hand, second group comprises forward right hand one hand, forward both hands, the right side body right hand, the 3rd group comprises dorsad both hands, left hand, left side body left hand dorsad, the 4th group comprises dorsad both hands, the right hand, the right side body right hand dorsad, and group forming criterion is the action nature transition in each group;
2) each frame with the everything in every group all intercepts, and each frame is called an attitude;
3) three-dimensional distance between any two attitudes of calculating in every group, represent these attitudes and the distance between them with a motion diagram, node on the motion diagram is an attitude, limit on the motion diagram is the three-dimensional framework distance between the represented attitude of its two node connecting, for each group all makes up a motion diagram.
3. a kind of act of shooting recognition methods of inquiring about with grouping motion of extracting based on human body contour outline figure as claimed in claim 1, it is characterized in that, the described profile that carries out following process under each visual angle extracts: foreground picture and background picture are relatively obtained error image, determine bounding box position in the image of human body place according to error image then, in bounding box, determine the precise image of human body again, the shade of removing human body at last disturbs, and the image that obtains is called the profile diagram step and comprises:
1), calculates the difference I of input picture and background image for the image that is filmed by video camera of an input
Differ, (x, color value y) are (R to certain point on the definition background image
Background(x, y), G
Background(x, y), B
Background(x, y)), the color value of corresponding point is (R on the input picture
Input(x, y), G
Input(x, y), B
Input(x, y)), I
DifferBe a gray-scale map, it has represented the difference of input picture and background image corresponding point, I
DifferThe formula that calculates is
s(x，y)＝max(|R
backgroud(x，y)-R
backgroud(x，y)|，|G
backgroud(x，y)-G
backgroud(x，y)|，|B
backgroud(x，y)-B
backgroud(x，y)|)；
I
differ(x，y)＝s(x，y)/3+s(x-1，y)/6+s(x+1，y)/6+s(x，y-1)/6+s(x，y+1)/6；
2) earlier with a big threshold value with I
DifferBinaryzation, prospect is a white, background is a black, and then expand, will obtain the agglomerate of some rough human bodies, and little noise is by elimination, at first find that maximum in these agglomerates agglomerate, setting a distance then is threshold value, and the agglomerate of the main agglomerate of those distances less than this threshold value is adsorbed onto on this main agglomerate, and the rectangular area R that finds the prospect in this image at last is the zone that human body occurs;
3) earlier the image inside the bounding box is carried out binaryzation with a little threshold value, this will obtain a profile diagram clearly;
4) carry out binaryzation in the bottom of this bounding box with a big threshold value, this will reject people's shadow;
5) again the image that extracts is carried out simple smoothing and noise-reducing process at last, and be placed on the picture centre after human body contour outline is scaled to unified size, obtain profile diagram.
4. a kind of act of shooting recognition methods of inquiring about with grouping motion of extracting based on human body contour outline figure as claimed in claim 1, it is characterized in that, the described point that finds every two field picture to hit the attitude place that eigenwert is the most close on the group in action, analyze the relation between these points, use the multistage expansion with the method for Local Search these some reparations to be become one section continuous on motion diagram, the action sequence of this section sequence representative is exactly that people's recognition result step of moving of shooting comprises:
1) the contour images sequence will be mapped to some points hitting on the motion diagram of group, at first those obviously be departed from the point of colony and remove, and naming a person for a particular job on the action diagram forms some tracts;
2) remove to expand the node that lacks forward and backward from each section beginning, for the node that will infer, with the most contiguous its node that frame was mapped to is the center, with the setpoint distance is a zone of radius, Qu Yu each point hereto, calculate its two dimensional image and the hamming distance of profile diagram on eigenwert that is frame number with this supposition node number, find that point just to think the node of this frame coupling apart from minimum;
3) find out the formed all sequences section of node on the motion diagram, for every section sequence, with the described expanding method of previous step, infer all nodes of whole action sequence, these nodes are formed new sequence, for each new sequence of infer, calculate it each node two dimensional image and the eigenwert of the corresponding frame of original contour figure the hamming distance and, that sequence of picking out this and minimum at last is exactly the result that we match;
4) the result action sequence is carried out interpolation, suitably increase frame number and link up to improve the level and smooth of action.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101229166A CN101794384B (en) | 2010-03-12 | 2010-03-12 | Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101229166A CN101794384B (en) | 2010-03-12 | 2010-03-12 | Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101794384A true CN101794384A (en) | 2010-08-04 |
| CN101794384B CN101794384B (en) | 2012-04-18 |
Family
ID=42587066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101229166A Expired - Fee Related CN101794384B (en) | 2010-03-12 | 2010-03-12 | Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101794384B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682302A (en) * | 2012-03-12 | 2012-09-19 | 浙江工业大学 | Human body posture identification method based on multi-characteristic fusion of key frame |
| CN103310191A (en) * | 2013-05-30 | 2013-09-18 | 上海交通大学 | Human body action identification method for motion information imaging |
| CN103955680A (en) * | 2014-05-20 | 2014-07-30 | 深圳市赛为智能股份有限公司 | Action recognition method and device based on shape context |
| CN104408616A (en) * | 2014-11-25 | 2015-03-11 | 苏州福丰科技有限公司 | Supermarket prepayment method based on three-dimensional face recognition |
| CN105396278A (en) * | 2015-12-29 | 2016-03-16 | 太原理工大学 | Basketball training device |
| CN106446757A (en) * | 2016-05-20 | 2017-02-22 | 北京九艺同兴科技有限公司 | Human body motion data similarity automatic evaluation method |
| CN107203638A (en) * | 2017-06-08 | 2017-09-26 | 北京深瞐科技有限公司 | Monitor video processing method, apparatus and system |
| CN107292899A (en) * | 2017-05-05 | 2017-10-24 | 浙江大学 | A kind of Corner Feature extracting method for two dimensional laser scanning instrument |
| CN107566911A (en) * | 2017-09-08 | 2018-01-09 | 广州华多网络科技有限公司 | A kind of live broadcasting method, device, system and electronic equipment |
| CN107688465A (en) * | 2016-08-04 | 2018-02-13 | 惠州学院 | A kind of motion analysis system that swings based on computer vision |
| CN108256433A (en) * | 2017-12-22 | 2018-07-06 | 银河水滴科技（北京）有限公司 | A kind of athletic posture appraisal procedure and system |
| CN108647710A (en) * | 2018-04-28 | 2018-10-12 | 上海与德科技有限公司 | A kind of method for processing video frequency, device, computer and storage medium |
| CN109191366A (en) * | 2018-07-12 | 2019-01-11 | 中国科学院自动化研究所 | Multi-angle of view human body image synthetic method and device based on human body attitude |
| CN109871893A (en) * | 2019-02-18 | 2019-06-11 | 清华大学 | The behavior prediction method and apparatus generated are kept based on circulation time domain |
| CN109961039A (en) * | 2019-03-20 | 2019-07-02 | 上海者识信息科技有限公司 | A kind of individual's goal video method for catching and system |
| CN111061916A (en) * | 2019-12-20 | 2020-04-24 | 中通服咨询设计研究院有限公司 | Video sharing system based on multi-target library image recognition |
| CN111417443A (en) * | 2017-12-06 | 2020-07-14 | 环球城市电影有限责任公司 | Interactive video game system |
| CN112507953A (en) * | 2020-12-21 | 2021-03-16 | 重庆紫光华山智安科技有限公司 | Target searching and tracking method, device and equipment |
| CN113944396A (en) * | 2021-10-25 | 2022-01-18 | 上海网车科技有限公司 | System and method for automatically opening front trunk door of vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5206915A (en) * | 1989-10-10 | 1993-04-27 | Unisys Corporation | Image-based document processing system providing lost image recovery |
| CN1477588A (en) * | 2003-07-01 | 2004-02-25 | �Ϻ���ͨ��ѧ | Automatic human face identification method based on personal image |
| US20040184646A1 (en) * | 2003-03-19 | 2004-09-23 | Fuji Photo Film Co., Ltd. | Method, apparatus, and program for judging images |
| CN101057736A (en) * | 2006-04-18 | 2007-10-24 | 明基电通股份有限公司 | Cradle device and method for using image recognition technology to control swinging state of cradle |
| CN101587541A (en) * | 2009-06-18 | 2009-11-25 | 上海交通大学 | Character recognition method based on human body contour outline |
-
2010
- 2010-03-12 CN CN2010101229166A patent/CN101794384B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5206915A (en) * | 1989-10-10 | 1993-04-27 | Unisys Corporation | Image-based document processing system providing lost image recovery |
| US20040184646A1 (en) * | 2003-03-19 | 2004-09-23 | Fuji Photo Film Co., Ltd. | Method, apparatus, and program for judging images |
| CN1477588A (en) * | 2003-07-01 | 2004-02-25 | �Ϻ���ͨ��ѧ | Automatic human face identification method based on personal image |
| CN101057736A (en) * | 2006-04-18 | 2007-10-24 | 明基电通股份有限公司 | Cradle device and method for using image recognition technology to control swinging state of cradle |
| CN101587541A (en) * | 2009-06-18 | 2009-11-25 | 上海交通大学 | Character recognition method based on human body contour outline |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682302B (en) * | 2012-03-12 | 2014-03-26 | 浙江工业大学 | Human body posture identification method based on multi-characteristic fusion of key frame |
| CN102682302A (en) * | 2012-03-12 | 2012-09-19 | 浙江工业大学 | Human body posture identification method based on multi-characteristic fusion of key frame |
| CN103310191A (en) * | 2013-05-30 | 2013-09-18 | 上海交通大学 | Human body action identification method for motion information imaging |
| CN103310191B (en) * | 2013-05-30 | 2016-12-28 | 上海交通大学 | The human motion recognition method of movable information image conversion |
| CN103955680A (en) * | 2014-05-20 | 2014-07-30 | 深圳市赛为智能股份有限公司 | Action recognition method and device based on shape context |
| CN103955680B (en) * | 2014-05-20 | 2017-05-31 | 深圳市赛为智能股份有限公司 | Action identification method and device based on Shape context |
| CN104408616A (en) * | 2014-11-25 | 2015-03-11 | 苏州福丰科技有限公司 | Supermarket prepayment method based on three-dimensional face recognition |
| CN105396278A (en) * | 2015-12-29 | 2016-03-16 | 太原理工大学 | Basketball training device |
| CN105396278B (en) * | 2015-12-29 | 2018-03-09 | 太原理工大学 | A kind of basketball training device |
| CN106446757A (en) * | 2016-05-20 | 2017-02-22 | 北京九艺同兴科技有限公司 | Human body motion data similarity automatic evaluation method |
| CN107688465A (en) * | 2016-08-04 | 2018-02-13 | 惠州学院 | A kind of motion analysis system that swings based on computer vision |
| CN107292899A (en) * | 2017-05-05 | 2017-10-24 | 浙江大学 | A kind of Corner Feature extracting method for two dimensional laser scanning instrument |
| CN107203638B (en) * | 2017-06-08 | 2020-09-25 | 北京深瞐科技有限公司 | Monitoring video processing method, device and system |
| CN107203638A (en) * | 2017-06-08 | 2017-09-26 | 北京深瞐科技有限公司 | Monitor video processing method, apparatus and system |
| CN107566911B (en) * | 2017-09-08 | 2021-06-29 | 广州方硅信息技术有限公司 | Live broadcast method, device and system and electronic equipment |
| CN107566911A (en) * | 2017-09-08 | 2018-01-09 | 广州华多网络科技有限公司 | A kind of live broadcasting method, device, system and electronic equipment |
| CN111417443A (en) * | 2017-12-06 | 2020-07-14 | 环球城市电影有限责任公司 | Interactive video game system |
| CN108256433A (en) * | 2017-12-22 | 2018-07-06 | 银河水滴科技（北京）有限公司 | A kind of athletic posture appraisal procedure and system |
| CN108647710A (en) * | 2018-04-28 | 2018-10-12 | 上海与德科技有限公司 | A kind of method for processing video frequency, device, computer and storage medium |
| CN109191366A (en) * | 2018-07-12 | 2019-01-11 | 中国科学院自动化研究所 | Multi-angle of view human body image synthetic method and device based on human body attitude |
| CN109871893B (en) * | 2019-02-18 | 2020-10-16 | 清华大学 | Behavior prediction method and device based on cyclic time domain retention generation |
| CN109871893A (en) * | 2019-02-18 | 2019-06-11 | 清华大学 | The behavior prediction method and apparatus generated are kept based on circulation time domain |
| CN109961039A (en) * | 2019-03-20 | 2019-07-02 | 上海者识信息科技有限公司 | A kind of individual's goal video method for catching and system |
| CN111061916A (en) * | 2019-12-20 | 2020-04-24 | 中通服咨询设计研究院有限公司 | Video sharing system based on multi-target library image recognition |
| CN111061916B (en) * | 2019-12-20 | 2023-05-30 | 中通服咨询设计研究院有限公司 | Video sharing system based on multi-target library image recognition |
| CN112507953A (en) * | 2020-12-21 | 2021-03-16 | 重庆紫光华山智安科技有限公司 | Target searching and tracking method, device and equipment |
| CN113944396A (en) * | 2021-10-25 | 2022-01-18 | 上海网车科技有限公司 | System and method for automatically opening front trunk door of vehicle |
| CN113944396B (en) * | 2021-10-25 | 2023-06-06 | 上海网车科技有限公司 | System and method for automatically opening front and back trunk doors of vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101794384B (en) | 2012-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101794384B (en) | Shooting action identification method based on human body skeleton map extraction and grouping motion diagram inquiry | |
| Chen et al. | Sports camera calibration via synthetic data | |
| US20230418389A1 (en) | Information processing device and method, program and recording medium for identifying a gesture of a person from captured image data | |
| CN106780620B (en) | Table tennis motion trail identification, positioning and tracking system and method | |
| US8929602B2 (en) | Component based correspondence matching for reconstructing cables | |
| CN108764065A (en) | A kind of method of pedestrian's weight identification feature fusion assisted learning | |
| US8578299B2 (en) | Method and computing device in a system for motion detection | |
| CN103279765B (en) | Steel wire rope surface damage detection method based on images match | |
| CN109145708B (en) | Pedestrian flow statistical method based on RGB and D information fusion | |
| CN109598684B (en) | Correlation filtering tracking method combined with twin network | |
| CN104851094A (en) | Improved method of RGB-D-based SLAM algorithm | |
| CN107424161B (en) | Coarse-to-fine indoor scene image layout estimation method | |
| CN105160310A (en) | 3D (three-dimensional) convolutional neural network based human body behavior recognition method | |
| CN101551852B (en) | Training system, training method and detection method | |
| CN111724439A (en) | Visual positioning method and device in dynamic scene | |
| CN106991370B (en) | Pedestrian retrieval method based on color and depth | |
| CN107798313A (en) | A kind of human posture recognition method, device, terminal and storage medium | |
| CN110866903A (en) | Ping-pong ball identification method based on Hough circle transformation technology | |
| CN101826155B (en) | Method for identifying act of shooting based on Haar characteristic and dynamic time sequence matching | |
| Yan et al. | Depth map generation for 2d-to-3d conversion by limited user inputs and depth propagation | |
| CN109684941A (en) | One kind picking region partitioning method based on MATLAB image procossing litchi fruits | |
| CN109272577A (en) | A kind of vision SLAM method based on Kinect | |
| CN116052222A (en) | Cattle face recognition method for naturally collecting cattle face image | |
| CN104484679B (en) | Non- standard rifle shooting warhead mark image automatic identifying method | |
| Sokolova et al. | Human identification by gait from event-based camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120418 Termination date: 20150312 |
|
| EXPY | Termination of patent right or utility model |