CN104063677B - For estimating the device and method of human body attitude - Google Patents
For estimating the device and method of human body attitude Download PDFInfo
- Publication number
- CN104063677B CN104063677B CN201310088425.8A CN201310088425A CN104063677B CN 104063677 B CN104063677 B CN 104063677B CN 201310088425 A CN201310088425 A CN 201310088425A CN 104063677 B CN104063677 B CN 104063677B
- Authority
- CN
- China
- Prior art keywords
- posture
- arm
- assumed
- trunk
- left arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000001514 detection method Methods 0.000 claims abstract description 40
- 230000004927 fusion Effects 0.000 claims abstract description 13
- 230000036544 posture Effects 0.000 claims description 185
- 210000000707 wrist Anatomy 0.000 claims description 20
- 230000000295 complement effect Effects 0.000 claims description 11
- 210000003414 extremity Anatomy 0.000 claims description 10
- 210000004197 pelvis Anatomy 0.000 claims description 10
- 238000010801 machine learning Methods 0.000 claims description 8
- 230000002093 peripheral effect Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 4
- 235000013399 edible fruits Nutrition 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 210000001364 upper extremity Anatomy 0.000 description 4
- 238000007689 inspection Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
It provides a kind of for estimating the device and method of human body attitude.A kind of equipment for estimating human body attitude includes: image acquisition unit, for obtaining the depth image including human object；Genius loci detection unit, for extracting human object from the depth image of acquisition and detecting each candidate site and feature of human body, the determining multiple skeletal points of least energy skeleton scanning are carried out to the depth image and construct MESS skeleton, and construct the PIXLA skeleton of each candidate site by the element marking result and depth distribution of each candidate site；Position generates unit, for being assumed by the result of fusion least energy skeleton scanning and the result of element marking to generate the position of each human body；Posture determination unit, for assuming to be assembled at least one posture the position it is assumed that evaluate to the hypothesis of each posture according to posture interpretational criteria and determining human body attitude.
Description
Technical field
The present invention relates to a kind of for estimating the device and method of human body attitude, more particularly to it is a kind of by fusion to including
The depth image of human object carries out the result of the scanning of least energy skeleton (MESS) and element marking to estimate human body attitude
Device and method.
Background technique
With the development of computer vision technique, people can be for the object shown in true 3d space or virtual 3d space
Interact operation.When carrying out this interactive operation, need to carry out naturally contactless remote control to the object of the display.
At this point, human body itself (for example, head, hand/finger/arm, trunk or entire body) can be used as the entity controlled, thus
The object of the display is operated by various movements of the physical feeling in real scene.It in this case, can benefit
With depth camera floor image or video, the posture of human body is estimated based on depth image data, analyzes user whereby
Intention, without can be also manipulated by means of mouse, keyboard, control stick or touch screen etc. in virtual 3d space or true
The object shown in 3d space.In addition, being also required to the posture of identification human body under many other application scenarios.
How people are to estimate that human body attitude has conducted extensive research, still, currently used for estimating the scheme of human body attitude
The only direct estimation human body attitude in the configuration space of single level, this causes operand larger, and estimated accuracy is not high.At this
In the case of kind, existing human body attitude estimation scheme often depends on a large amount of posture sample, still, even if sample size
Very much, it is also difficult to cover the various samples for different building shape, different postures (simple posture or complicated posture), and establish such
The attitude data library of great amount of samples also becomes the problem in machine learning method.
For example, US20100278384 U.S. Patent application " Human body pose estimation " proposes
A kind of system identifying human body attitude based on a large amount of human body attitude samples.This scheme is largely dependent upon sample
Posture, and the training time is too long.Since above scheme can not be related to the institute of various figures when establishing posture tranining database
There is complicated posture, therefore, performance significantly reduces in estimation complicated posture.In addition, US20100197390 United States Patent (USP)
Application " Pose tracking pipeline " discloses a kind of scheme that physical feeling is generated based on pixel clusters, depends on
Element marking result disclosed in US20100278384 United States Patent (USP).In the above scheme, the algorithm comparison of Attitude estimation
Complexity, and carried out in the configuration space of single level, therefore, the accuracy of Attitude estimation result is not high.In addition, the
US20090252423 U.S. Patent application " Controlled human pose estimation from depth image
Streams ", US2010049675A1 U.S. Patent application " Recovery of 3D Human Pose by Jointly
Learning Metrics and Mixtures of Experts ", US2011025834A1 U.S. Patent application
" Method and apparatus of identifying human body posture " also there is a problem of similar, lead
Cause is only applicable to simple posture, and accuracy is not high when estimating complicated posture, alternatively, can not be applicable in since operand is big
In real-time system.
In conclusion there are two main problems for traditional human body attitude estimation scheme tool.One problem is excessively to rely on appearance
Aspect notebook data, and actually collect that cover the different bodily form, the posture sample of different simple/complicated postures extremely difficult,
And if sample size is excessive in attitude data library, sizable difficulty will be also brought to machine-learning process.Another is asked
Topic is not classify to the posture of estimation, for example, simple posture and complicated posture, front/side/intersection etc., it is difficult to estimate
Accurate human body attitude.
Summary of the invention
The purpose of the present invention is to provide one kind can merge least energy skeleton scanning technique and pixel in complementary fashion
Labelling technique estimates the device and method of human body attitude, without relying on huge posture sample data, just compared with subject to
Really human body attitude is estimated from human depth's image.
According to an aspect of the present invention, it provides a kind of for estimating the equipment of human body attitude, comprising: image acquisition unit,
For obtaining the depth image including human object；Genius loci detection unit, for extracting human body from the depth image of acquisition
It is multiple to carry out the scanning determination of least energy skeleton to the depth image for object and each candidate site and feature for detecting human body
Skeletal point simultaneously constructs MESS skeleton, and constructs each candidate by the element marking result and depth distribution of each candidate site
The PIXLA skeleton at position；Position generates unit, the result and element marking for being scanned by fusion least energy skeleton
As a result assume to generate the position of each human body；Posture determination unit, for assuming to be assembled at least the position
One posture according to posture interpretational criteria it is assumed that evaluate to the hypothesis of each posture and determining human body attitude.
Preferably, for any candidate site, genius loci detection unit is according to pixel each in the candidate site
Depth continuity between element marking and pixel determines the continuous skeletal point at the position, to construct the PIXLA at the position
Skeleton.
Preferably, the candidate head that position generates unit self-test, which generates, assumes head, and according to the letter of metastomium
The PIXLA confidence level of breath and the pixel in the hypothesis head generated evaluates the hypothesis head.
Preferably, position generate unit by fusion least energy skeleton scanning result, the result of element marking and
The candidate head of detection estimates trunk hypothesis.
Preferably, position generates unit and determines rough torso area according to the prospect that least energy skeleton scans, and estimates
2D trunk direction is gone from rough torso area unless torso pixel, executes 2D to rough torso area based on element marking result
Trunk modeling, uses the shoulder around the trunk upper/lower detected by element marking/pelvis pixel to determine 3D shoulder respectively
Portion and 3D pelvis.
Preferably, position generates unit and executes the modeling of 2D trunk to rough torso area by following operation: being based on head
Region determines at the top of trunk；Mass center and leg area based on body determine trunk bottom；By according to trunk dimension constraint item
Part project along the inclined direction of trunk the left border and right side boundary of determining trunk, determines from rough torso area final
Torso area.
Preferably, position generates unit also from the complicated posture of the depth image of acquisition identification human body, and to described multiple
The candidate site that miscellaneous posture is related to re-starts label.
The complexity posture may include that leg intersects and hand arm held upward.
Preferably, position generates unit and uses the result of MESS, the result of element marking and motor area in complementary fashion
Domain generates the hypothesis at each position of a small amount of limbs.
Preferably, position, which generates unit and generates lower-left arm by executing following operation, assumes: when detecting left elbow and left wrist
When, lower-left arm is generated by connection left arm and left wrist (left hand) and is assumed；When detecting lower-left arm and left wrist, by connecting lower-left
Arm and left wrist generate lower-left arm and assume；When detecting left elbow and lower-left arm, lower-left arm is generated by connection lower-left arm and left elbow
It is assumed that；Lower-left arm is generated by the PIXLA skeleton extracted from the lower-left arm of detection to assume；Upper body but head is not belonging to from being located at
MESS skeleton generates lower-left arm and assumes；When not finding reliable lower-left arm, it is false that lower-left arm is generated from the moving region of trunk
It is fixed, wherein moving region is detected by frame difference；Assume that the lower-left arm of removal overlapping in the middle assumes from multiple lower-left arms of generation；
Weighting is assumed to each lower-left arm of generation, and removes the low lower-left arm of weight and assumes, wherein according to falling into the lower-left arm
It is assumed that foreground area pixel number and these pixels belong to lower-left arm described in the determine the probability of lower-left arm hypothesis power
Value；Wherein, position generates unit and executes similar operations to generate the hypothesis of right arm, left leg and right leg position.
Preferably, position generates unit and removes unreasonable position it is assumed that with essence also according to the relationship between different parts
Position is selected to assume.
Preferably, posture determination unit includes: posture categorization module, for that will assume to be assembled by each position of human body
At least one described posture is it is assumed that and determine that each posture assumes according at least one positional parameter that each posture assumes
Probability distribution between the posture classification predetermined；And posture evaluation module, for what is assumed using each posture
At least one position binding characteristic assumes probability distribution between the posture classification predetermined to assess each posture,
Then posture hypothesis corresponding with the maximum probability value in the probability distribution of all postures hypothesis after assessment is determined as human body
Posture.
Preferably, the posture categorization module be based on machine learning algorithm, according to each posture assume positional parameter come
Determine that each posture assumes the probability distribution between the posture classification predetermined.
Preferably, at least one described positional parameter includes at least one of following item: the direction of metastomium, arm
The intersection region between intersection region size, leg position between the distance between position and metastomium, arm position is big
It is small.
Preferably, at least one described position binding characteristic includes at least one of following item: the two dimension at arm position
Or depth edge axial continuity, arm or leg in three-dimensional length, the two-dimentional or three-dimensional length at leg position, arm or leg
Middle depth is along perpendicular to axially direction and the contrast of peripheral region, the prospect coverage rate at each position, the depth at each position
Spend the distance between consistency, adjacent parts and angle.
According to another aspect of the present invention, provide a kind of for estimating the method for human body attitude, comprising: A) it obtains including people
The depth image of body object；B human object) is extracted from the depth image of acquisition and detects each candidate site and the spy of human body
Sign carries out the determining multiple skeletal points of least energy skeleton scanning to the depth image and constructs MESS skeleton, and by each
The element marking result and depth distribution of a candidate site construct the PIXLA skeleton of each candidate site；C) minimum by fusion
The result of energy skeleton scanning and the result of element marking assume to generate the position of each human body；D) by the position
It is assumed that being assembled at least one posture it is assumed that evaluate to the hypothesis of each posture according to posture interpretational criteria and determining human body
Posture.
Preferably, for any candidate site, according to the element marking and pixel of pixel each in the candidate site
Between depth continuity determine the continuous skeletal point at the position, to construct the PIXLA skeleton at the position.
Preferably, in step C), the candidate head of self-test, which generates, assumes head, and according to the information of metastomium
And the PIXLA confidence level of the pixel in the hypothesis head generated evaluates the hypothesis head.
Preferably, in step C), pass through result, the result of element marking and the inspection of fusion least energy skeleton scanning
The candidate head of survey estimates trunk hypothesis.
Preferably, in step C), rough torso area is determined according to the prospect that least energy skeleton scans, estimates 2D
Trunk direction is gone from rough torso area unless torso pixel, executes 2D body to rough torso area based on element marking result
Dry modeling, uses the shoulder around the trunk upper/lower detected by element marking/pelvis pixel to determine 3D shoulder respectively
With 3D pelvis.
Preferably, in step C), the modeling of 2D trunk is executed to rough torso area by following operation: being based on header area
Domain determines at the top of trunk；Mass center and leg area based on body determine trunk bottom；By according to trunk size constraint
The left border and right side boundary for project along the inclined direction of trunk determining trunk determine finally from rough torso area
Torso area.
Preferably, in step C), the complicated posture of human body is also identified from the depth image of acquisition, and to the complexity
The candidate site that posture is related to re-starts label.
The complexity posture may include that leg intersects and hand arm held upward.
Preferably, in step C), result, the result of element marking and the moving region of MESS are used in complementary fashion
Generate a small amount of hypothesis limbs.
Preferably, in step C), pass through to execute following operation and generate lower-left arm and assume: when detecting left elbow and left wrist
When, lower-left arm is generated by connection left arm and left wrist (left hand) and is assumed；When detecting lower-left arm and left wrist, by connecting lower-left
Arm and left wrist generate lower-left arm and assume；When detecting left elbow and lower-left arm, lower-left arm is generated by connection lower-left arm and left elbow
It is assumed that；Lower-left arm is generated by the PIXLA skeleton extracted from the lower-left arm of detection to assume；Upper body but head is not belonging to from being located at
MESS skeleton generates lower-left arm and assumes；When not finding reliable lower-left arm, it is false that lower-left arm is generated from the moving region of trunk
It is fixed, wherein moving region is detected by frame difference；Assume that the lower-left arm of removal overlapping in the middle assumes from multiple lower-left arms of generation；
Weighting is assumed to each lower-left arm of generation, and removes the low lower-left arm of weight and assumes, wherein according to falling into the lower-left arm
It is assumed that foreground area pixel number and these pixels belong to lower-left arm described in the determine the probability of lower-left arm hypothesis power
Value；Wherein, in step C), similar operations are executed also to generate the hypothesis of right arm, left leg and right leg position.
Preferably, in step C), unreasonable position is removed also according to the relationship between different parts it is assumed that with selected
Position assumes.
Preferably, step D) it include: that will assume to be assembled at least one described posture it is assumed that simultaneously by each position of human body
Determine that each posture assumes in the posture classification predetermined according at least one positional parameter that each posture assumes
Between probability distribution；Each posture is assessed using at least one position binding characteristic that each posture assumes to assume described
Probability distribution between posture classification predetermined, then by with after assessment all postures assume probability distribution in most
The corresponding posture hypothesis of high probability values is determined as human body attitude.
It is preferably based on machine learning algorithm, determines that each posture assumes according to the positional parameter that each posture assumes
Probability distribution between the posture classification predetermined.
Preferably, at least one described positional parameter includes at least one of following item: the direction of metastomium, arm
The intersection region between intersection region size, leg position between the distance between position and metastomium, arm position is big
It is small.
Preferably, at least one described position binding characteristic includes at least one of following item: the two dimension at arm position
Or depth edge axial continuity, arm or leg in three-dimensional length, the two-dimentional or three-dimensional length at leg position, arm or leg
Middle depth is along perpendicular to axially direction and the contrast of peripheral region, the prospect coverage rate at each position, the depth at each position
Spend the distance between consistency, adjacent parts and angle.
Detailed description of the invention
By the description carried out below with reference to the attached drawing for showing exemplary embodiment of the present, of the invention is above and other
Purpose and feature will become apparent, in which:
Fig. 1 is the system for showing the human body attitude according to an exemplary embodiment of the present invention based on user and carrying out human-computer interaction
Block diagram；
Fig. 2 is the logic diagram for showing the equipment according to an exemplary embodiment of the present invention for being used to estimate human body attitude；
Fig. 3 is the overview flow chart for showing the method according to an exemplary embodiment of the present invention for being used to estimate human body attitude；
Fig. 4 shows the example from the depth image human body detected and feature；
Fig. 5 is the example for showing the MESS skeleton and PIXLA skeleton that construct according to an exemplary embodiment of the present；
Fig. 6 is the flow chart for showing the processing of the step S330 in Fig. 2；
Fig. 7 is the flow chart for showing generation trunk according to an exemplary embodiment of the present invention and assuming；
Fig. 8 is the example for showing the trunk estimated according to an exemplary embodiment of the present；
Fig. 9 and Figure 10 is shown by the example of complicated posture classification according to an exemplary embodiment of the present invention；
Figure 11 is the example for showing the human body attitude by estimating according to an exemplary embodiment of the present.
Specific embodiment
Exemplary embodiment of the present invention will now be described in detail, examples of the embodiments are shown in the accompanying drawings, wherein phase
Same label refers to identical position always.It will illustrate the embodiment, by referring to accompanying drawing below to explain this hair
It is bright.
Fig. 1 is the system for showing the human body attitude according to an exemplary embodiment of the present invention based on user and carrying out human-computer interaction
Block diagram.Referring to Fig.1, the system of the human-computer interaction includes input interface unit 110, attitude estimating device 120, display interface dress
Set 130, Network Interface Unit 140 and Applied layer interface 150.
Input interface unit 110 receives input data (such as from such as depth camera, color camera, stereo camera etc.
Depth image, color image etc.).
Attitude estimating device 120 carries out human body attitude from the received depth image of input interface unit 110 for use and estimates
Meter.The attitude estimating device 120 can also carry out other image procossings for human-computer interaction, at motion detection, color
Reason etc..
DIU display interface unit 130 is used to show input data from input interface unit 110, comes from attitude estimating device
120 human body attitude flow data and other can processing result image (may include, but be not limited to, attitude data, current movement
Speed, acceleration, human body and skeleton size etc.).
Network Interface Unit 140 can send what attitude estimating device 120 exported by local area network, internet or wireless network
Data, and receive related data.Applied layer interface 150 receives posture flow data, identification user's meaning from attitude estimating device 120
Figure, and provide a user relevance feedback.
As described above, attitude estimating device 120 can be integrated into embedded system, to provide automatic Attitude estimation function
Energy.
Fig. 2 shows according to an exemplary embodiment of the present invention for estimating the logic diagram of the equipment of human body attitude.It is described
Equipment can be realized as the attitude estimating device 120 or one component in Fig. 1.
Referring to Fig. 2, for estimating that the equipment of human body attitude includes image acquisition unit 210, genius loci detection unit
220, position generates unit 230, posture determination unit 240 and posture output unit 250.
Image acquisition unit 210 is for obtaining the depth image including human object.For example, the input interface for passing through Fig. 1
Device 110 receives the depth image.
The depth image that genius loci detection unit 220 is used to obtain from image acquisition unit 210 extracts human object simultaneously
The each candidate site and feature for detecting human body determine the depth image progress scanning of least energy skeleton (MESS) multiple
Skeletal point simultaneously constructs MESS skeleton, and passes through the element marking of each candidate site (PIXLA) result and depth distribution structure
Build the PIXLA skeleton of each candidate site.
Genius loci detection unit 220 can be used various existing human object detections and extractive technique from the depth map
As extracting human object, in the processing that the human object detects and extracts, element marking is carried out to the depth image.?
On the basis of this, genius loci detection unit 220 can detect the candidate of each human body by various existing Human Detections
Position and feature.The candidate site of detection includes, but are not limited to rigid body or nearly rigid body position and joint part, detection
Feature includes, but are not limited to foreground features and shape feature.In (the human body detection application No. is 201210141357.2
Method) Chinese patent application in disclose and a kind of detect human object and the method that detects human body from depth image.Fig. 4
It is exemplarily illustrated the position of detection and the example of feature.220 detections of genius loci detection unit/extraction human body as a result,
Although position and feature can not reach 100% verification and measurement ratio or accuracy, but can realize 0% false alarm rate.
Genius loci detection unit 220 also determines multiple skeletal points to depth image progress MESS and constructs MESS bone
Frame.It is a kind of right application No. is being disclosed in the Chinese patent application of 201210176875.8 (human body image resolver and methods)
Human depth's image carries out the scanning of least energy skeleton (MESS), to detect the technical solution of human skeleton point.Certainly, this hair
Bright genius loci detection unit 220 is not limited to the method using above-mentioned Chinese patent application publication when executing MESS.
In addition, according to the present invention, genius loci detection unit 220 also passes through the element marking of each candidate site
(PIXLA) result and depth distribution construct the PIXLA skeleton of each candidate site.Wherein, for any candidate site, position is special
It levies detection unit 220 and institute is determined according to the depth continuity between the element marking and pixel of pixel each in the candidate site
The continuous skeletal point at position is stated, to construct the PIXLA skeleton at the position.
For example, genius loci detection unit 220 can construct the PIXLA skeleton of lower-left arm as follows:
(1) skeletal point of the position center line detected is found.
A. left end point is found: by from center scan image to the left, until depth sharply changes or multiple continuous pictures
Until element is not belonging to lower-left arm；
B. right endpoint is found: by from center scan image to the right, until depth sharply changes or multiple continuous pictures
Until element is not belonging to lower-left arm；
C. using the center between left end point and right endpoint as skeletal point.
(2) the continuous skeletal point for finding position box center or more one by one, until there is no lower-left arm pixel.
(3) position box center continuous skeletal point below is similarly found.
(4) the PIXLA skeleton of lower-left arm position is made of the skeletal point found from (1)~(3), to connect each skeletal point
Line segment indicates lower-left arm position.
Fig. 5 shows the example of the physical feeling of detection and the skeleton pattern of building.It is each that (a) therein shows human body
The mark figure at position (b) shows the position detected indicated with box, (c) shows the PIXLA skeleton at each position of building,
(d) the MESS skeleton of building is shown.
Since PIXLA skeleton is by combining PIXLA result and depth continuity to be extracted, when PIXLA marks result
When comparing accurate, the position skeleton low with surrounding Depth contrasts can be restored.Therefore, when Depth contrasts are low, PIXLA skeleton
It is more complete compared with MESS skeleton.As shown in figure 5, the PIXLA skeleton of (c) arm and leg in Fig. 5 is compared with MESS skeleton in (d)
Corresponding portion is more complete.However, PIXLA depend on PIXLA as a result, and when PIXLA result badly, can not extract
Position skeleton, and MESS skeleton is likely more and is easily extracted.Therefore, PIXLA marks result and MESS scanning result very
It is complementary in big degree.Therefore, the invention proposes combine PIXLA label result and MESS scanning result in a complementary fashion
To estimate the technical solution of human body attitude.
Here, position generates unit 230 and is used to pass through the result and element marking for merging the scanning of least energy skeleton
As a result assume to generate the position of each human body.Unit 230 is generated later with reference to Fig. 6-Figure 10 detailed description position to produce
The processing that the position of raw each human body assumes.
Posture determination unit 240 is used to assume the position that position generation unit 230 generates to be assembled at least one appearance
State according to posture interpretational criteria it is assumed that evaluate to the hypothesis of each posture and determining human body attitude.It here, can be according to generation
The number of posture hypothesis and the genius loci detected determine that the posture interpretational criteria, the posture interpretational criteria may include
Any combination in the element marking result of posture, the 2D length of key position and 3D length and depth continuity etc..
An exemplary embodiment of the present invention, posture determination unit 240 include that posture categorization module and posture assess mould
Block (is not shown) in Fig. 2.
Posture categorization module by each position of human body for that will be assumed be assembled at least one described posture it is assumed that simultaneously root
Determined according at least one positional parameter that each posture assumes each posture assume the posture classification predetermined it
Between probability distribution.Posture categorization module is based on machine learning algorithm, is determined according to the positional parameter that each posture assumes every
A posture assumes the probability distribution between the posture classification predetermined.At least one described positional parameter includes following
At least one of: intersecting between the distance between the direction of metastomium, arm position and metastomium, arm position
Intersection region size between area size, leg position.
Posture evaluation module is used at least one position binding characteristic for assuming using each posture to assess each posture
It is assumed that the probability distribution between the posture classification predetermined, the probability that then will be assumed with all postures after assessment
The corresponding posture hypothesis of maximum probability value in distribution is determined as human body attitude.At least one described position binding characteristic include with
At least one of lower item: the two-dimentional or three-dimensional length at arm position, leg position two-dimentional or three-dimensional length, arm or leg
Middle depth along axial continuity, arm or leg depth along perpendicular to axially direction and the contrast of peripheral region, every
The distance between the prospect coverage rate at a position, the depth consistency at each position, adjacent parts and angle.
Posture output unit 250 is used to export the data for the human body attitude that posture determination unit 240 determines.
Fig. 3 is the overview flow chart for showing the method according to an exemplary embodiment of the present invention for being used to estimate human body attitude.
Referring to Fig. 3, in step S310, image acquisition unit 210 obtains the depth image including human object.
In step S320, genius loci detection unit 220 extracts human object from the depth image of acquisition and detects human body
Each candidate site and framework characteristic, the scanning of least energy skeleton is carried out to the depth image and determines multiple skeletal points and structure
MESS skeleton is built, and each candidate site is constructed by the element marking result and depth distribution of each candidate site
PIXLA skeleton.According to an alternative embodiment of the invention, for any candidate site, genius loci detection unit 320 is according to described
Depth continuity in candidate site between the element marking and pixel of each pixel determines the continuous skeletal point at the position, with
Construct the PIXLA skeleton at the position.
In step S330, position generates unit 230 and passes through the result and element marking of fusion least energy skeleton scanning
Result come generate each human body position assume.In step S330, in a complementary fashion merge MESS result and
PIXLA result generates the hypothesis including head, trunk, four limbs and joint part.
Fig. 6 shows the flow chart that position generates processing of the unit 230 in step S330.It may be noted that the place shown in Fig. 6
Reason is only exemplary process, and the present invention is not limited to processing shown in Fig. 6, and part shown in Fig. 6 also can be performed as needed
Processing.
Referring to Fig. 6, in step 610, the candidate head that position generates 230 self-test of unit, which generates, assumes head, and root
The hypothesis head is commented according to the PIXLA confidence level of the pixel in the information of metastomium and the hypothesis head of generation
Valence.When generating head hypothesis, the head of error detection can be removed according to the information of the metastomium of estimation, or tie from MESS
Fruit finds the head of missing.
In step 620, position generates unit 230 and passes through the result of fusion least energy skeleton scanning, the knot of element marking
The candidate head of fruit and detection estimates trunk hypothesis.Trunk as visible position maximum in human body with four limbs by connecting
It connects, contains the abundant information for carrying out motion analysis naturally.But due in complicated posture, trunk and arm it
Between complicated mutual hiding relation, it is not easy to accurately, steadily estimate trunk.Fig. 7 shows according to the present invention exemplary
The process of the processing of the trunk estimation of embodiment.
Referring to Fig. 7, in step 710, position generates unit 230 and scans the foreground zone that (MESS) goes out according to least energy skeleton
Domain determines rough torso area.Preferred embodiment in accordance with the present invention can mark result to determining rough trunk according to PIXLA
Region carries out micronization processes.For example, the boundary at the top of trunk can be refined by the head pixel in limitation trunk, similarly by
Limit the boundary of the leg pixel refinement trunk bottom in trunk.
In step 720, position generates unit 230 and estimates 2D trunk direction.
In step 730, position is generated unit 230 and is gone from rough torso area based on element marking result unless torso
Element, and calculate trunk depth and predict the size of trunk.
In step 740, position generates unit 230 and executes the modeling of 2D trunk to rough torso area.It is according to the present invention to show
Example property embodiment, position generate unit 230 and execute the modeling of 2D trunk by following processing:
A. it is determined at the top of trunk based on head zone.When there is no head, it is assumed that head is blocked by arm, will be above trunk
Depth areas as head.
B. trunk bottom is determined based on the mass center of body and leg area.
C. by project along the inclined direction of trunk the left border of determining trunk according to trunk size constraint
And right side boundary, final torso area is determined from rough torso area.Assuming that the weight of torso pixel is 2, rough trunk area
The weight of remaining foreground pixel of domain is 1.By will be along the weight of whole torso pixels of the position of the inclined direction of trunk
It is added the predicted value for calculating the position.Right boundary is determined as to be lower than the prediction of specific threshold from center to two Slideslips
Value.
D. final torso area, i.e. the 6 of human trunk model point are determined.For example, can be using TorsoNL/TorsoNR as certainly
Top center is scanned to left/right both sides, until prospect boundary or rough torso area boundary；Using TorsoPL/TorsoPR as
It is scanned from bottom centre to left/right both sides, until prospect boundary or rough torso area boundary；TorsoSL/TorsoSR is made
To be scanned from the point lower than 1/3 torso length of top center to left/right, until prospect boundary or rough torso area boundary.
In step 750, position generates the joint part that unit 230 estimates trunk using PIXLA result, specifically, point
The shoulder around the trunk upper/lower detected by element marking/pelvis pixel is not used to determine 3D shoulder and 3D pelvis.
By the processing of step 710~750, position generates unit 230 and estimates one or more trunks hypothesis.Fig. 8 shows
The trunk and corresponding joint part estimated out.
Since the major beat of some human body attitudes is complicated, it is not easy to differentiate the accurate location of four limbs and other positions at this time
And skeleton, therefore, preferred embodiment in accordance with the present invention, position generates unit 230 and executes step 630, from the depth map of acquisition
As the complicated posture of identification human body, and the candidate site being related to the complicated posture re-starts label.The complexity appearance
State includes but is not limited to that leg intersects and hand arm held upward.
By taking leg intersects posture as an example, it is not easy correctly to distinguish the position of left leg and right leg at this time.Leg intersects to be divided into again
Both legs top intersects, both legs lower part intersects, leg intersects with another lower leg on one.It can be by a variety of methods (such as based on engineering
The method of habit) determine that leg intersects posture.A kind of use MESS depth areas and PIXLA result introduced below determine that leg intersects
The method of state.
Leg in front will not be sheltered from by another one leg, and lower leg and upper leg are continuous in depth.Therefore, if
Meet one of the following conditions, then can be identified as leg in front:
1) through the PIXLA upper leg detected and lower leg in same MESS depth areas, as shown in (a) in Fig. 9；
2) leg region (descending MESS depth areas existing for leg, either left leg or right leg) supreme leg region has under
Depth continuity, as shown in (b) in Fig. 9.
It can determine that leg is left leg or right leg by the attribute of the upper leg detected.For the leg of front, can re-flag
Candidate site based on PIXLA.For using left leg as the leg of front, it can remove positioned at the bottom right leg in foreleg region or by position
Bottom right leg in foreleg region re-flags as left lower leg, and can remove the not left lower leg in foreleg region or will not have
There is the left lower leg in foreleg region to re-flag for bottom right leg.This can substantially reduce the ambiguity of PIXLA result, therefore can
Leg is accurately generated to assume.
Another complexity posture is that arm lifts excessive posture, and the shoulder detection and ancon for this posture based on PIXLA are examined
Survey is easy to happen mistake.As shown in (a) in Figure 10, the shoulder and ancon being initially detected are misplaced.In this posture, inspection
The ancon of survey will be on shoulder, therefore can re-flag the shoulder and ancon of error detection.When the arm of detection is in trunk
Or when more than head, the posture can be determined as hand arm held upward.Then, compare the position of the shoulder and ancon that detect, such as
Shoulder and the ancon dislocation that fruit detects, then change the label of shoulder and ancon, assumes to be conducive to subsequent generation arm.
Intersect and hand arm held upward as previously mentioned, complicated posture is not limited to leg, it may be determined that and the more complicated postures of addition,
And different testing results, the identification for designing complicated posture in different ways and its position label may be selected.
Fig. 6 is returned to, after completing step 630, in step 640,650 and 660, position generates unit 230 in complementary fashion
The hypothesis at each position of a small amount of limbs is generated using the result of MESS, the result of element marking and moving region.With the vacation of lower-left arm
For fixed generation, following operation is executed:
When detecting left elbow and left wrist, lower-left arm is generated by connection left arm and left wrist (left hand) and is assumed；
When detecting lower-left arm and left wrist, lower-left arm is generated by connection lower-left arm and left wrist and is assumed；
When detecting left elbow and lower-left arm, lower-left arm is generated by connection lower-left arm and left elbow and is assumed；
Lower-left arm is generated by the PIXLA skeleton extracted from the lower-left arm of detection to assume；
From be located at upper body but be not belonging to head MESS skeleton generate lower-left arm assume；
When not finding reliable lower-left arm, lower-left arm is generated from the moving region of trunk and is assumed, wherein is poor by frame
Detect moving region；
Assume that the lower-left arm of removal overlapping in the middle assumes from multiple lower-left arms of generation；
Weighting is assumed to each lower-left arm of generation, and removes the low lower-left arm of weight and assumes, wherein according to falling into
The number and these pixels for stating the pixel of the foreground area of lower-left arm hypothesis belong to lower-left arm described in the determine the probability of lower-left arm
It is assumed that weight.
Similarly, position generates the executable operation similar with the hypothesis generation of lower-left arm of unit 230, to generate right arm, left leg
With the hypothesis of right leg position.It should be understood that the above method is only used for merging MESS result, PIXLA knot in a complementary fashion
Fruit and moving region generate a kind of exemplary approach of the hypothesis at each position of limbs, and other strategies can be used to generate limbs
The hypothesis at position.
In addition, different joints can be generated from the result of the candidate joint part detected it is assumed that for example, can will be based on
The center of the shoulder of PIXLA detection is determined as left shoulder and assumes.
Later, in step 670, position generates unit 230 and removes unreasonable portion also according to the relationship between different parts
Position is it is assumed that with the hypothesis of selected position.For example, can carry out it is selected so that upper arm is in close to shoulder, and between lower arm and upper arm
Closely etc..Various rational information and posture restraint can be used to carry out the selected of the position hypothesis.
The processing of the step S330 in Fig. 3 has been described in detail above by reference to Fig. 6~Figure 10, position generates unit 230 and passes through
Fusion least energy skeleton scanning result and element marking as a result, produce each human body position assume.
Turning now to Fig. 3, in the position that step S340, posture determination unit 240 generate position generation unit 230
It is assumed that being assembled at least one posture it is assumed that evaluate to the hypothesis of each posture according to posture interpretational criteria and determining human body
Posture.Various posture interpretational criterias and any combination thereof can be used to evaluate the hypothesis of at least one posture of assembling, such as
(depth is along axial in the direction of such as metastomium, the length of arm and leg, arm or leg for the parameter for the key position being related to
Continuity, the prospect coverage rate at each position, the depth consistency at each position, the distance between adjacent parts and angle
Deng).
Preferred embodiment in accordance with the present invention, step S340 include: that posture categorization module is false by each position by human body
Surely at least one described posture is assembled into it is assumed that and determining each appearance according at least one positional parameter that each posture assumes
State assumes the probability distribution between the posture classification predetermined；Posture evaluation module is assumed extremely using each posture
Lack a position binding characteristic to assess probability distribution of each posture hypothesis between the posture classification predetermined, so
Posture hypothesis corresponding with the maximum probability value in the probability distribution of all postures hypothesis after assessment is determined as human body appearance afterwards
State.
Preferred embodiment in accordance with the present invention, posture categorization module are based on machine learning algorithm, are assumed according to each posture
Positional parameter determine that each posture assumes the probability distribution between the posture classification predetermined.
Preferred embodiment in accordance with the present invention, at least one described positional parameter include at least one of following item: body
Intersection region size, leg position between the distance between direction, arm position and metastomium of cadre position, arm position
Between intersection region size.
Preferred embodiment in accordance with the present invention, at least one described position binding characteristic include at least one in following item
A: depth is along axial in the two-dimentional or three-dimensional length at arm position, the two-dimentional or three-dimensional length at leg position, arm or leg
Depth is covered along the prospect perpendicular to axially direction and the contrast of peripheral region, each position in continuity, arm or leg
The distance between lid rate, the depth consistency at each position, adjacent parts and angle.
Hereafter, in step S350, posture output unit 250 exports the letter for the human body attitude that posture determination unit 240 determines
Breath.The information of the human body attitude may include each human body dimension information (such as 2D/3D length and 2D/3D width),
The position 3D of joint part and tilt angle etc..
Since the method and apparatus according to the present invention for estimating human body attitude is relatively smaller based on effective observation generation
The attitude data of amount, therefore can relatively rapid, accurately obtain the estimated result of different postures.Figure 11 is shown for different people
Body posture, the final carriage estimated by the depth image shot indicate (image of rightmost in each four-tuple).
Therefore, according to the present invention for estimating that the method and apparatus of human body attitude merges the result of MESS in complementary fashion
It, can be relatively accurately from human depth's image with PIXLA's as a result, in the case where not depending on huge posture sample data
In estimate human body attitude.
Although being particularly shown and describing the present invention, those skilled in the art referring to its exemplary embodiment
It should be understood that in the case where not departing from the spirit and scope of the present invention defined by claim form can be carried out to it
With the various changes in details.
Claims (29)
1. a kind of for estimating the equipment of human body attitude, comprising:
Image acquisition unit, for obtaining the depth image including human object；
Genius loci detection unit, for extracting human object from the depth image of acquisition and detecting each candidate site of human body
And feature, the determining multiple skeletal points of least energy skeleton scanning are carried out to the depth image and construct the scanning of least energy skeleton
MESS skeleton, and construct by the element marking result and depth distribution of each candidate site the pixel mark of each candidate site
Remember PIXLA skeleton；
Position generates unit, for each to generate by the result of fusion least energy skeleton scanning and the result of element marking
The position of a human body assumes；
Posture determination unit is assembled at least one posture it is assumed that according to posture interpretational criteria for assuming the position
The hypothesis of each posture is carried out to evaluate and determine human body attitude,
Wherein, for any candidate site, genius loci detection unit is according to the pixel mark of pixel each in the candidate site
Depth continuity between note and pixel determines the continuous skeletal point at the position, to construct the PIXLA skeleton at the position,
Wherein, at least one position binding characteristic that posture determination unit is assumed using each posture assumes to assess each posture
Probability distribution between posture classification predetermined, by the highest in the probability distribution assumed with all postures after assessment
The corresponding posture hypothesis of probability value is determined as human body attitude.
2. equipment as described in claim 1, which is characterized in that the candidate head that position generates unit self-test, which generates, assumes head
Portion, and according to the information of metastomium and the PIXLA confidence level for assuming the pixel in head of generation to the hypothesis head
It is evaluated in portion.
3. equipment as claimed in claim 2, which is characterized in that position generates unit and passes through fusion least energy skeleton scanning
As a result, the result of element marking and the candidate head of detection estimate trunk hypothesis.
4. equipment as claimed in claim 3, which is characterized in that before position generation unit is scanned according to least energy skeleton
Scape determines rough torso area, estimates 2D trunk direction, is gone from rough torso area unless torso based on element marking result
Element executes the modeling of 2D trunk to rough torso area, is used around the trunk upper/lower detected by element marking respectively
Shoulder/pelvis pixel determine 3D shoulder and 3D pelvis.
5. equipment as claimed in claim 4, which is characterized in that position generates unit by following operation to rough torso area
Execute the modeling of 2D trunk:
It is determined at the top of trunk based on head zone；
Mass center and leg area based on body determine trunk bottom；
By the left border and the right side that project along the inclined direction of trunk determining trunk according to trunk size constraint
Boundary,
Final torso area is determined from rough torso area.
6. equipment as claimed in claim 5, which is characterized in that position generates unit and also identifies human body from the depth image of acquisition
Complicated posture, and the candidate site being related to the complicated posture re-starts label.
7. equipment as claimed in claim 6, which is characterized in that the complexity posture includes that leg intersects and hand arm held upward.
8. equipment as claimed in claim 7, which is characterized in that position generate unit use in complementary fashion MESS result,
The result of element marking and moving region generate the hypothesis at each position of a small amount of limbs.
9. equipment as claimed in claim 8, which is characterized in that position generates unit and generates lower-left arm by executing following operation
It is assumed that:
When detecting left elbow and left wrist, lower-left arm is generated by connection left arm and left wrist (left hand) and is assumed；
When detecting lower-left arm and left wrist, lower-left arm is generated by connection lower-left arm and left wrist and is assumed；
When detecting left elbow and lower-left arm, lower-left arm is generated by connection lower-left arm and left elbow and is assumed；
Lower-left arm is generated by the PIXLA skeleton extracted from the lower-left arm of detection to assume；
From be located at upper body but be not belonging to head MESS skeleton generate lower-left arm assume；
When not finding reliable lower-left arm, lower-left arm is generated from the moving region of trunk and is assumed, wherein is detected by frame difference
Moving region；
Assume that the lower-left arm of removal overlapping in the middle assumes from multiple lower-left arms of generation；
Weighting is assumed to each lower-left arm of generation, and removes the low lower-left arm of weight and assumes, wherein according to falling into the left side
The number and these pixels of the pixel for the foreground area that lower arm assumes belong to the hypothesis of lower-left arm described in the determine the probability of lower-left arm
Weight,
Wherein, position generates unit and executes similar operations to generate the hypothesis of right arm, left leg and right leg position.
10. equipment as claimed in claim 8, which is characterized in that position generates unit also according to the relationship between different parts
Unreasonable position is removed it is assumed that with the hypothesis of selected position.
11. equipment as claimed in claim 10, which is characterized in that posture determination unit includes:
Posture categorization module, for that will assume be assembled at least one described posture by each position of human body it is assumed that simultaneously basis
At least one positional parameter that each posture assumes is general between posture classification predetermined to determine each posture hypothesis
Rate distribution；And
Posture evaluation module, at least one position binding characteristic for being assumed using each posture are assumed to assess each posture
Probability distribution between the posture classification predetermined, the probability distribution that then will be assumed with all postures after assessment
In the corresponding posture hypothesis of maximum probability value be determined as human body attitude.
12. equipment as claimed in claim 11, wherein the posture categorization module is based on machine learning algorithm, according to each
The positional parameter that posture assumes determines that each posture assumes the probability distribution between the posture classification predetermined.
13. equipment as claimed in claim 12, wherein at least one described positional parameter includes at least one in following item
It is a: intersection region size, leg between the distance between direction, arm position and metastomium of metastomium, arm position
Intersection region size between portion position.
14. equipment as claimed in claim 13, wherein at least one described position binding characteristic include in following item at least
One: depth is along axial in the two-dimentional or three-dimensional length at arm position, the two-dimentional or three-dimensional length at leg position, arm or leg
Continuity, in arm or leg depth along the prospect perpendicular to axially direction and the contrast of peripheral region, each position
The distance between coverage rate, the depth consistency at each position, adjacent parts and angle.
15. a kind of for estimating the method for human body attitude, comprising:
A the depth image including human object) is obtained；
B human object) is extracted from the depth image of acquisition and detects each candidate site and feature of human body, to the depth map
As carrying out the determining multiple skeletal points of least energy skeleton scanning and constructing least energy skeleton scanning MESS skeleton, and by each
The element marking result and depth distribution of a candidate site construct the element marking PIXLA skeleton of each candidate site；
C the portion of each human body) is generated by the result of fusion least energy skeleton scanning and the result of element marking
Position assumes；
D) by the position assume be assembled at least one posture it is assumed that according to posture interpretational criteria to each posture assume into
Human body attitude is evaluated and determined to row,
Wherein, in step B), for any candidate site, according to the element marking of pixel each in the candidate site and
Depth continuity between pixel determines the continuous skeletal point at the position, to construct the PIXLA skeleton at the position,
Wherein, step D) it include: that at least one the position binding characteristic assumed using each posture is assumed to assess each posture
Probability distribution between posture classification predetermined, by the highest in the probability distribution assumed with all postures after assessment
The corresponding posture hypothesis of probability value is determined as human body attitude.
16. method as claimed in claim 15, which is characterized in that in step C), the candidate head of self-test generates hypothesis
Head, and according to the information of metastomium and the PIXLA confidence level for assuming the pixel in head of generation to the hypothesis
It is evaluated on head.
17. the method described in claim 16, which is characterized in that in step C), pass through fusion least energy skeleton scanning
Result, the result of element marking and the candidate head of detection estimate trunk hypothesis.
18. method as claimed in claim 17, which is characterized in that in step C), scanned according to least energy skeleton
Prospect determines rough torso area, estimates 2D trunk direction, is gone from rough torso area unless trunk based on element marking result
Pixel executes the modeling of 2D trunk to rough torso area, uses the trunk upper/lower week detected by element marking respectively
The shoulder enclosed/pelvis pixel determines 3D shoulder and 3D pelvis.
19. method as claimed in claim 18, which is characterized in that in step C), by following operation to rough trunk area
Domain executes the modeling of 2D trunk:
It is determined at the top of trunk based on head zone；
Mass center and leg area based on body determine trunk bottom；
By the left border and the right side that project along the inclined direction of trunk determining trunk according to trunk size constraint
Boundary,
Final torso area is determined from rough torso area.
20. method as claimed in claim 19, which is characterized in that in step C), also identify people from the depth image of acquisition
The complicated posture of body, and the candidate site being related to the complicated posture re-starts label.
21. method as claimed in claim 20, which is characterized in that the complexity posture includes that leg intersects and hand arm held upward.
22. method as claimed in claim 21, which is characterized in that in step C), in complementary fashion using MESS result,
The result of element marking and moving region generate a small amount of hypothesis limbs.
23. method as claimed in claim 22, which is characterized in that in step C), generate lower-left by executing following operation
Arm assumes:
When detecting left elbow and left wrist, lower-left arm is generated by connection left arm and left wrist (left hand) and is assumed；
When detecting lower-left arm and left wrist, lower-left arm is generated by connection lower-left arm and left wrist and is assumed；
When detecting left elbow and lower-left arm, lower-left arm is generated by connection lower-left arm and left elbow and is assumed；
Lower-left arm is generated by the PIXLA skeleton extracted from the lower-left arm of detection to assume；
From be located at upper body but be not belonging to head MESS skeleton generate lower-left arm assume；
When not finding reliable lower-left arm, lower-left arm is generated from the moving region of trunk and is assumed, wherein is detected by frame difference
Moving region；
Assume that the lower-left arm of removal overlapping in the middle assumes from multiple lower-left arms of generation；
Weighting is assumed to each lower-left arm of generation, and removes the low lower-left arm of weight and assumes, wherein according to falling into the left side
The number and these pixels of the pixel for the foreground area that lower arm assumes belong to the hypothesis of lower-left arm described in the determine the probability of lower-left arm
Weight,
Wherein, in step C), similar operations are executed also to generate the hypothesis of right arm, left leg and right leg position.
24. method as claimed in claim 22, which is characterized in that in step C), also according to the relationship between different parts
Unreasonable position is removed it is assumed that with the hypothesis of selected position.
25. method as claimed in claim 24, which is characterized in that step D) include:
It will assume to be assembled at least one described posture by each position of human body it is assumed that and being assumed at least according to each posture
One positional parameter determines that each posture assumes the probability distribution between posture classification predetermined；
Each posture is assessed using at least one position binding characteristic that each posture assumes to assume described predetermined
Probability distribution between posture classification, then by the maximum probability value phase in the probability distribution assumed with all postures after assessment
The posture hypothesis answered is determined as human body attitude.
26. method as claimed in claim 25, which is characterized in that be based on machine learning algorithm, assumed according to each posture
Positional parameter determines that each posture assumes the probability distribution between the posture classification predetermined.
27. method as claimed in claim 26, which is characterized in that at least one described positional parameter include in following item extremely
One few: the intersection region between the distance between direction, arm position and metastomium of metastomium, arm position is big
Intersection region size small, between the position of leg.
28. method as claimed in claim 27, which is characterized in that at least one described position binding characteristic includes in following item
At least one: depth in the two-dimentional or three-dimensional length at arm position, the two-dimentional or three-dimensional length at leg position, arm or leg
Depth is along perpendicular to axially direction and the contrast of peripheral region, each position along axial continuity, arm or leg
Prospect coverage rate, the depth consistency at each position, the distance between adjacent parts and angle.
29. a kind of man-machine interactive system characterized by comprising input interface unit, attitude estimating device, display interface dress
It sets, Network Interface Unit and Applied layer interface；
Wherein, the attitude estimating device carries out human body attitude estimation for any method of 5-28 according to claim 1.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310088425.8A CN104063677B (en) | 2013-03-19 | 2013-03-19 | For estimating the device and method of human body attitude |
| KR1020140001444A KR20140114741A (en) | 2013-03-19 | 2014-01-06 | Apparatus and method for human pose estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310088425.8A CN104063677B (en) | 2013-03-19 | 2013-03-19 | For estimating the device and method of human body attitude |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104063677A CN104063677A (en) | 2014-09-24 |
| CN104063677B true CN104063677B (en) | 2019-04-30 |
Family
ID=51551384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310088425.8A Expired - Fee Related CN104063677B (en) | 2013-03-19 | 2013-03-19 | For estimating the device and method of human body attitude |
Country Status (2)
| Country | Link |
|---|---|
| KR (1) | KR20140114741A (en) |
| CN (1) | CN104063677B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701822B (en) * | 2016-01-15 | 2018-05-08 | 上海交通大学 | The method of extraction human body side leg skeleton based on two dimensional image |
| CN108885683B (en) | 2016-03-28 | 2020-05-22 | 北京市商汤科技开发有限公司 | Method and system for pose estimation |
| CN106127120B (en) * | 2016-06-16 | 2018-03-13 | 北京市商汤科技开发有限公司 | Posture estimation method and device, computer system |
| CN106600626B (en) * | 2016-11-01 | 2020-07-31 | 中国科学院计算技术研究所 | Three-dimensional human motion capture method and system |
| KR101896827B1 (en) * | 2017-02-15 | 2018-09-07 | 연세대학교 산학협력단 | Apparatus and Method for Estimating Pose of User |
| CN107492108A (en) * | 2017-08-18 | 2017-12-19 | 成都通甲优博科技有限责任公司 | A kind of skeleton line extraction algorithm, system and storage medium based on deep learning |
| KR102147930B1 (en) * | 2017-10-31 | 2020-08-25 | 에스케이텔레콤 주식회사 | Method and apparatus for recognizing pose |
| CN108154169A (en) * | 2017-12-11 | 2018-06-12 | 北京小米移动软件有限公司 | Image processing method and device |
| CN108062526B (en) * | 2017-12-15 | 2021-05-04 | 厦门美图之家科技有限公司 | Human body posture estimation method and mobile terminal |
| CN108245893B (en) | 2018-02-09 | 2021-06-29 | 腾讯科技（深圳）有限公司 | Method, device and medium for determining posture of virtual object in three-dimensional virtual environment |
| CN109377513B (en) * | 2018-09-20 | 2021-04-27 | 浙江大学 | Global three-dimensional human body posture credible estimation method for two views |
| KR102345760B1 (en) * | 2018-11-14 | 2021-12-31 | 한국과학기술원 | Target recognizing method and apparatus |
| CN109508698B (en) * | 2018-12-19 | 2023-01-10 | 中山大学 | Human behavior recognition method based on binary tree |
| CN110347255B (en) * | 2019-07-03 | 2023-06-13 | 灶灶科技有限公司 | Somatosensory interaction system based on 3D dynamic simulation technology |
| JP7350602B2 (en) * | 2019-10-07 | 2023-09-26 | 株式会社東海理化電機製作所 | Image processing device and computer program |
| CN111461017B (en) * | 2020-04-01 | 2024-01-19 | 杭州视在科技有限公司 | High-precision recognition method for kitchen work clothes after catering in urban scale |
| GB2623085A (en) * | 2022-10-04 | 2024-04-10 | Continental Automotive Gmbh | Method and system of pose estimation |
| CN116843748B (en) * | 2023-09-01 | 2023-11-24 | 上海仙工智能科技有限公司 | Remote two-dimensional code and object space pose acquisition method and system thereof |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101982836A (en) * | 2010-10-14 | 2011-03-02 | 西北工业大学 | Mark point identification initializing method based on principal components analysis (PCA) in motion capture system |
| CN102609684A (en) * | 2012-01-16 | 2012-07-25 | 宁波江丰生物信息技术有限公司 | Human body posture detection method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8503720B2 (en) * | 2009-05-01 | 2013-08-06 | Microsoft Corporation | Human body pose estimation |
-
2013
- 2013-03-19 CN CN201310088425.8A patent/CN104063677B/en not_active Expired - Fee Related
-
2014
- 2014-01-06 KR KR1020140001444A patent/KR20140114741A/en not_active Application Discontinuation
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101982836A (en) * | 2010-10-14 | 2011-03-02 | 西北工业大学 | Mark point identification initializing method based on principal components analysis (PCA) in motion capture system |
| CN102609684A (en) * | 2012-01-16 | 2012-07-25 | 宁波江丰生物信息技术有限公司 | Human body posture detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| Simultaneous 3d human motion tracking and voxel reconstruction;Junchi Yan 等;《Optical Engineering》;20100930;第49卷(第9期);1-10 |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20140114741A (en) | 2014-09-29 |
| CN104063677A (en) | 2014-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104063677B (en) | For estimating the device and method of human body attitude | |
| US10417775B2 (en) | Method for implementing human skeleton tracking system based on depth data | |
| Kamal et al. | A hybrid feature extraction approach for human detection, tracking and activity recognition using depth sensors | |
| KR101588254B1 (en) | Improvements in or relating to three dimensional close interactions | |
| CN103718175B (en) | Detect equipment, method and the medium of subject poses | |
| CN102609683B (en) | Automatic labeling method for human joint based on monocular video | |
| Liu et al. | Markerless motion capture of multiple characters using multiview image segmentation | |
| CN107357427A (en) | A kind of gesture identification control method for virtual reality device | |
| Wu et al. | Hand pose estimation in object-interaction based on deep learning for virtual reality applications | |
| Ning et al. | People tracking based on motion model and motion constraints with automatic initialization | |
| CN104850219A (en) | Equipment and method for estimating posture of human body attached with object | |
| Kim et al. | Real-time dance evaluation by markerless human pose estimation | |
| She et al. | A real-time hand gesture recognition approach based on motion features of feature points | |
| CN103577792A (en) | Device and method for estimating body posture | |
| Shen et al. | Exemplar-based human action pose correction | |
| Iwasawa et al. | Real-time, 3D estimation of human body postures from trinocular images | |
| CN110286749A (en) | Hand gesture estimation and method for tracing based on depth data | |
| Hu et al. | Human interaction recognition using spatial-temporal salient feature | |
| Guo et al. | Chinese traffic police gesture recognition in complex scene | |
| CN102156994B (en) | Joint positioning method for single-view unmarked human motion tracking | |
| CN103810480B (en) | Method for detecting gesture based on RGB-D image | |
| Iwasawa et al. | Human body postures from trinocular camera images | |
| CN108108648A (en) | A kind of new gesture recognition system device and method | |
| Półrola et al. | Real-time hand pose estimation using classifiers | |
| Nie et al. | A child caring robot for the dangerous behavior detection based on the object recognition and human action recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190430 |