CN104700433B - A kind of real-time body's whole body body motion capture method of view-based access control model and system thereof - Google Patents

Abstract

The invention discloses a kind of real-time body's whole body body motion capture method and system thereof of view-based access control model, comprising: multiple-camera synchronous acquisition unit, for gathering three dimensional spatial scene video flowing interested; Three-dimensional body and three-dimensional motion stream real-time reconstruction unit, for the three-dimensional body of real-time reconstruction human object and the three-dimensional motion stream information of whole body body; Whole Body body Attitude estimation and tracking cell, for estimating in real time 3 D human body whole body body athletic posture and follow the tracks of; Virtual role by exercise data Real Time Drive display unit, for the skeleton pose data Real Time Drive virtual role captured showing on the display apparatus.Real-time body's all-around exercises capture system of view-based access control model disclosed in this invention and method are by non-contacting computer vision methods, wearing marking equipment that need not be extra, can achieve Whole Body skeleton pose data Real-time Obtaining and real-time to virtual role drive, simple and convenient relative to general motion capture technical equipment.

Description

A kind of real-time body's whole body body motion capture method of view-based access control model and system thereof

Technical field

The present invention relates to real-time body's whole body body motion capture method and the system thereof in electronic informatics science field, particularly view-based access control model.

Background technology

Human motion analysis is one of popular problem of current computer graphics and computer vision research.Its reason is that human body movement data is in many neck `e amusements and field of human-computer interaction.Such as, in many video-games or computer animation kind, human body movement data is often used to drive virtual role, makes the action of virtual role naturally true to nature; To the Tracking Recognition of human motion as gesture, the action such as human posture, gesture can be converted into certain computer command, thus as a kind of mode of man-machine interaction; It is in medical treatment and physical culture that 3rd class is mainly applied, such as gait analysis is a kind of method to observation and analysis such as human action, health mechanics function and muscle activity abilities in therapeutic treatment, and its core is systematic observation and the assessment such as joint position, posture to the object of observation; In athletic training, motion analysis technique also usually with help train with sportsman's analyzing and training move in situation, in order to improve sports level.

Can be analyzed human motion by modes such as eye-observation, shooting or photographies.But, the human body movement data of accurate quantitative analysis be obtained, only have and adopt special motion capture equipment.Current, motion capture equipment is mainly divided three classes, and the first kind is optical sensing capture device, and namely captured the movement locus of the monumented point on wearable device by shooting, then analysis meter calculates the joint motions data of human body; Equations of The Second Kind is mechanical capture device, and it is by directly measuring the data such as human synovial angle, generally needs the measured to dress the mechanical measurement device being similar to skeleton structure; 3rd class is electromagnetic sensing capture device, namely adopts the data such as electromagnetic sensor measurements and calculations human synovial angle, also needs the electromagnetic transmission of wear special and receiving equipment or mark.Above-mentioned motion capture equipment all needs human object to dress upper complicated sensor flag equipment, and this sometimes can produce restriction or impact to the motion of human object; And these equipment needs expensive supporting computation and analysis equipment, and outfit of equipment price is very expensive.This much has the general user of motion capture demand unaffordable.This impels people to find more cheaply and easily motion capture technology.

In recent years, in computer vision and field of Computer Graphics, the movement capturing technology without mark becomes study hotspot.Namely adopt the non-contacting method based on image without mark is motion-captured, from image, directly calculate the attitude motion data of human object, thus no longer need the necessary wearable device of conventional motion capture technology, reach quick and easy object.But, directly recover below 3 D human body attitude existence difficulty from image: 1) human body three-dimensional attitude motion is high-dimensional, comprises more than at least 30 degree of freedom; And view data is two-dimentional, obtain observation data by image and often comprise noise because the factors such as illumination, motion blur or video camera slight jitter affect, and there is complicated nonlinear relationship often in itself and human body three-dimensional attitude; 2) feature dimension of human body is different, and attitude motion is unconfined, and be difficult to set up dynamic model or predict accurately it accurately, this all brings very large challenge to Attitude estimation and Human Modeling; 3) human body is in motion process, and each limbs usually can exist and block or close contact, thisly in two dimensional image, often causes the unusual of observation data from blocking, thus likely causes the singular value of Attitude estimation.In order to make problem simple, current nearly all method can to the motion of human object, catch in addition some restriction of the factor such as environment and video camera.Such as can require to catch in environment and only have single human body, only limit is in indoor environment, and background is simple, illumination is controlled.

Compared with single camera method, the method for employing multiple-camera better can process the problems such as oneself blocks, singular value, can obtain higher precision and stability.Although there is multiple multiple-camera method, seldom have method can obtain the performance matched based on the motion capture technology identified with tradition, all will be poor in its precision in human body tracking, stability and real-time often.Therefore, research can obtain more high precision, have having important Theory and applications without mark motion capture method and being worth of better stability and real-time.

Summary of the invention

The invention discloses a kind of real-time body's whole body body motion capture method and system thereof of view-based access control model, the method can by non-contacting computer vision methods, wearing marking equipment that need not be extra, can realize Whole Body skeleton pose data Real-time Obtaining and to drive virtual role in real time.

Technical scheme of the present invention is:

Real-time body's whole body body motion capture method of view-based access control model, is characterized in that, comprise the following steps:

S1: multiple-camera sync pulse jamming obtains human object video image;

S2: reconstruction of three-dimensional body data in many cameras vedio data;

By processing the multiple-camera image obtained, reconstruct human body three-dimensional point cloud and three-dimensional light stream; Reconstruction of three-dimensional point cloud takes the three-dimensional rebuilding method based on profile in computer vision; Reconstruction of three-dimensional light stream adopts the method based on two-dimentional light stream reconstruction of three-dimensional optical flow field;

S3: motion-captured initialization; Three-dimensional (3 D) manikin is matched the human object three-dimensional body data that initial time is corresponding, the size of three-dimensional (3 D) manikin is conformed to attitude with the size of attitude with initial time human object; Specifically comprise two steps below:

S3.1: three-dimensional body is rebuild and three-dimensional (3 D) manikin: take the three-dimensional rebuilding method based on profile in computer vision, reconstruct human object point cloud or body blocks of data; Design or obtain a kind of size, three-dimensional virtual human body Model that attitude is adjustable by third party's graphics software such as Maya;

S3.2: adopt a kind of attitude based on model and size estimation method, matches manikin accurately in the 3 D human body morphological data that reconstructs, virtual human model is presented with identical attitude and close feature dimension;

S4: three-dimensional framework Attitude estimation and tracking; Specifically comprise three steps below:

S4.1: the Attitude estimation based on a kind of stochastic search methods: integrated use previous moment Attitude estimation result, manikin and motion model, adopt a kind of stochastic search methods generating sampling, from the approximation probability distributed model of the dynamic attitude of time dependent human body the Attitude estimation of current time out;

S4.2: the attitude parameter correction based on local optimum: to the result by Attitude estimation in step S4.1, adopts a kind of local optimization methods, revises local attitude parameter;

S4.3: attitude data smoothing processing: to the smoothing process of the attitude data of certain hour window, revise the attitude parameter of exception, makes the attitude motion data obtained without jump or exception of trembling;

S5: be synchronized with the movement by the attitude motion data-driven virtual role obtained;

Acquisition attitude motion data are directly carried out role's driving by third party software by plug-in unit, reaches the effect of virtual role and human object real-time synchronization.

In the present invention, the concrete steps of step 3.2 are as follows:

S3.2.1, the method adopting graduation to optimize, successively mate upper body, left and right lower limb, left and right upper limbs, head, detailed process is as follows: first estimate the position of trunk and direction parameter, comprise three positions and three rotation parameters; Then head, thigh and upper arm joint parameter are estimated; Finally the joint parameter of shank and underarm is estimated;

Wherein to the position (x of trunk ₀, y ₀, z ₀) and direction parameter estimate to adopt with the following method:

A1, the method for the human body point cloud rebuild by section is detected to the point obtaining metastomium corresponding and converge conjunction; Wherein dicing method refers to and obtains three-dimensional point cloud perpendicular to the some set on trunk plane or perpendicular with certain sampling interval;

The point that b1, calculating are found converges the centre of gravity place of conjunction

$\stackrel{\‾}{x}=\frac{\Σx_i}{N}$

$\stackrel{\‾}{y}=\frac{\Σy_i}{N}$

$\stackrel{\‾}{z}=\frac{\Σz_i}{N}$

C1, to establish for (x ₀, y ₀, z ₀) initial value, carry out position (x ₀, y ₀, z ₀) and orientation the size estimation of parameter and metastomium, concrete steps are: first keep (x ₀, y ₀, z ₀) with initial value and dimensional parameters constant, adopt and make residual error minimum based on particle group optimizing method, obtain then keep constant with dimensional parameters, proceed residual minimization estimate obtain (x ' ₁, y ' ₁, z ' ₁); Keep afterwards (x ' ₁, y ' ₁, z ' ₁) constant, estimate to obtain new dimensional parameters; And so forth, until each parameter values is stablized;

Time wherein to human body semi-match, adopt the method for iteration optimization, be specially:

A) fixed model dimensional parameters, if the coupling attitude result of previous stage is as initial attitude, stochastic sampling part model epidermis point is as match point; Suppose that the random model points set obtained is M _n={ p _n,i| i=1,2 ..., m _n, wherein p _n,ifor the probabilistic model point of body part n, m _nfor the sampled point number of body part n;

B) to any probabilistic model point, find the reconstruction of three-dimensional point of corresponding minimum distance, the distance sum of answering using all-pair is as residual error; Namely following equation is solved:

${\hat{X}}_n=\underset{X_n}{\arg \min }\underset{\∀p_{n,i}\∈M_n}{\Σ}\left|\right|p_{n,i}-y_{n,i}^{*}|{|}^2$

Wherein p is corresponded to for what find _n,inearest Three-dimensional Gravity lay foundations;

C) adopt and carry out iteration optimization based on the full search algorithm of particle group optimizing, make residual values level off to minimum stationary value; For making optimization to restrain, introduce following weight constraints:

${\mathrm{\λ}}_{n,i}=\left\{\begin{array}{cc}1& y_{n,i}^{*}\↔p_{n,i}\\ \mathrm{\λ}\left(k\right)& y_{n,i}^{*}\↔\left\{p_{n,j}\right|j=1,...,k\}\end{array}\right.$

The weighting function that in above formula, λ (k) is is variable with k (k > 1), k is larger, and weighted value is also larger, simply can be set to linear function, wherein represent corresponding relation, the problems referred to above become:

${\hat{X}}_n=\underset{X_n}{\arg \min }\underset{\∀p_{n,i}\∈M_n}{\Σ}\left|\right|p_{n,i}-y_{n,i}^{*}|{|}^2$

D) fixing previous step obtains attitude parameter, and using dimensional parameters as variable, iteration optimization obtains new dimensional parameters;

E) so iteration, until each parameter no longer changes, obtains whole human body attitude and dimensional parameters;

S3.2.2, local correction is carried out to whole human body attitude parameter, finally obtain size and attitude parameter accurately; Described local pose correction refers to: after obtaining whole human body attitude and dimensional parameters by S3.2.1, keep upper body trunk position and direction parameter and other joint parameters constant, with its existing numerical value, initial value is done to left thigh and 6 rotation parameters corresponding to shank, using entirety as variable, by above-mentioned steps a)-e) method for parameter estimation reappraises, and the joint parameter of same method to other three limbs right lower extremities, left upper extremity and right upper extremities is revised.

In the present invention, step S4.1 specifically comprises the following steps:

A) hypothesis is at moment t-1 gesture distribution p (s _t-1| z _1:t-1), p (s _t-1| z _1:t-1) represent the attitude s in t-1 moment _t-1posterior probability distribution function, it can by the particle assembly of not Weight carry out approximate representation, being sampled by the gesture distribution of t-1 obtains the expectation attitude sampling particle assembly of t by multidimensional Gauss dynamic model namely wherein the standard deviation vector of Σ is p;

B) initialization t sampling particle assembly: right $i=1,...,N,x_0^i\←s_t^{\′i},p_i\←x_0^i,f_i\←obj\left(x_0^i\right),$ wherein obj (x) represents objective function, p _ifor the location point of desired value optimum in all historical junctures of particle i, namely meet nb (i, K) represents neighborhood optimal function, the historical juncture location point of optimum in the set of particle centered by i and K-1 particle around thereof can be found out by it;

C) iterative search is carried out M time in the steps below: i) establish g=0; Ii) to i=1 ..., N, if then to i=1 ..., N, upgrades neighborhood optimal value p _n,i← nb (i, K); Iv) right wherein u=0.5 (p _i+ p _n,i), Σ _bfor diagonal covariance matrix, corresponding standard deviation vector is | p _i-p _n,i|;

D) neighborhood optimal particle value set as new sampling particle position, each particle gives new value according to likelihood function, namely obtains Weight particle assembly ${\left\{(s_t^{*i},{\mathrm{\π}}_t^{*i})\right\}}_{i=1}^N,$ Wherein $s_t^{*i}=p_{n,i},{\mathrm{\π}}_t^{*i}\∝p\left(z_t\right|s_t^{*i});$ Wherein for observation function and likelihood function;

E) two class particle assemblies merging are obtained the particle assembly that number is 2N: ${\left\{(s_t^j,\frac{1}{2N})\right\}}_{j=1}^{2N}={\left\{(s_t^{\′i},\frac{1}{2N})\right\}}_{i=1}^N\∪{\left\{(s_t^{*i},{\mathrm{\π}}_t^{*i})\right\}}_{i=1}^N;$

F) correction factor is calculated ${\mathrm{\λ}}_t^j=f\left(s_t^j\right)/g_t\left(s_t^j\right),1\≤j\≤2N;$

G) final weight is calculated and regularization makes

H) calculate as final carriage estimated result;

I) to set carry out resampling and obtain new set for subsequent time Attitude estimation is prepared.

In the present invention, to local attitude parameter optimization correction in step S4.2, its be to the limb part of human body comprise upper body trunk, Jing head, left and right upper arm, left and right forearm, left and right thigh, left and right shank attitude parameter selectively revise; To human body part k (human body is divided into 10 parts and k=1 ..., 10), supposing that one group of Three-dimensional Gravity that it comprises under the attitude recovered by step S4.1 is laid foundations with the corresponding set of manikin point is then obtain following optimization problem:

${\arg }_{{\mathrm{\θ}}_j^{*}}\min \frac{1}{2}\underset{k\∈B_j}{\Σ}\underset{i}{\Σ}\left|\right|p_i^k\left({\mathrm{\θ}}_j^{*}\right)-v_i^k|{|}^2$

Wherein for corresponding to the attitude parameter of human body part k, expression is subject to the human body parts of impact;

Solve above-mentioned optimization problem and adopt LM gradient optimal method.

The invention provides a kind of real-time body's whole body body motion capture system of view-based access control model, this system comprises:

A) multiple-camera video frame sync filming apparatus: obtain the video image of three dimensions interested and human object for sync pulse jamming and be real-time transmitted to calculation element;

B) three-dimensional body is rebuild and is calculated and display device: for processing the multiple-camera video image obtained in real time, therefrom real-time reconstruction goes out human body three-dimensional body, and carries out color rendering; Show on the display apparatus to the three-dimensional body rebuild, also can show with dummy model object in camera scene simultaneously;

C) 3 D human body attitude is estimated and tracking calculation element in real time: for calculating to recover 3 D human body whole body body attitude motion information in real time from the three-dimensional body view data of rebuilding;

D) exercise data drives virtual role to be synchronized with the movement display device: for the whole body body skeleton pose data Real Time Drive virtual three-dimensional role recovered, make it to do the same motion with human object.

In this system, described multiple-camera video frame sync filming apparatus comprises the video camera that number is no less than 9, each video camera is centered around length, width and height with suitable pose angle installation and respectively can reaches around the solid space of 2-3 rice, each video camera all can to this solid space complete imaging, and can sync pulse jamming; Multiple-camera video frame sync filming apparatus also comprises the video image acquisitions such as supporting data line, image card and transmission equipment and the utility appliance for camera calibration;

In this system, three-dimensional body is rebuild to calculate and is comprised a 1# computing machine and peripheral apparatus thereof supporting GPU to calculate with display device; The multiple PCI image pick-up card work of this 1# computing power Supporting connectivity.

In this system, described 3 D human body attitude is estimated to comprise a 2# computing machine and peripheral apparatus thereof supporting GPU to calculate with tracking calculation element in real time.

In this system, exercise data drives the virtual role display device that is synchronized with the movement to comprise a display be connected with 2# computing machine.

Compared with prior art, the beneficial effect that the present invention has is:

Real-time body's all-around exercises capture system of view-based access control model disclosed in this invention and method are by non-contacting computer vision methods, wearing marking equipment that need not be extra, can achieve Whole Body skeleton pose data Real-time Obtaining and real-time to virtual role drive, simple and convenient relative to general motion capture technical equipment.

Accompanying drawing explanation

The general flow chart of real-time body's whole body body motion capture method of a kind of view-based access control model of Fig. 1 the present invention;

The schematic diagram of Fig. 2 Whole Body body skeleton pattern and movement definition;

Fig. 3 multiple-camera synchronous shooting device exemplary plot;

Fig. 4 three dimension data reconstruct exemplary plot;

The process flow diagram of Fig. 5 three-dimensional framework Attitude estimation and tracking.

Embodiment

Below in conjunction with accompanying drawing, specific embodiments of the invention are described in detail, but are not construed as limiting the invention.

Motion-captured mainly towards under single human object, indoor environment of Whole Body body motion capture system of the present invention and method.Fig. 1 gives the five steps composition of motion capture method of the present invention: multiple-camera sync pulse jamming obtains reconstruction of three-dimensional body data (three-dimensional point cloud, three-dimensional light stream) (102), motion-captured initialization (103), three-dimensional framework Attitude estimation and tracking (104) in human object video image (101), many cameras vedio data, is synchronized with the movement (105) by the attitude motion data-driven virtual role obtained.

Whole Body body movement definition is the skeleton pose motion of 3 D human body in human body motion process, and skeleton pose refers to one group of parameter that can describe position of human body and major joint attitude.As Fig. 2 the example given, comprise 15 joints in this skeleton pattern, the attitude parameter of each joint and correspondence is: trunk main joint (207) comprises 3 global position parameters (in order to describe human body position in three dimensions) and 3 overall rotation parameters (in order to describe human body in three-dimensional angle), trunk back of the body joint (203) comprises 3 rotation parameters (in order to describe the angular relationship relative to joint 207), cervical vertebra joint (201) comprises 3 rotation parameters (in order to describe the angular relationship relative to joint 203), shoulder joint (202 on left and right, 210) 3 rotation parameters (respectively in order to describe the angular relationship relative to joint 203) are respectively comprised, left and right elbow joint (204, 211) 3 rotation parameters are respectively comprised (in order to describe respectively relative to joint 202, the angular relationship of 210), right-hand man's wrist joint (205, 212) 3 rotation parameters are comprised (in order to describe respectively relative to joint 204, the angular relationship of 211), the large leg joint (206 in left and right, 213) 3 rotation parameters (in order to describe respectively relative to the angular relationship in joint 207) are respectively comprised, left and right knee joint (208, 214) 3 rotation parameters are respectively comprised (in order to describe respectively relative to joint 206, the angular relationship of 213), ankle joint, left and right (209, 215) 3 rotation parameters are respectively comprised (in order to describe respectively relative to joint 208, the angular relationship of 214).

Multiple-camera sync pulse jamming obtains human object video image (101): completed by multiple-camera audio video synchronization filming apparatus.Fig. 3 gives the example of multiple-camera synchronous shooting device.Mainly comprise: more than 9 colour TV cameras (301), orientation calibration plate (303), display (304), computing machine (305) and image capture devices are as data line, capture card (306) etc.Multiple-camera is fixedly mounted on around on the correct position in stage space, and aims at stage space, to ensure that each camera views can the aerial image of complete covering stage with proper angle.Stage space refers to the three dimensions of equal about 3 meters of length and width, and human object (302) is taken exercises in stage space.Each video camera is connected with the image pick-up card (306) in computing machine (305) by the transmission equipment such as data line, switch, and image acquisition card request bandwidth is enough, and computing machine requires to support abundant image pick-up card.For the ease of requiring subsequent treatment, require ambient lighting enough and stablize, background environment is simple, without large dynamic object.

Many camera shooting and videos view data reconstruction of three-dimensional body data (three-dimensional point cloud, three-dimensional light stream) (102): by processing the multiple-camera image obtained, reconstruct human body three-dimensional point cloud and three-dimensional light stream.Reconstruction of three-dimensional point cloud can take the three-dimensional rebuilding method (prior art) based on profile in computer vision.Reconstruction of three-dimensional light stream can adopt the method (prior art) based on two-dimentional light stream reconstruction of three-dimensional optical flow field.In the present invention, three-dimensional point cloud refers to the outmost epidermis three-dimensional point of human body, and in body, point does not participate in playing up and subsequent calculations.Three-dimensional light stream refers to the description of its direction of motion in the time at lower a moment of three-dimensional point in some three-dimensional point cloud and motion size, three-dimensional arrow is adopted to represent, wherein arrow starting point is the reconstruction of three-dimensional point of current time, arrow direction indication represents the direction that three-dimensional point is moved, and arrow length represents motion size.Not all three-dimensional point all has effective three-dimensional light stream, needs to carry out rejecting invalid three-dimensional light stream.By setting the constraint of simple size threshold value, the three-dimensional light stream of partial invalidity just can be rejected.Fig. 4 gives the example of three-dimensional point cloud and three-dimensional light stream reconstruction.

Motion-captured initialization (103): initialized object is manikin in order to obtain coupling and initial 3 d pose, provides initial value and model for following the tracks of.Mainly comprise two steps: a) three-dimensional body is rebuild and manikin: take the three-dimensional rebuilding method based on profile in computer vision, reconstruct the data such as human object point cloud or body block; Design or obtain a kind of size, three-dimensional virtual human body Model that attitude is adjustable by third party's graphics software; B) adopt a kind of attitude based on model and size estimation method, manikin is matched accurately in the 3 D human body morphological data that reconstructs, virtual human model is presented with identical attitude and close feature dimension.Wherein in technical scheme, there is concrete introduction based on the attitude of model and the concrete steps of size estimation method, repeat no more here.In order to simplify, human object must be asked to stand with specific initial attitude.

Three-dimensional framework Attitude estimation and tracking (104): Fig. 5 give three-dimensional framework Attitude estimation and tracking flow process.It mainly comprises following three steps:

1) based on a kind of Attitude estimation of stochastic search methods: integrated use previous moment Attitude estimation result, manikin and motion model, adopt a kind of stochastic search methods generating sampling, from the approximation probability distributed model of the dynamic attitude of time dependent human body the Attitude estimation of current time out; This stochastic search methods is applicable to parallelization and calculates; Objective function and likelihood function set up according to the corresponding relation between three-dimensional reconstruction data and manikin number of sampling points certificate.-the log of objective function is likelihood function.Likelihood function comprise based on the shortest observed reading of the spacing that reconstruction of three-dimensional point is corresponding with manikin and corresponding to manikin based on reconstruction of three-dimensional point color between the similar observed reading etc. of color distribution.Three-dimensional reconstruction data comprise three-dimensional body reconstruction point cloud and three-dimensional light stream.Specifically comprise the following steps:

A) particle prediction (501): suppose at moment t-1 gesture distribution p (s _t-1| z _1:t-1) can by the particle assembly of not Weight carry out approximate representation, being sampled by the gesture distribution of t-1 obtains the expectation attitude sampling particle assembly of t by multidimensional Gauss dynamic model namely wherein the standard deviation vector of Σ is p.

B) particle assembly initialization (502): initialization t sampling particle assembly: right p _n,i← nb (i, K); Wherein obj (x) represents objective function, p _ifor the location point of desired value optimum in all historical junctures of particle i, namely meet nb (i, K) represents neighborhood optimal function, the historical juncture location point of optimum in the set of particle centered by i and K-1 particle around thereof can be found out by it.

C) Optimizing Search (503): carry out iterative search M time in the steps below: i) establish g=0; Ii) to i=1 ..., N, if then to i=1 ..., N, upgrades neighborhood optimal value p _n,i← nb (i, K); Iv) right wherein u=0.5 (p _i+ p _n,i), Σ _bfor diagonal covariance matrix, corresponding standard deviation vector is | p _i-p _n,i|.

D) neighborhood optimal particle is selected to upgrade (504) with weight: neighborhood optimal particle value set as new sampling particle position, each particle gives new value according to likelihood function, namely obtains Weight particle assembly wherein wherein for observation function (or likelihood function).

E) particle merges and weight correction (505): two class particle assemblies are merged and obtains the particle assembly that number is 2N: ${\left\{(s_t^j,\frac{1}{2N})\right\}}_{j=1}^{2N}={\left\{(s_t^{\′i},\frac{1}{2N})\right\}}_{i=1}^N\∪{\left\{(s_t^{*i},{\mathrm{\π}}_t^{*i})\right\}}_{i=1}^N;$

F) weight correction: calculate correction factor calculate final weight and regularization makes $\Σ{\mathrm{\π}}_t^j=1;$

G) random search estimated result (506) is obtained: calculate as final carriage estimated result;

H) to set carry out resampling and obtain new set for subsequent time Attitude estimation is prepared.

2) based on the attitude parameter correction (507) of local optimum: to by step 1) result of Attitude estimation, adopt a kind of local optimization methods, local attitude parameter is revised; Mainly to local attitude parameter optimization correction, its main thought be to the limb part of human body comprise upper body trunk, Jing head, left and right upper arm, left and right forearm, left and right thigh, left and right shank attitude parameter selectively revise.To human body part k (human body is divided into 10 parts and k=1 ..., 10), suppose that it is by step 2) one group of Three-dimensional Gravity comprising under the attitude recovered lays foundations with the corresponding set of manikin point and is then obtain following optimization problem:

${\arg }_{{\mathrm{\θ}}_j^{*}}\min \frac{1}{2}\underset{k\∈B_j}{\Σ}\underset{i}{\Σ}\left|\right|p_i^k\left({\mathrm{\θ}}_j^{*}\right)-v_i^k|{|}^2$

Wherein for corresponding to the attitude parameter of human body part k, expression is subject to the human body parts of impact.Solve above-mentioned optimization problem and can adopt LM constant gradient optimized algorithm.

3) attitude data smoothing processing (508): to the smoothing process of the attitude data of certain hour window, the attitude parameter of exception is revised, make obtain attitude data without jump or tremble wait exception.

Be synchronized with the movement (105) by the attitude motion data-driven virtual role obtained: the skeleton pose data of acquisition can be converted into standard format as exercise datas such as BVH.Such as, characteristics of human body's animation moving data format that BVH mono-kind is general, is supported by various animation softs popular now widely, and packet contains bone and the limbs joint spin data of role.The present invention supports the multiple popular moving data format such as BVH.By acquisition exercise data by plug-in unit directly by third party software as MotionBuilder carries out role's driving, the effect of virtual role and human object real-time synchronization can be reached.

Although the above is the complete description to specific embodiments of the present invention, various amendment, variant and alternative can be taked.These equivalents and alternative are included within the scope of the invention.Therefore, scope of the present invention should not be limited to described embodiment, but should be defined by the appended claims.

Claims (3)

1. real-time body's whole body body motion capture method of view-based access control model, is characterized in that, comprise the following steps:

S1: multiple-camera sync pulse jamming obtains human object video image;

S2: reconstruction of three-dimensional body data in many cameras vedio data;

By processing the multiple-camera image obtained, reconstruct human body three-dimensional point cloud and three-dimensional light stream; Reconstruction of three-dimensional point cloud takes the three-dimensional rebuilding method based on profile in computer vision; Reconstruction of three-dimensional light stream adopts the method based on two-dimentional light stream reconstruction of three-dimensional optical flow field;

S3: motion-captured initialization; Three-dimensional (3 D) manikin is matched the human object three-dimensional body data that initial time is corresponding, the size of three-dimensional (3 D) manikin is conformed to attitude with the size of attitude with initial time human object; Specifically comprise two steps below:

S3.1: three-dimensional body is rebuild and three-dimensional (3 D) manikin: take the three-dimensional rebuilding method based on profile in computer vision, reconstruct human object point cloud or body blocks of data; Design or obtain a kind of size, three-dimensional virtual human body Model that attitude is adjustable by third party's graphics software Maya;

S3.2: adopt a kind of attitude based on model and size estimation method, matches manikin accurately in the 3 D human body morphological data that reconstructs, virtual human model is presented with identical attitude and close feature dimension;

S3.2.1, the method adopting graduation to optimize, successively mate upper body, left and right lower limb, left and right upper limbs, head, detailed process is as follows: first estimate the position of trunk and direction parameter, comprise three positions and three rotation parameters; Then head, thigh and upper arm joint parameter are estimated; Finally the joint parameter of shank and underarm is estimated;

Wherein to the position (x of trunk ₀, y ₀, z ₀) and direction parameter estimate to adopt with the following method:

A1, the method for the human body point cloud rebuild by section is detected to the point obtaining metastomium corresponding and converge conjunction; Wherein dicing method refers to and obtains three-dimensional point cloud perpendicular to the some set on trunk plane or perpendicular with certain sampling interval;

The point that b1, calculating are found converges the centre of gravity place of conjunction

$\stackrel{\‾}{x}=\frac{{\mathrm{\Σx}}_i}{N}$

$\stackrel{\‾}{y}=\frac{{\mathrm{\Σy}}_i}{N}$

$\stackrel{\‾}{z}=\frac{{\mathrm{\Σz}}_i}{N}$

C1, to establish for (x ₀, y ₀, z ₀) initial value, carry out position (x ₀, y ₀, z ₀) and orientation the size estimation of parameter and metastomium, concrete steps are: first keep (x ₀, y ₀, z ₀) with initial value and dimensional parameters constant, adopt and make residual error minimum based on particle group optimizing method, obtain then keep constant with dimensional parameters, proceed residual minimization estimate obtain (x ' ₁, y ' ₁, z ' ₁); Keep afterwards (x ' ₁, y ' ₁, z ' ₁) constant, estimate to obtain new dimensional parameters; And so forth, until each parameter values is stablized;

Time wherein to human body semi-match, adopt the method for iteration optimization, be specially:

A) fixed model dimensional parameters, if the coupling attitude result of previous stage is as initial attitude, stochastic sampling part model epidermis point is as match point; Suppose that the random model points set obtained is M _n={ p _n,i| i=1,2 ..., m _n, wherein p _n,ifor the probabilistic model point of body part n, m _nfor the sampled point number of body part n;

B) to any probabilistic model point, find the reconstruction of three-dimensional point of corresponding minimum distance, the distance sum of answering using all-pair is as residual error; Namely following equation is solved:

${\hat{X}}_n=\underset{X_n}{\arg \min }\underset{\∀p_{n,i}\∈M_n}{\Σ}\left|\right|p_{n,i}-y_{n,i}^{*}|{|}^2$

Wherein p is corresponded to for what find _n,inearest Three-dimensional Gravity lay foundations;

C) adopt and carry out iteration optimization based on the full search algorithm of particle group optimizing, make residual values level off to minimum stationary value; For making optimization to restrain, introduce following weight constraints:

${\mathrm{\λ}}_{n,i}=\left\{\begin{array}{cc}1& y_{n,i}^{*}\↔p_{n,i}\\ \mathrm{\λ}\left(k\right)& y_{n,i}^{*}\↔\left\{p_{n,j}\right|j=1,...,k\}\end{array}\right.$

The weighting function that in above formula, λ (k) is is variable with k (k > 1), k is larger, and weighted value is also larger, simply can be set to linear function, wherein represent corresponding relation, the problems referred to above become:

${\hat{X}}_n=\underset{X_n}{\arg \min }\underset{\∀p_{n,i}\∈M_n}{\mathrm{\Σ}}{\mathrm{\λ}}_{n,i}\left|\right|p_{n,i}-y_{n,i}^{*}|{|}^2$

D) fixing previous step obtains attitude parameter, and using dimensional parameters as variable, iteration optimization obtains new dimensional parameters;

E) so iteration, until each parameter no longer changes, obtains whole human body attitude and dimensional parameters;

S3.2.2, local correction is carried out to whole human body attitude parameter, finally obtain size and attitude parameter accurately; Described local pose correction refers to: after obtaining whole human body attitude and dimensional parameters by S3.2.1, keep upper body trunk position and direction parameter and other joint parameters constant, with its existing numerical value, initial value is done to left thigh and 6 rotation parameters corresponding to shank, using entirety as variable, by above-mentioned steps a)-e) method for parameter estimation reappraises, and the joint parameter of same method to other three limbs right lower extremities, left upper extremity and right upper extremities is revised;

S4: three-dimensional framework Attitude estimation and tracking; Specifically comprise three steps below:

S4.1: the Attitude estimation based on a kind of stochastic search methods: integrated use previous moment Attitude estimation result, manikin and motion model, adopt a kind of stochastic search methods generating sampling, from the approximation probability distributed model of the dynamic attitude of time dependent human body the Attitude estimation of current time out;

S4.2: the attitude parameter correction based on local optimum: to the result by Attitude estimation in step S4.1, adopts a kind of local optimization methods, revises local attitude parameter;

S4.3: attitude data smoothing processing: to the smoothing process of the attitude data of certain hour window, revise the attitude parameter of exception, makes the attitude motion data obtained without jump or exception of trembling;

S5: be synchronized with the movement by the attitude motion data-driven virtual role obtained;

Acquisition attitude motion data are directly carried out role's driving by third party software by plug-in unit, reaches the effect of virtual role and human object real-time synchronization.

2. real-time body's whole body body motion capture method of view-based access control model according to claim 1, it is characterized in that, step S4.1 specifically comprises the following steps:

A) hypothesis is at moment t-1 gesture distribution p (s _t-1| z _1:t-1), p (s _t-1| z _1:t-1) represent the attitude s in t-1 moment _t-1posterior probability distribution function, it can by the particle assembly of not Weight carry out approximate representation, being sampled by the gesture distribution of t-1 obtains the expectation attitude sampling particle assembly of t by multidimensional Gauss dynamic model namely wherein the standard deviation vector of Σ is p;

B) initialization t sampling particle assembly: to i=1 ..., N, p _n,i← nb (i, K); Wherein obj (x) represents objective function, p _ifor the location point of desired value optimum in all historical junctures of particle i, namely meet nb (i, K) represents neighborhood optimal function, the historical juncture location point of optimum in the set of particle centered by i and K-1 particle around thereof can be found out by it;

C) iterative search is carried out M time in the steps below: i) establish g=0; Ii) to i=1 ..., N, if then to i=1 ..., N, upgrades neighborhood optimal value p _n,i← nb (i, K); Iv) to i=1 ..., N, wherein u=0.5 (p _i+ p _n,i), Σ _bfor diagonal covariance matrix, corresponding standard deviation vector is | p _i-p _n,i|;

D) neighborhood optimal particle value set as new sampling particle position, each particle gives new value according to likelihood function, namely obtains Weight particle assembly wherein wherein for observation function and likelihood function;

E) two class particle assemblies merging are obtained the particle assembly that number is 2N;

F) correction factor is calculated ${\mathrm{\λ}}_t^j=f_t\left(s_t^j\right)/g_t\left(s_t^j\right),1\≤j\≤2N;$

G) final weight is calculated and regularization makes

H) calculate as final carriage estimated result;

I) to set carry out resampling and obtain new set for subsequent time Attitude estimation is prepared.

3. real-time body's whole body body motion capture method of view-based access control model according to claim 1, it is characterized in that, to local attitude parameter optimization correction in step S4.2, its be to the limb part of human body comprise upper body trunk, Jing head, left and right upper arm, left and right forearm, left and right thigh, left and right shank attitude parameter selectively revise; To human body part k (human body is divided into 10 parts and k=1 ..., 10), supposing that one group of Three-dimensional Gravity that it comprises under the attitude recovered by step S4.1 is laid foundations with the corresponding set of manikin point is then obtain following optimization problem:

${\arg }_{{\mathrm{\θ}}_j^{*}}\min \frac{1}{2}\underset{k\∈B_j}{\Σ}\underset{i}{\Σ}\left|\right|p_i^k\left({\mathrm{\θ}}_j^{*}\right)-v_i^k|{|}^2$

Wherein for corresponding to the attitude parameter of human body part k, expression is subject to the human body parts of impact;

Solve above-mentioned optimization problem and adopt LM gradient optimal method.