CN101692284B

CN101692284B - Three-dimensional human body motion tracking method based on quantum immune clone algorithm

Info

Publication number: CN101692284B
Application number: CN2009100234183A
Authority: CN
Inventors: 韩红; 岳立川; 伍星; 焦李成; 李阳阳
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2009-07-24
Filing date: 2009-07-24
Publication date: 2012-01-04
Anticipated expiration: 2029-07-24
Also published as: CN101692284A

Abstract

The invention discloses a three-dimensional human body motion tracking method based on quantum immune clone algorithm. The tracking process comprises: detecting articulation points of the human body in a two dimension picture in a unary picture sequence with human body gesture existing, using a kalman filter to predict the articulation points not detected, and simultaneously establishing a three-dimensional human body model; applying quantum immune clone algorithm in human body tracking, initializing species group, setting initial parameters of human body motion, then carrying out clone operation, performing operation of immunizing genes and immunoselection to generate a new species group; substituting the state parameters of the new species group in the three-dimensional human body model, using likelihood function to calculate the distance affinity, retaining the optimal solution, performing multiple times of substitution and calculation to obtain the ideal human body motion parameters to restore the three-dimensional human body gesture. The invention has the advantages of lower calculation cost and capable of obtaining three-dimensional human body gesture quickly, and is applicable to tracking of human body motion gesture in scene monitoring, motion analysis and health assessment.

Description

Three-dimensional human motion tracking method based on quantum immune clone algorithm

Technical Field

The invention belongs to the technical field of image processing, relates to a human body tracking method, and can be used for tracking human body movement in images or image sequences to realize scene monitoring, virtual reality, game making, sports analysis and health evaluation.

Background

On the basis of image processing, in order to enable a computer to understand images, computer vision is generally introduced, and the main purpose of the introduction is to provide a quantitative means for vision, realize human-like visual functions through an artificial intelligence system, and perform visual navigation, scene monitoring, extraction of structural or motion features of various types of objects, people and animals and the like by using a monocular and monocular camera. However, it is very difficult to make the computer have human visual ability, because human lives in a three-dimensional geometric space, the visual sensor can only obtain projection information of three-dimensional world, namely two-dimensional image, this process results in a great deal of information loss, and the computer visual task becomes more complicated by the dynamic scene information obtained by the moving visual sensor.

In the study of human motion analysis based on vision, joint motion analysis based on a marker point and motion analysis without a marker point can be classified according to the presence or absence of a joint marker point. In Motion analysis based on the mark points, the joint mark points are accurately detected by stereo vision, and the three-dimensional positions of the joint points can be easily obtained, so that the three-dimensional reconstruction of the human skeleton is performed. In the motion analysis without the mark point, the motion analysis based on the multi-view camera and the human body motion analysis based on the monocular camera can be further classified.

In monocular image-based human motion analysis, many previous studies manually mark an initial point on a first frame image to achieve initial tracking or estimation. The predecessors also have many precedents for adopting an optimization algorithm to track the human body, and the traditional optimization algorithms such as an evolutionary algorithm and a genetic algorithm are applied to human body tracking. The human body tracking dimension is high, the fitness curved surface is complex, the optimization problem becomes extremely difficult, and many traditional optimization methods are insufficient. However, the evolutionary algorithm can better solve the problems mainly in that the evolutionary algorithm basically does not need domain knowledge of the problems and has no limitation on the types of functions and the shapes of search spaces, but the evolutionary algorithm is troubled mainly in that the algorithm possibly falls into local extreme points, the human body tracking dimension is high, the fitness curved surface is complex, and once the algorithm falls into the local extreme points, the human body posture cannot be well recovered.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a three-dimensional human motion tracking method based on a quantum immune clone algorithm so as to further improve the accuracy of the three-dimensional human motion posture and realize human motion tracking.

The technical scheme for realizing the purpose of the invention is as follows: the quantum immune clone algorithm has the advantage of efficiently searching a global optimal solution in a high-order space, and based on human key joint points obtained by a two-dimensional human key joint point automatic detection method, global optimal state parameters are obtained through a distance similarity function of the two-dimensional key joint points and three-dimensional projection points, and finally, robust and stable three-dimensional human postures are recovered. The specific process is as follows:

(1) detecting key joint points of a human body surface layer in a two-dimensional image in a monocular image sequence with human body gestures; a classical kalman filter is used for predicting shielding points and missing detection points of the detected two-dimensional human body key joint points, so that the motion of the two-dimensional human body key joint points is more reasonable and stable;

(2) establishing a virtual human body three-dimensional skeleton model according to the detected and predicted two-dimensional human body key joint points, so that dynamic posture adjustment and matching are realized in the tracking process;

(3) introducing a quantum immune clone algorithm into human motion tracking, firstly initializing a population, setting human motion initial parameters, and then carrying out clone operation to increase the search space of human posture parameters to be estimated;

(4) performing immunogene operation and immunoselection on the population subjected to the cloning operation to generate a new population;

(5) substituting the state parameters of the new population into the three-dimensional human body model to generate a three-dimensional coordinate P of a key joint point_i＝(P_ix，P_iy，P_iz) And recording the key point projected to the image plane by the three-dimensional coordinate of the key point as p_i＝(p_ix，p_iy) Recording the detected two-dimensional human body key joint point as q_i＝(q_ix，q_iy) And constructing a distance function by using the two key points as follows:

(6) constructing a similarity function according to the distance function G (X) as follows: calculating the distance affinity between the two-dimensional detection point and the three-dimensional projection point by using the similarity function, keeping the optimal solution, and stopping the calculation if the optimal solution meets the set termination condition; otherwise, returning to the step (3), obtaining ideal human body motion parameters through multi-generation calculation, and recovering the three-dimensional human body posture.

Compared with the prior art, the invention has the following advantages:

1. the invention adopts automatic detection of two-dimensional key joint points of the human body to establish a reasonable three-dimensional human body model, so that the three-dimensional model can realize dynamic posture adjustment in the tracking process and can be efficiently matched with the original rotary table.

2. According to the invention, the distance similarity function of the two-dimensional key joint points and the three-dimensional projection key joint points is adopted, so that human body tracking can be well combined with a quantum immune clone operator, and human body state parameters can be quickly obtained.

3. The invention adopts the chaos variation method, so that the search space of the human body parameters is enlarged and the calculation cost is lower in the search process.

Drawings

FIG. 1 is a schematic flow diagram of the present invention;

FIG. 2 is a diagram of a human skeletal model used in the present invention;

FIG. 3 is a three-dimensional body pose diagram of an implementation of the present invention;

FIG. 4 is a graph of the average of the sum of the distances between the three-dimensional skeleton point projection and the two-dimensional corresponding detection points in the present invention.

Detailed Description

The invention provides a novel method for tracking three-dimensional human motion based on a quantum immune clone algorithm from the continuity of an image sequence, namely, the method utilizes the super-strong capability of the quantum immune clone algorithm for searching global optimal solution to track the three-dimensional human motion. On the basis of some improvements on the quantum evolutionary algorithm, relevant optimal parameters and statistical characteristics are established through experiments, then a quantum updating operator and a quantum crossing operator are used for inducing state parameters, and a tracking result is compared with a corresponding two-dimensional detection point. As detailed in fig. 1, 3 and 4.

Referring to fig. 1, the specific implementation process of the present invention is as follows:

step 1, detecting key joint points of a human body in a two-dimensional image, and estimating missed detection points by using a kalman filter.

The image sequence used by the invention is obtained from a video sequence in which human body movement exists, 660 frames are totally obtained, because the invention does not research a segmentation method, the human body movement of the video sequence is not self-shielded and shielded, and because the rotation detection of the human body trunk has a great problem, the human body movement in the invention does not reflect the rotation of the human body trunk, and meanwhile, in order to reduce the dimension of a human body model, it is generally assumed that the joint points on the human body trunk have no degree of freedom.

Finding key joint points of a human body from silhouette information of the human body is very challenging work, is mainly selected by experience without much theory, and the three-dimensional positions of the key joint points are not accurately obtained like a motion capture device. Before detecting the joint point, the following assumptions are first made: 1) supposing that the first two frames of images have no human body silhouette self-shielding or are shielded, the hypothesis can ensure that the subsequent shielded joint points can be predicted according to a certain rule; 2) the assumption that the projection of the center of the silhouette is not far away from the root node of the three-dimensional skeleton can ensure that the root node is obtained under any condition. The human silhouette of the foreground is segmented from the image sequence, and then the human silhouette area is refined into a single-pixel image skeleton by using a digital morphology method. The measuring process is as follows:

(1.1) head point detection: in the human body silhouette, the shape of the head is generally not blocked, the characteristics are most stable, and the head is generally circular, so that the three-dimensional pixel data detected by a person before reference is used for referenceAnd a three-dimensional spherical shell template of the middle head part provides a 2-dimensional concentric circle template for detecting the head area of the video image sequence. The center of the inner circle of the concentric circle is assumed to be R₁The center of the excircle is R₂The center of the concentric circle template is along the skeleton S ═ S of the side shadow_iSearching, wherein i is 1, 2, …, N is the number of skeleton points of the silhouette region of the current frame, and calculating a contour C which falls between an inner circle and an outer circle and C is { C }_jJ is 1, 2, …, M is the number of contour points of the current frame, and when the number of points falling into the inner and outer circles is the maximum, s is the maximum_iThe head point is the common excircle radius which is more than 2 times of the inner circle radius;

(1.2) root node detection: let the set of side-shadow points of the human body be denoted as A ═ a_kThe number of points is L, and then the central point is set to

The empirical test method of the root node is to take the skeleton point S ═ S_iA point on the human body side shadow nearest to the center point;

(1.3) shoulder joint and crotch joint detection: assuming that the head point and the root node are detected, binding the left shoulder joint, the right shoulder joint, the crotch joint and the whole trunk to keep the relative positions unchanged in the detection process, matching the head projection point and the root node projection point of the three-dimensional framework with the corresponding points on the image, and projecting the three-dimensional shoulder joint points to the silhouette area of the person;

(1.4) limb endpoint detection: on the silhouette skeleton line, searching the end points of the skeleton line by using an end point searching method, generally speaking, under the condition of no burr, the positions of four limbs are easily detected, but under the condition of high noise, the method is invalid, so that a search forbidding area is provided, namely a rectangular area is defined between a head point and a root node of a human body and between two shoulder joint points, the searched end points in the area are discarded, and the detection accuracy of the four limb points is greatly improved;

(1.5) detecting the knee joint and the elbow joint: firstly, defining a point set of a silhouette skeleton line above a root node as an upper half body, defining other points as a lower half body, making parallel lines of two feet passing through the center point of the lower half body, wherein the intersection point of the parallel lines and the skeleton line is a knee joint point, and elbow joint detection is to find one point on the skeleton line so that the difference between the distance between the point and the hand and the distance between the point and the shoulder is minimum;

when the human body movement generates white shielding or some key points cannot be correctly detected due to the imperfection of the detection algorithm, a shielding point prediction mechanism is required to be implemented, and a classical Kalman filter is used for shielding point prediction.

The following brief description of the basic theory of kalman filtering considers a discrete-time linear stochastic dynamical system:

x_k+1＝F_kx_k+Γ_kw_k

z_k＝H_kx_k+v_k

where k ∈ N is a time index, x_k∈RⁿIs the system state vector at time k, F_kIs a system state transition matrix, w_kThen, it is the process-evolving noise, Γ_kIs a noise matrix, z_k∈R^mIs the measurement vector of time k to the system state, H_kTo measure the matrix, v_kIs the measurement noise.

The key joint points missing between two frames can be estimated and obtained through simple extrapolation, and meanwhile, filtering is carried out on the extraction framework and the noise in the detection process, so that the positions of all detected two-dimensional key joint points are more reasonable.

And 2, establishing a virtual human body three-dimensional skeleton model according to the detected and predicted two-dimensional human body key joint points.

(2.1) the skeletal body model used in the present invention is a kinematic chain connected with connecting rigid bodies and uses a few a priori constraints to generate the trunk, root nodes and crotch of the body, neck joints and shoulders with no freedom of movement, only with the trunk;

(2.2) in order to reduce the size of a search space as much as possible, in a model which is generally adopted, the human skeleton has 21 degrees of freedom, including 3 integral translations, 6 crotch joints, 2 knee joints, 6 shoulder joints, 2 elbow joints and 2 neck joints;

(2.3) establishing a three-dimensional human body model, firstly roughly determining the proportion of each part of the human body according to the human anatomy proportion, wherein the center of a coordinate system of the human body model is an abdominal node, namely a root node, each joint point has a local coordinate system, and the coordinates of the sub-nodes of the joint point use a matrix M_P，CMove to parent coordinates:

M_P，C＝T(t_x，t_y，t_z)·R_z(γ)·R_y(β)·R_x(α)

wherein T (T)_x，t_y，t_z) Representing translation t of child node coordinates along the x-axis_xTranslation t along the y-axis_yTranslation t along the z-axis_z，R_x(α) denotes a rotation of the child node coordinate by α degrees about the x-axis, R_y(β) represents a rotation of the child node coordinates by β degrees about the y-axis, R_z(γ) represents the rotation of the child node coordinates by γ degrees around the z-axis, and their matrix is represented by:

T (t_{x}, t_{y}, t_{z}) = [\begin{matrix} 0 & 0 & 0 & t_{x} \\ 0 & 0 & 0 & t_{y} \\ 0 & 0 & 0 & t_{z} \\ 0 & 0 & 0 & 1 \end{matrix}]

for any point in the local coordinate system of the child node, p is (x, y, z, 1)^TAnd the position in the parent node local coordinate system is P ═ (X, Y, Z, 1)^TThe interconversion relationship between them is:

thus, through several successive transformation operations, a kinematic chain is formed, and the relationship between the 3D coordinate of each point and each rotation angle is expressed, so that the position of each point in the world coordinate system can be obtained, for example, the coordinates of each point of the left lower limb are:

P_LH＝T(t_x，t_y，t_z)R_0zR_0yR_0xT_LHR·[0，0，0，1]^T

P_LK＝T(t_x，t_y，t_z)R_0zR_0yR_0xT_LHRR_1zR_1yR_1xT_LKH[0，0，0，1]^T

P_LA＝T(t_x，t_y，t_z)R_0zR_0yR_0xT_LHRR_1zR_1yR_1xT_LKHR_2zR_2yR_2xT_LAK·[0，0，0，1]^T

P_LF＝T(t_x，t_y，t_z)R_0zR_0yR_0xT_LHRR_1zR_1yR_1xT_LKH

R_2zR_2yR_2xT_LAKR_3zR_3yR_3xT_LFK·[0，0，0，1]^T

wherein, P_LH、P_LK、P_LAAnd P_LFRespectively, a left hip point, a left knee point, a left ankle point and a left foot point, T (T)_x，t_y，t_z) For the displacement of the root node to the origin of the world coordinate system, R_ixFor the rotation matrix of the ith node around x, R_iyIs a rotation matrix of the ith node around y, R_izThe rotation matrix around z for the ith node, subscripts 0x, 0y and 0z represent the overall rotation of the torso; t is_LHR、T_LKH、T_LAKAnd T_LFARepresenting the initial translation of the left hip to the root node, the initial translation of the left knee to the left hip node, the initial translation of the left ankle to the left knee, and the initial translation of the left foot to the left ankle, respectively; the method for solving the upper limb points is similar to that for the lower limb points, the lower limb points are not listed one by one, and after the coordinates of all the joint points in the world coordinate system are obtained, the projection values of all the joint points in the image coordinate system can be easily deduced according to various projection models, so that the three-dimensional human body model is integrally designed, as shown in fig. 2.

And 3, initializing a population, setting initial parameters of human motion, and carrying out cloning operation.

(3.1) according to the degree of freedom of each key joint point, setting the human body parameter set X to be estimated as follows:

X＝{x，y，z，θ₁，θ₂，…，θ₁₈}

where x, y, z represent the translation distance of the global translation along the x, y, z axes, respectively, θ₁，θ₂，…，θ₁₈Representing the rotation angle of the key joint point of the human body to be estimated; because of the continuity of human motion, the angle of change of human body limbs between every two frames of images is not large, and the rotation angle theta of the key joint point of the human body to be estimated between the two frames of images is set according to the statistics of the connected frames of the image sequence₁，θ₂，…，θ₁₈Has a range of [ -15, +15]Degree; since the motion translation is also small, the overall translation x, y, z is set here to range from [ -15, +15]A pixel.

In the estimation process, each frame of image only needs to obtain the difference value Δ X of the parameters between two frames:

ΔX＝{Δx，Δy，Δz，Δθ₁，Δθ₂，…，Δθ₁₈}

where Δ x, Δ y, Δ z represent the difference in the global translation between two frames along the x, y, z axes, respectively, and Δ θ₁，Δθ₂，…，Δθ₁₈Representing the difference of the three-dimensional human body state angle to be estimated between two frames.

According to the setting of the human body parameters to be estimated, setting an initial population A (0) as follows:

wherein each column element value is uniformly assigned to the k +1 numbers from +15 to-15; each row represents 21 parameter values to be estimated; k represents the size of the state value which can be selected by each element to be estimated; each initial element is correspondingly provided with an equal probability of 1/k + 1;

(3.2) cloning was performed to generate a cloned population A' (k).

According to the initialization of the population of human state parameters, in order to expand the search space, a cloning operation, denoted T, is performed on the initial population_c ^C：

T_{c}^{C} (A) = {[\begin{matrix} T_{c}^{C} (a_{1}) & T_{c}^{C} (a_{2}), & . . ., & T_{c}^{C} (a_{N}) \end{matrix}]}^{T}

Wherein,

(I1, 2.., N), N represents the number of elements in each line a (k), I_iIs q with an element value of 1_iDimension row vector, q_iIs an antibody a_iScale after cloning, a_iDenotes all elements of line i in A (k), q_iTaking the following steps:

int (x) represents the smallest integer greater than or equal to x, and N_cIs a set value in relation to the scale after cloning and satisfies N_c＞N，D(a_i(k) Represents the magnitude of the probability of each element in A (k),

after cloning, the initial population forms a plurality of search populations according to each element probability, and the populations are defined as A' (k):

A′＝{A，A′₁，A′₂，...，A′_N}

wherein, A'_iRepresents a new population generated after cloning operation, and N represents the size of the population after cloning.

And 4, performing immunogene operation and immunoselection on the population subjected to the cloning operation to generate a new population.

(4.1) carrying out immune gene operation on the cloned sub-population to generate an immune population A' (k);

after cloning operation, the search space of human state parameters is expanded by several times according to the distance affinity of human joint points, and in order to prevent the slow convergence rate of the human state function and premature collapse of the human state function in a local optimal solution, immune gene operation is carried out, which mainly comprises cloning recombination and cloning variation. Realizing clone recombination by using a quantum updating operator, and realizing clone mutation by using a quantum crossing operator; the quantum updating operator adopts a quantum revolving gate to accelerate convergence and chaotic mutation operation so as to prevent precocity.

Quantum spin gate in quantum update operator, denoted as U (θ):

where θ is the angle of the rotational update.

And judging the direction and angle of rotation according to the probability of the element in A' (k), and finally realizing the updating operation through continuous updating and rotation.

The chaotic variation in the quantum updating operator is trapped in local optimization in order to prevent the antibody from evolving towards one aspect, and the chaotic variation is defined here to increase the local searching capability. For the antibody updated by the quantum rotation, the variation probability p is used_mRandomly selecting one or a plurality of bits, and implementing the following operations:

<math> <mrow> <msubsup> <mi>a</mi> <mi>ji</mi> <mo>′</mo> </msubsup> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <msub> <mi>a</mi> <mi>ji</mi> </msub> <mo>+</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <msub> <mi>a</mi> <mi>ji</mi> </msub> <mo>)</mo> </mrow> <mo>×</mo> <msub> <mi>Logistic</mi> <mi>ji</mi> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mtd> <mtd> <mi>if</mi> </mtd> <mtd> <mi>rand</mi> <mo>></mo> <mn>0.5</mn> </mtd> </mtr> <mtr> <mtd> <msub> <mi>a</mi> <mi>ji</mi> </msub> <mo>-</mo> <msub> <mi>a</mi> <mi>ji</mi> </msub> <mo>×</mo> <msub> <mi>Logistic</mi> <mi>ji</mi> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mtd> <mtd> <mi>if</mi> </mtd> <mtd> <mi>rand</mi> <mo>≤</mo> <mn>0.5</mn> </mtd> </mtr> </mtable> </mfenced> </mrow> </math>

wherein a is_jiIs the probability value of the ith quantum bit in the jth quantum antibody before mutation, a'_jiIs the probability value of the ith quantum bit in the j quantum antibody after mutation.

The variation scale of chaotic variation consists of two parts, namely:

I.1-a_jior a_jiCan automatically ensure that the antibody variable is still at [0, 1 ] after mutation]In the meantime.

II.Logistic_ji(k) A kth sequence value of a Logistic mapping, wherein the sequence length of the Logistic mapping is q _i1, i.e. the original antibody information is retained, and this mutation is performed in vivo on the subgroup, the Logistic mapping is described below:

x_n+1＝μx_n(1-x_n) (n＝0，1，2，...)

wherein, mu is 4, x_n＝a_ji。

(4.2) immunoselection of population A' (k) to form a new population.

Following immunogenetic manipulation, immunoselection, denoted T, was performed_S ^CIt selects excellent antibody from each filial generation and corresponding father generation after clone immune gene operation, thus forming new population. Namely:

according to the degree of affinity,

<math> <mrow> <mo>&ForAll;</mo> <mi>i</mi> <mo>=</mo> <mn>1,2</mn> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mi>N</mi> <mo>,</mo> </mrow> </math>

if there is

b_i(k)＝{a″_ij(k)|maxD(a″_ij)j＝1，2，...，q_i-1}

So that

D(a_i(k))＜D(b_i(k))，i＝1，2，...，N

Then

a_i(k+1)＝b_i(k)，i＝1，2，...，N

Wherein a ″)_ijIndicates the probability of the j-th position of the i-th quantum antibody after the update operation, D (a ″)_ij) Denotes a ″)_ijAffinity of (a), D (a)_i(k) And D (b)_i(k) A represents a_i(k) Affinity of (a) and (b)_i(k) Affinity, a_i(k +1) denotes the ith row element of the next generation.

Namely b_i(k) Substitution of a in A' (k)_i(k) If such b is not present_i(k) Then a is_i(k) After immunoselection was completed, the next generation antibody population a (k +1) was obtained without change.

And 5, substituting the state parameters in the population into the three-dimensional human body model to obtain the three-dimensional coordinates of the key joint points of the human body, and designing a distance function by using the distance between the two-dimensional key joint points and the three-dimensional projection points.

In the human motion estimation based on the monocular image sequence, a projection model is assumed under the condition of unknown camera parameters. The motion parameters to be estimated include the whole translation of the human body model, and 21 parameters X ═ X, y, z, theta of each joint rotation angle₁，θ₂，…，θ₁₈15 key points P on the human body three-dimensional model₁，P₂，…，P₁₅Is determined by these 21 parameters, where P_i＝(P_ix，P_iy，P_iz) Coordinate p after projection onto image plane_i＝(p_ix，p_iy)

p_{ix} = f \frac{P_{ix}}{D}

p_{iy} = f \frac{P_{iy}}{D}

The detected key points in the image are noted as q_i＝(q_ix，q_iy) Let G (X) be the corresponding weighted sum of the two-position detection point and the three-dimensional projection point, then,

and 6, designing a similarity function, calculating distance affinity, storing an optimal solution, and recovering the three-dimensional human body posture.

(6.1) after the distance function G (X) is established, the parameter estimation problem is to solve a nonlinear optimization problem:

X＝minG(X)

calculating the weighted sum of the distances between the two-dimensional key joint points and the three-dimensional projection points by using a similarity function X (argminG (X)), and then solving the minimum value of the weighted sum, wherein the optimal solution is stored in each generation, and the optimal solution in the generation is not greater than the global optimal solution and is not stored; and setting a minimum value delta, terminating the calculation if the global optimal solution is less than or equal to the minimum value delta, and returning to the step (3.2) if the global optimal solution is not less than or equal to the minimum value delta.

And (6.2) finally, according to the previously designed three-dimensional human body skeleton model, substituting the stored optimal solution into the model, and using computer graphics related knowledge to rotate and translate the human body initial skeleton model to obtain a three-dimensional human body reconstruction image.

The effects of the present invention can be further illustrated by the following simulations:

1. simulation content:

the quantum immune clone algorithm provided by the invention is adopted to carry out a three-dimensional human posture recovery experiment. Wherein the image sequence is from a self-timer video sequence, the image frame number is acquired by 660 frames, and the image sequence interval is 1/24 s. In the estimation process, one frame is usually sampled every 5 frames for estimation, and a result of 132 frames is obtained.

The head point detection method is firstly tested by two sequences, and under the condition that the initial radius is proper, the head detection accuracy rate reaches 100 percent and exceeds the requirement of generally used initial points. The detection of other joint points basically meets the requirements of the experiment. For the missed detection point, the characteristic point which can not be detected can be easily estimated usually by Kalman filtering.

The invention resolves the human motion estimation problem into a deterministic non-linear optimization problem, which is generally difficult to find the optimal solution, and theoretically, the problem can be 2^NAnd N is the number of rotational degrees of freedom. The selection of the initial values and whether the motion constraints are appropriate will greatly affect the optimization results, in this experiment, the initial posture of the person is body standing, the hands are extended horizontally, and the values are reflected as

X₀＝(167，96，0，

15，0，0，-30，

0，0，

0，0，0，10，

0，0，0，-10)。

The first three elements are pixels, the others are angles, the 17 th element and the 21 st element are angles of the left elbow and the right elbow and are respectively set to be 10 degrees and-10 degrees, namely, the arms are slightly bent, the 4 th element and the 8 th element are arranged to ensure that thighs are slightly bent forwards, and the 7 th element and the 11 th element are arranged to ensure that the calves stretch backwards. The optimization algorithm can set the search range of each parameter component, the lower boundary of each parameter is marked as LB, the upper boundary is marked as UB, and in summary, the initial value is set to ensure that the motion of the whole person is reasonable in the algorithm search process, and the inversion of the elbow and knee cannot occur.

And according to the affinity evaluation state value, storing the optimal solution and recovering the three-dimensional posture of the human body.

The hardware platform is as follows: intel Core2 Duo CPU E65502.33GHZ, 2GB RAM. The software platform was MATLAB 7.0.

2. Simulation results and analysis

The simulation results are shown in fig. 3 and 4.

FIG. 3 shows a partial estimation, wherein FIG. 3a is the motion of right leg extension and upper arm lifting; FIG. 3b is a front view of the recovered three-dimensional body pose; FIG. 3c is a side view of the recovered three-dimensional body pose; FIG. 3d left leg extended forward with upper arm raised and bent back; FIG. 3e is a front view of the recovered three-dimensional body pose; FIG. 3f is a side view of the recovered three-dimensional body pose; FIG. 3g shows the forward extension of the right leg and the backward extension of the arms; FIG. 3h is a front view of the recovered three-dimensional body pose; fig. 3i is a side view of the restored three-dimensional body posture.

As can be seen from fig. 3, satisfactory three-dimensional human body posture can be estimated by using quantum immune clone algorithm. As can be seen from fig. 4, the matching of the three-dimensional projection points and the two-dimensional correspondence detection points is perfect. And the introduced algorithm is low in calculation cost when being applied to human body tracking, and is not easy to fall into a local optimal solution.

Claims

1. A three-dimensional human motion tracking method based on quantum immune clone algorithm comprises the following processes:

(1) detecting two-dimensional human body key joint points in a two-dimensional image in a monocular image sequence with human body gestures; a classical kalman filter is used for predicting shielding points and missing detection points of the detected two-dimensional human body key joint points, so that the motion of the two-dimensional human body key joint points is more reasonable and stable;

(4) for the population subjected to cloning operation, a quantum updating operator and a quantum crossing operator are used for respectively carrying out cloning recombination and cloning mutation, the quantum updating operator adopts a quantum revolving gate to accelerate convergence, adopts chaotic mutation operation to prevent precocity, and according to the degree of affinity,

if there is

b_i(k)＝{a″_ij(k)|max D(a″_ij)j＝1，2，...，q_i-1}

So that D (a)_i(k))＜D(b_i(k) 1, 2, N, then a_i(k+1)＝b_i(k)，

Wherein a ″)_ijIndicates the probability of the j-th position of the i-th quantum antibody after the update operation, D (a ″)_ij) Denotes a ″)_ijAffinity of (a), D (a)_i(k) And D (b)_i(k) A represents a_i(k) Affinity of (a) and (b)_i(k) Affinity of (a)_i(k +1) denotes the element of the ith row of the next generation, N denotes the number of elements per row of the population A (k), a_i(k) All elements of row i of the population A (k), b_i(k)＝{a″_ij(k)|max D(a″_ij)j＝1，2，...，q_i-1}, wherein q_iRepresents antibody a_iThe scale after cloning is carried out, so that antibodies with relatively high distance affinity are selected from respective filial generations and corresponding parents after cloning immune gene operation to serve as excellent antibodies, and a new population is generated;

(5) substituting the state parameters of the new population into the three-dimensional skeleton model to generate a three-dimensional coordinate P of the key joint point_i＝(P_ix，P_iy，P_iz) And projecting the three-dimensional coordinates of the key joint points to key point marks in an image planeIs p_i＝(p_ix，p_iy) Recording the detected two-dimensional human body key joint point as q'_i＝(q′_ix，q′_iy) Constructing a distance function as:

x is a set of human body parameters to be estimated in total, which are set according to the degree of freedom of each key joint point;

(6) constructing a similarity function according to the distance function G (X) as follows: calculating the weighted sum of the distances between the two-dimensional human key joint points and the three-dimensional projection points by using the similarity function, and then calculating the minimum value to be reserved as the optimal solution, wherein the optimal solution in the present generation is not greater than the global optimal solution and is not saved; if the optimal solution meets the set termination condition, terminating the calculation; otherwise, returning to the step (3), obtaining ideal human body motion parameters through multi-generation calculation, and recovering the three-dimensional human body posture.

2. The three-dimensional human motion tracking method according to claim 1, wherein the detecting of the key joint points of the two-dimensional human body in the two-dimensional image in the process (1) comprises head node detection, root node detection, extremity end point detection and knee joint and elbow joint detection, and the head node detection comprises the following steps:

(2a) in the front human body outline image, the center of an inner circle of a concentric circle is set as R₁The center of the excircle is R₂Forming a concentric circle template;

(2b) c, setting the center of the concentric circle template to be along the skeleton point set of the human body silhouette as { C }_jSearching, wherein j is 1, 2, …, N ', N' is the number of skeleton point pixels in the current human skeleton area; calculating a set of contour points S ═ S between the inner circle and the outer circle_iI is 1, 2, …, and M is the pixel number of the contour point of the current frame, when the pixel number falling into the inner and outer circles is the maximum, c_jI.e. the head node.