CN106846370A - For human-computer interaction based on laser sensor depth camera system data processing method - Google Patents

Abstract

The invention discloses it is a kind of for human-computer interaction based on laser sensor depth camera system data processing method, including data acquisition and noise processed, it is characterised in that the method for the noise processed is：The view data of the acquired original that will be obtained from camera system, it is divided into two parts, dynamic adaptive filter is carried out to wherein Part I, Unscented kalman filtering is carried out to wherein Part II, finally the above-mentioned data handled well are carried out with cluster filter, the data after output treatment.The present invention can effectively remove the noise in the data of depth camera head system acquisition, the high accurancy and precision and common-path interference of the track following of system be ensure that, while ensuring that human-computer interaction system can have good stability and robustness.

Description

For human-computer interaction based on the treatment of laser sensor depth camera system data Method

Technical field

The present invention relates to a kind of data processing method, the noise of the view data in more particularly to a kind of human-computer interaction system Suppressing method.

Background technology

Human-computer interaction is exactly to realize the interaction between people and machine.With multimedia technology and the quick hair of shadow casting technique Exhibition, application of the human-computer interaction system in our life is more and more extensive.The human-computer interaction technology in such as museum and exhibition People can be allowed preferably to receive popular science knowledge and merchandise news.But the quality of life of present people improves, and also requires that people The each side of machine interaction systems has performance higher.In human-computer interaction system, the Position location accuracy of target trajectory and mutually The anti-interference of dynamic system, robustness is all critically important index.In order to ensure the good of interaction effect, in design interaction systems When will emphatically consider these indexs.In current human-computer interaction system, it is mainly based upon voice or machine vision comes real Existing.System based on interactive voice, inefficiency, interaction effect is poor, especially in a noisy environment, is extremely difficult to anticipation Target.The system of view-based access control model interaction, the low cost of hardware, interactive mode is varied.But current man-machine interaction Tracking of the system to target trajectory is inaccurate, and antijamming capability is also poor, therefore can not well reach the man-machine friendship of real-time and precise Mutual target.

Based on laser sensor depth camera head system, infrared signal can be received, user coordinates hand using infrared pen Gesture carries out human-computer interaction, and man-machine interaction will be made not limited by regular display screen, and the application to man-machine interaction has significantly Meaning.But, the data of camera system how are processed, it is to need to solve to adapt to complex vision applied environment Problem.

The content of the invention

Goal of the invention of the invention be to provide it is a kind of for human-computer interaction based on laser sensor depth camera head system Data processing method, it is ensured that the precision and anti-interference of target trajectory tracking, to lift Interactive Experience.

To achieve the above object of the invention, the technical solution adopted by the present invention is：It is a kind of for human-computer interaction based on laser Sensor depth camera system data processing method, including data acquisition and noise processed, the method for the noise processed is：

The view data of the acquired original that will be obtained from camera system, is divided into two parts, and wherein Part I is carried out Dynamic adaptive filter, Unscented kalman filtering is carried out to wherein Part II, and finally the above-mentioned data handled well are gathered Class is filtered, the data after output treatment.

In above-mentioned technical proposal, the dynamic adaptive filter method is as follows：

According to the polar value (l of sensing range-measurement system_i,j,λ_i,j), data analysis window is designed as：

In formula, i represents that human-computer interaction system senses the sampling instant of ranging data；J refers to measurement point in a frame data Numbering, 9 measured values in above-mentioned data analysis window have larger correlation on room and time, define Δ l_minIt is l_i,j With the difference of adjacent measured values, it is as follows：

Δl_min=min | l_t+i,s+j-l_i,j|, t, s=-1,0,1＆t ≠ 0, s=0＆t=0, s ≠ 0 } (2)

If Δ l_min>δ (l, υ), then measured value l_i,jJust it is taken as measurement noise and casts out, δ (l, v) is neighbouring difference threshold Value.

In dynamic environment, neighbouring difference threshold design is defined as follows：

In formula, σ (l) is the standard deviation that depth camera head system is inductively measured, and distance is inductively measured by different interaction systems Value Data is obtained, v_goalIt is the movement velocity of dynamic environment target.

In above-mentioned technical proposal, the method for the Unscented kalman filtering is：

1. one group of sampled point is obtained using formula (4) and (5), its correspondence weights, X is calculated using formula (6)^(t)It is t System mode,It is state average, P is to calculate variance, and λ is scaling parameter, and n is the dimension of state,

w^(t)It is the corresponding weights of t system mode, subscript m is average, and c is covariance, and α, β are non-negative rights to be selected Coefficient,

X⁽ⁱ⁾(k | k) is the state vector at the k moment based on k moment system state estimations,It is to be based on the k moment The predicted state vector at the k moment of state estimation of uniting, P (k | k) is the calculating side at the k moment based on k moment system state estimations Difference,

2. the 2n+1 one-step prediction of Sigma point sets is calculated using formula (7),

X⁽ⁱ⁾(k+1 | k) is the one-step prediction of the k+1 moment system modes based on k moment system state estimations,

X⁽ⁱ⁾(k+1 | k)=f [k, X⁽ⁱ⁾(k|k)] (7)

3. using the one-step prediction and covariance matrix of formula (8) (9) computing system quantity of state,

It is the predicted state vector at the k+1 moment based on k moment system state estimations, w⁽ⁱ⁾It is the i moment to be The corresponding weights of system state,

P (k+1 | k) is the calculating variance at the k+1 moment based on k moment system state estimations, and Q is the association side of system noise Difference battle array,

4. UT conversion is carried out according to one-step prediction value again, produces new sigma point sets,

5. the sigma point sets 4. step predicted substitute into observational equation, the observed quantity predicted, such as formula (10) institute Show,

Z⁽ⁱ⁾(k+1 | k) is the observed quantity of the prediction at the k+1 moment estimated based on the k moment, and h is non-Systems with Linear Observation equation letter Number,

Z⁽ⁱ⁾(k+1 | k)=h [X⁽ⁱ⁾(k+1|k)] (10)

6. the observation predicted value of sigma point sets is 5. obtained by step, by weighted sum obtain system prediction average and Shown in covariance, such as formula (11) (12) (13),

It is the observed quantity average of the prediction at the k+1 moment estimated based on the k moment,It is to be seen based on the k moment The calculating variance of measurement, R is the covariance matrix of observation noise,

7. Kalman gain matrixs are calculated using formula (14),

K (k+1) is the kalman gain matrix at k+1 moment,

8. the state of computing system updates and covariance updates, such as shown in formula (15) (16),

It is the system mode of the prediction at the k+1 moment estimated based on the k+1 moment, and P (k+1 | k) it is to be based on The calculating variance at the k+1 moment of k+1 moment system state estimations,

In above-mentioned technical proposal, the cluster filter is carried out using Mean shift clustering algorithms, retains infrared after cluster The ultrared depth map picture point that pen sends, filters out other noise spots.

Because above-mentioned technical proposal is used, the present invention has following advantages compared with prior art：

The present invention can effectively remove the noise in the data of depth camera head system acquisition by Federated filter method, The high accurancy and precision and common-path interference of the track following of system are ensure that, while it is good to ensure that human-computer interaction system can have Stability and robustness.

Specific embodiment

With reference to embodiment, the invention will be further described：

Embodiment one：It is a kind of for human-computer interaction based on laser sensor depth camera system data processing method, Including data acquisition and noise processed.

The data of the acquired original of pretreatment will be needed first, be divided into two parts.Part I is dynamic adaptive filter, It is easy to eliminate external noise.Part II is Unscented kalman filtering, using the principle of its Unscented transform, process it is nonlinear from The covariance of the system of dissipating and the non-linear problem of transmission of average.It is finally that data to having pre-processed carry out cluster filter, uses The data that Mean-shift filtering algorithms will have been pre-processed are divided into different set.Shown in comprising the following steps that：

Step one：

The data that laser sensor depth camera is received are infrared rays, but due in light, can all contain in sunshine There is certain infrared ray, therefore these external noises can form interference in depth image, it is therefore desirable to filter these outside dry Disturb, the data that pretreatment is collected.The mode of pretreatment is exactly using dynamic adaptive filter method.

Filtering is even more important dynamic environment to the real-time for reaching human-computer interaction system online.By the data to collecting Compared in real time, and filtering is compared with the threshold value for pre-setting, so as to reach the effect of filtering interfering.Use herein Online dynamic adaptive filter method (abbreviation DAF, Dynamic Adaptive Filter) eliminates extraneous environmental noise disturbance.

According to the polar value (l of sensing range-measurement system_i,j,λ_i,j), data analysis window may be designed as：

In above formula, i represents that human-computer interaction system senses the sampling instant of ranging data；J refers to being measured in a frame data The numbering of point.9 measured values in above-mentioned data analysis window have larger correlation on room and time, define Δ l_minFor l_i,jWith the difference of adjacent measured values, it is as follows：

Δl_min=min | l_t+i,s+j-l_i,j|, t, s=-1,0,1＆t ≠ 0, s=0＆t=0, s ≠ 0 } (2)

If Δ l_min>δ (l, υ), then measured value l_i,jJust it is taken as measurement noise and casts out.In dynamic environment, dynamic The influence of target speed is most important, then can be designed adjacent to difference threshold and be defined as follows：

In formula, σ (l) is the standard deviation that 3D sensing systems are inductively measured, and distance value is inductively measured by different interaction systems Data are obtained.v_goalIt is the movement velocity of dynamic environment target.

In order to the data to current time differentiate, it is necessary to first the measured value at current time is placed in buffer, etc. Treat that the measurement data of subsequent time is received, DAF treatment could be realized.Therefore the data of the filtering update meeting than actual data Postpone a cycle, but this delay is little to the entire effect of system.

Step 2：

Data after being pre-processed are Higher Order Discrete Systems, because in the motion target tracking of Pure orientation, without mark The effect of Kalman filtering is fine, so we have selected Unscented kalman filtering that data are further filtered with treatment.

Unscented kalman filtering (Unscented Kalman Filter, UKF) has been abandoned traditional to nonlinear function line The way of property, using Kalman's linear filtering framework, for one-step prediction equation, the non-linear transmission of covariance and average is asked Topic can utilize the method without mark change to process.The distribution of the UKF algorithms nonlinear function probability density that has been approximately, is not picture EKF the same linearization approximate nonlinear function, but approach shape with the sample of a series of determination The distribution of the posterior probability density of state.Therefore this kind of filtering algorithm need not carry out derivation to Jacobian matrixes.Because UKF does not have There is the higher order term for ignoring system, so needing the computational accuracy for having comparing high to the statistic of nonlinear Distribution, so can It is poor efficiently against EKF stability, the low shortcoming of estimated accuracy.

Implement flow：

1. one group of sampled point is obtained using formula (4) and (5), its correspondence weights is calculated using formula (6),

2. the 2n+1 one-step prediction of Sigma point sets is calculated using formula (7),

X⁽ⁱ⁾(k+1 | k)=f [k, X⁽ⁱ⁾(k|k)] (7)

3. using the one-step prediction and covariance matrix of formula (8) (9) computing system quantity of state,

4. UT conversion is carried out according to one-step prediction value again, produces new sigma point sets,

5. the sigma point sets 4. step predicted substitute into observational equation, the observed quantity predicted, such as formula (10) institute Show,

Z⁽ⁱ⁾(k+1 | k)=h [X⁽ⁱ⁾(k+1|k)] (10)

6. the observation predicted value of sigma point sets is 5. obtained by step, by weighted sum obtain system prediction average and Shown in covariance, such as formula (11) (12) (13),

7. Kalman gain matrixs are calculated using formula (14),

8. the state of computing system updates and covariance updates, such as shown in formula (15) (16),

Step 3：

For pretreated data, due to the presence of still some noise so that the validity of feature extraction is significantly Reduce, it is therefore desirable to which continuation carries out smoothing denoising treatment to figure.Cluster filter meets above demand.One data set is pressed Different class or clusters are divided into according to certain specific standard (such as distance criterion), can so increase the data in same cluster The similitude of object, at the same time can also make the difference of data object not in same cluster bigger.Make after clustering same The data of class gather one piece as far as possible, and inhomogeneity data are tried one's best and allow it to separate, the infrared pen required for us are then left again Ultrared depth map picture point is sent, other noise spots are filtered out.Filtered using Mean shift clustering algorithms herein Ripple.

Mean shift algorithms are a kind of estimation procedures of non-parametric density, are a kind of feature based space density gradient sides To and the iterative search that carries out, so as to obtain the sample data of local density's maximum.Compared to other filtering, its advantage It is the characteristics of need not understanding feature space in advance, it is only necessary to which the sample point according to providing can be carried out estimating filtering.Mean The core of shift algorithms is namely based on the printenv method of estimation of Density Estimator, and each point is moved to density function by it Local modulus maxima at, i.e., density gradient is 0 point, is also called mode point.

Shown in Density Estimator such as formula (17) at the x of hyperspace midpoint.

Scaling function K (x) should meet：

Wherein | | | | expression makes Euclidean distance, R^dWhat is represented is d dimensional feature spaces, and parameter h is the letter of sample size n Number.In order to ensure that sample is estimated less than density estimation, so that it is guaranteed that the unbiased esti-mator of density can be obtained.Need to meet formula (22) (23)(24)。

Euclidean distance unbiased esti-mator：

When kernel function uses EPanechikov cores, integration mean square deviation is minimum.

Kernel functions are as follows：

Wherein c_dFor d ties up unit sphere volume.

Using the differentiability of kernel function, the gradient of Density Estimator is constantly equal to Density gradient estimation, can obtain

Density estimation kernel function G at x points is expressed as：

Now density gradient is shown as shown in formula (28)：

Mean shift vectors are obtained, such as shown in formula (29)：

Mean shift algorithms implement step：

1. Mean shift vector Ms are calculated_h,G(x)；

2. according to M_h,GX () is assigned to x；

3. above-mentioned two step is iteratively repeated, M is calculated_h,G(x)-x, until stopping bar of the density gradient less than certain setting Part, exits circulation.

The Federated filter algorithm can effectively filter out noise, ensure that the high accurancy and precision of the track following of system and resist by force dry Immunity, while ensuring that human-computer interaction system can have good stability and robustness.

Claims (5)

1. it is a kind of for human-computer interaction based on laser sensor depth camera system data processing method, including data acquisition And noise processed, it is characterised in that the method for the noise processed is：

The view data of the acquired original that will be obtained from camera system, is divided into two parts, and Mobile state is entered to wherein Part I Adaptive-filtering, Unscented kalman filtering is carried out to wherein Part II, finally carries out cluster filter to the above-mentioned data handled well Ripple, the data after output treatment.

2. it is according to claim 1 for human-computer interaction based on laser sensor depth camera system data treatment side Method, it is characterised in that：The dynamic adaptive filter method is as follows：

According to the polar value (l of sensing range-measurement system_i,j,λ_i,j), data analysis window is designed as：

$\left[\begin{array}{ccc}{l}_{i-1,j-1}& l_{i-1,j}& l_{i-1,j+1}\\ l_{i,j-1}& l_{i,j}& l_{i,j+1}\\ l_{i+1,j-1}& l_{j+1,j}& l_{i+1,j+1}\end{array}\right]---\left(1\right)$

In formula, i represents that human-computer interaction system senses the sampling instant of ranging data；J refers to the volume of measurement point in a frame data Number, 9 measured values in above-mentioned data analysis window have larger correlation on room and time, define Δ l_minIt is l_i,jAnd phase The difference of adjacent measured value, it is as follows：

Δl_min=min | l_t+i,s+j-l_i,j|, t, s=-1,0,1＆t ≠ 0, s=0＆t=0, s ≠ 0 } (2)

If Δ l_min>δ (l, υ), then measured value l_i,jJust it is taken as measurement noise and casts out, δ (l, v) is neighbouring difference threshold.

3. it is according to claim 2 for human-computer interaction based on laser sensor depth camera system data treatment side Method, it is characterised in that：In dynamic environment, neighbouring difference threshold design is defined as follows：

$\mathrm{\δ}(l,v)=\mathrm{\σ}\left(l\right)+\frac{1}{25}\left|v_{goal}\right|---\left(3\right)$

In formula, σ (l) is the standard deviation that depth camera head system is inductively measured, and distance value number is inductively measured by different interaction systems According to obtaining, v_goalIt is the movement velocity of dynamic environment target.

4. it is according to claim 1 for human-computer interaction based on laser sensor depth camera system data treatment side Method, it is characterised in that：The method of the Unscented kalman filtering is：

1. one group of sampled point is obtained using formula (4) and (5), its correspondence weights, X is calculated using formula (6)^(t)It is for t System state,It is state average, P is to calculate variance, and λ is scaling parameter, and n is the dimension of state,

$\left\{\begin{array}{c}{X}^{\left(0\right)}=\stackrel{\‾}{X},i=0\\ X^{\left(i\right)}=\stackrel{\‾}{X}+{\left(\sqrt{(n+\mathrm{\λ})P}\right)}_i,i=1~n\\ X^{\left(i\right)}=\stackrel{\‾}{X}-{\left(\sqrt{(n+\mathrm{\λ})P}\right)}_i,i=n+1~2n\end{array}\right.---\left(4\right)$

w^(t)It is the corresponding weights of t system mode, subscript m is average, and c is covariance, and α, β are non-negative weight coefficients to be selected,

$\left\{\begin{array}{c}{w}_m^{\left(0\right)}=\frac{\mathrm{\λ}}{n+\mathrm{\λ}}\\ w_c^{\left(0\right)}=\frac{\mathrm{\λ}}{n+\mathrm{\λ}}+(1-{\mathrm{\α}}^2+\mathrm{\β})\\ w_m^{\left(i\right)}=w_c^{\left(i\right)}=\frac{\mathrm{\λ}}{2(n+\mathrm{\λ})},i=1~2n\end{array}\right.---\left(5\right)$

X⁽ⁱ⁾(k | k) is the state vector at the k moment based on k moment system state estimations,Etching system shape when being based on k The predicted state vector at the k moment that state is estimated, P (k | k) is the calculating variance at the k moment based on k moment system state estimations,

$X^{\left(i\right)}\left(k\right|k)=\[\begin{array}{ccc}\hat{X}\left(k\right|k)& \hat{X}\left(k\right|k)+\sqrt{(n+\mathrm{\λ})P\left(k\right|k)}& \hat{X}\left(k\right|k)-\sqrt{(n+\mathrm{\λ})P\left(k\right|k)}\end{array}\]---\left(6\right)$

2. the 2n+1 one-step prediction of Sigma point sets is calculated using formula (7),

X⁽ⁱ⁾(k+1 | k) is the one-step prediction of the k+1 moment system modes based on k moment system state estimations, X⁽ⁱ⁾(k+1 | k)= f[k,X⁽ⁱ⁾(k|k)] (7)

3. using the one-step prediction and covariance matrix of formula (8) (9) computing system quantity of state,

It is the predicted state vector at the k+1 moment based on k moment system state estimations, w⁽ⁱ⁾It is i moment system modes Corresponding weights,

$\hat{X}(k+1|k)=\underset{i=0}{\overset{2n}{\Σ}}{w}^{\left(i\right)}{X}^{\left(i\right)}(k+1|k)---\left(8\right)$

P (k+1 | k) is the calculating variance at the k+1 moment based on k moment system state estimations, and Q is the covariance matrix of system noise,

$P(k+1|k)=\underset{i=0}{\overset{2n}{\Σ}}{w}^{\left(i\right)}\[\hat{X}(k+1|k)-X^{\left(i\right)}(k+1|k)\]{\[\hat{X}(k+1|k)-X^{\left(i\right)}(k+1|k)\]}^T+Q---\left(9\right)$

4. UT conversion is carried out according to one-step prediction value again, produces new sigma point sets,

5. the sigma point sets 4. step predicted substitute into observational equation, shown in the observed quantity predicted, such as formula (10),

Z⁽ⁱ⁾(k+1 | k) is the observed quantity of the prediction at the k+1 moment estimated based on the k moment, and h is non-Systems with Linear Observation equation functions,

Z⁽ⁱ⁾(k+1 | k)=h [X⁽ⁱ⁾(k+1|k)] (10)

6. the observation predicted value of sigma point sets is 5. obtained by step, average and the association side of system prediction is obtained by weighted sum Shown in difference, such as formula (11) (12) (13),

It is the observed quantity average of the prediction at the k+1 moment estimated based on the k moment,It is based on k moment observed quantities Variance is calculated, R is the covariance matrix of observation noise,

$\stackrel{\‾}{Z}(k+1|k)=\underset{i=0}{\overset{2n}{\Σ}}{w}^{\left(i\right)}{Z}^{\left(i\right)}(k+1|k)---\left(11\right)$

$P_{z_k{z}_k}=\underset{i=0}{\overset{2n}{\Σ}}\[Z^{\left(i\right)}(k+1|k)-\stackrel{\‾}{Z}(k+1|k)\]{\[Z^{\left(i\right)}(k+1|k)-\stackrel{\‾}{Z}(k+1|k)\]}^T+R---\left(12\right)$

$P_{x_k{z}_k}=\underset{i=0}{\overset{2n}{\Σ}}{w}^{\left(i\right)}\[X^{\left(i\right)}(k+1|k)-\stackrel{\‾}{Z}(k+1|k)\]{\[Z^{\left(i\right)}(k+1|k)-\stackrel{\‾}{Z}(k+1|k)\]}^T---\left(13\right)$

7. Kalman gain matrixs are calculated using formula (14),

K (k+1) is the kalman gain matrix at k+1 moment,

$K(k+1)=P_{x_k{z}_k}{P}_{z_k{z}_k}^{-1}---\left(14\right)$

8. the state of computing system updates and covariance updates, such as shown in formula (15) (16),

It is the system mode of the prediction at the k+1 moment estimated based on the k+1 moment, and P (k+1 | k) it is to be based on k+1 The calculating variance at the k+1 moment of moment system state estimation,

$\hat{X}(k+1|k+1)=\hat{X}(k+1|k)+K(k+1)\[Z(k+1)-\hat{Z}(k+1|k)\]---\left(15\right)$

$P(k+1|k+1)=P(k+1|k)-K(k+1)P_{z_k{z}_k}{K}^T(k+1)---\left(16\right).$

5. it is according to claim 1 for human-computer interaction based on laser sensor depth camera system data treatment side Method, it is characterised in that：The cluster filter is carried out using Mean shift clustering algorithms, and retain that infrared pen sends after cluster is red The depth map picture point of outside line, filters out other noise spots.