CN104200199A - TOF (Time of Flight) camera based bad driving behavior detection method - Google Patents

Abstract

The invention discloses a TOF (Time of Flight) camera based bad driving behavior detection method. The TOF camera based bad driving behavior detection method comprises step 1, obtaining a depth image; step 2, detecting and matching head areas; step 3, detecting and tracking hand areas. The TOF camera based bad driving behavior detection method can accurately and effectively detect bad driver driving behaviors under the condition that normal driving of a driver is not interfered, improves the identification accuracy and accordingly reduces the accident occurrence rate.

Description

Bad steering behavior detection method based on TOF camera

Technical field

The present invention relates to a kind of identification of the motion based on TOF camera and the auxiliary driving technology of safety, belong to the intelligent traffic safety field of 3D vision.

Background technology

Along with improving constantly of Chinese Urbanization and motorization level, traffic safety problem is increasingly serious.Analyze China in recent years road traffic accident reason find, approximately 90% traffic hazard be due to driver bad steering behavior cause.Therefore, to the research of driving behavior especially how to unsafe driving behavior intervene just to become and ensure the key problem of traffic safety.

Mainly to utilize 2D camera to obtain driver's video image to the research of driving behavior both at home and abroad at present, thereby the closed situation of the action to eyelid, eyes, nod and detect with the drivers' such as mouth action external state, and develop alert detecting equipment and wake driver up.But because 2D camera is easily subject to the impact of illumination, shade, human body complexion etc., when simultaneously 3D real scene projects to 2D plane, have the loss of range information, so carrying out can bringing when driving behavior is identified the loss of information with 2D camera, detect the problems such as inaccurate, discrimination is not high.

Summary of the invention

The present invention overcomes the weak point that prior art exists, a kind of bad steering behavior detection method based on TOF camera has been proposed, can in the situation that not disturbing driver's normal driving, effectively detect driver's bad steering behavior, improve recognition accuracy, thus the incidence of reduction accident.

Technical solution problem of the present invention adopts following technical scheme:

The feature of a kind of bad steering behavior detection method based on TOF camera of the present invention is to carry out as follows:

Step 1, acquisition depth image:

Utilize TOF camera to obtain the (t at time period T= ₁, t ₂... t _v..., t _m) interior driver's the depth image of driving behavior, choose t ₁the driving behavior depth image in moment is initial depth image; t ₂moment is to t _mthe driving behavior depth image in moment is sequence depth image;

The detection of step 2, head zone and coupling:

2.1, utilize AdaBoost algorithm to detect the human face region in described initial depth image, and utilize head rectangle frame to mark detected human face region, and obtain the center of described head rectangle frame;

2.2, header portion of institute rectangle frame is outwards expanded to A%, obtain head and expand rectangle frame;

2.3, formed with reference to three-dimensional point cloud by the pixel of human face region in described head rectangle frame; The pixel that is expanded human face region in rectangle frame by described head forms three-dimensional point cloud to be matched, utilize ICP algorithm to described three-dimensional point cloud to be matched and describedly carry out three-dimensional registration with reference to three-dimensional point cloud, obtain the final iterations of ICP algorithm, compare according to the threshold value of described final iterations and setting, if the threshold value of described final iterations >=set, be judged as bad steering behavior, otherwise be judged as normal driving behavior;

The detection and tracking of step 3, hand region:

3.1, the whole upper part of the body of driver in described initial depth image is marked, obtain rectangle frame above the waist;

3.2, using the region of depth value minimum in described upper part of the body rectangle frame as hand region, and described hand region is marked with boundary rectangle frame, the wide of described boundary rectangle frame is the wide of described head rectangle frame, and the length of described boundary rectangle frame is the twice of the length of described head rectangle frame;

3.3, utilize Kalman filtering algorithm to follow the tracks of the hand region in external rectangle frame in described sequence depth image, obtain the center of external rectangle frame in described sequence depth image;

3.4, obtain the Euclidean distance between the center of described boundary rectangle frame and the center of described head rectangle frame;

If the distance threshold of 3.5 described Euclidean distances >=set, is judged as bad steering behavior, otherwise is judged as normal driving behavior.

The feature that the present invention is based on the bad steering behavior detection method of TOF camera is also:

In described step 2.1, AdaBoost algorithm is that following steps are carried out:

Step a, to define described sequence depth image be training sample I={ (x ₁, y ₁), (x ₂, y ₂) ..., (x _i, y _i) ..., (x _n, y _n), 1≤i≤N, x _irepresent i training sample, y _i∈ { 0,1}, y _i=0 represents positive sample; y _i=1 negative sample;

Step b, the each training sample in described training sample I is carried out to the extraction of Harr feature, obtain Harr characteristic set F={f ₁, f ₂..., f _j..., f _m, f _jrepresent j Harr feature, 1≤j≤M;

Step c, utilize formula (1) to obtain j Harr feature f of i training sample _jsorter h _j(x _i):

In formula (1), p _jfor described sorter h _j(x _i) direction parameter, p _j=± 1, θ is threshold value;

Steps d, repeating step c, thus M sorter set F={h obtained ₁(x _i), h ₂(x _i) ..., h _j(x _i) ..., h _m(x _i);

Step e, utilize formula (2) to obtain j sorter h of described i training sample _j(x _i) weighting error in classification ε _j:

${\mathrm{\ϵ}}_j=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\ω}}_i|h_j\left(x_i\right)-y_i|---\left(2\right)$

In formula (2), ω _irepresent i the weight after training sample normalization;

Step f, repeating step e, thus M weighting error in classification set E={ ε obtained ₁, ε ₂..., ε _j..., ε _m;

Step g, choose the sorter of weighting error in classification minimum in described weighting error in classification set E, with the corresponding Harr feature of sorter of described weighting error in classification minimum, the human face region in described initial depth image is detected.

ICP algorithm in described step 2.3 is to carry out as follows:

Step a, definition three dimensions are R ³;

The initial coordinate that defines pixel in described three-dimensional point cloud to be matched is { P _i| P _i∈ R ³, i=1 ...., N _p, N _prepresent pixel sum in described three-dimensional point cloud to be matched;

Defining the described initial coordinate with reference to pixel in three-dimensional point cloud is { Q _i| Q _i∈ R ³, i=1 ...., N _q; N _qrepresent to state the sum with reference to pixel in three-dimensional point cloud;

Step b, definition iteration total degree are W;

While defining the k time iteration, k ∈ W; To the described initial coordinate { Q with reference to pixel in three-dimensional point cloud _i| Q _i∈ R ³, i=1 ...., N _qin each coordinate points, the coordinate of pixel in three-dimensional point cloud to be matched in all find out distance { Q _i| Q _i∈ R ³, i=1 ...., N _qin the nearest point of each coordinate points, formed with reference to the coordinate of pixel in three-dimensional point cloud by described nearest point and be

When step c, the k+1 time iteration of definition, the coordinate that utilizes formula (3) and formula (4) to obtain respectively pixel in three-dimensional point cloud to be matched is with the coordinate with reference to pixel in three-dimensional point cloud be $\left\{Q_i^{k+1}\right|Q_i^{k+1}\∈R^3,i=1,....,N_Q\};$

$P_i^{k+1}=R_O^k{P}_i^k+t^k---\left(3\right)$

$Q_i^{k+1}=R_O^k{Q}_i^k+t^k---\left(4\right)$

In formula (3) and formula (4), R _o ^k+1represent three-dimensional rotation matrix; t ^k+1represent translation vector;

Steps d, utilize formula (5) to obtain the coordinate of pixel in the three-dimensional point cloud to be matched of the k+1 time iteration coordinate with pixel in the reference three-dimensional point cloud of the k time iteration between Euclidean distance d ^k+1:

$d^{k+1}=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{\left|\right|P_i^{k+1}-Q_i^k\left|\right|}^2---\left(5\right)$

Step e, repeating step b-steps d, obtain described Euclidean distance d ^k+1minimum value time, stop iteration and obtain final iterations W'.

Kalman filtering algorithm in described step 3.3 is to carry out as follows:

1), make t ₂the hand state in moment is initialization

2), utilize formula (6) to obtain t in the boundary rectangle frame of described sequence depth image _vthe hand predicted state in moment

$X_{t_v}^{\′}=A_{t_v}{X}_{t_{v-1}}---\left(6\right)$

In formula (6), represent hand state-transition matrix, $A_{t_v}=\left[\begin{array}{cccc}1& 0& t& 0\\ 0& 1& 0& t\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ T is the interval between adjacent two moment; represent the interior t of external rectangle frame in described sequence depth image _v-1the hand state in moment;

3), utilize formula (7) to obtain t _v-1hand state in the boundary rectangle frame in moment covariance matrix

$P_{t_{v-1}}=\mathrm{cov}(X_{t_{v-1}}^{\′}-X_{t_{v-2}})---\left(7\right)$

In formula (7), represent t in the boundary rectangle frame of described sequence depth image _v-1the hand predicted state in moment; represent t in the boundary rectangle frame of described sequence depth image _v-2the hand state in moment;

4), utilize formula (8) to obtain described t _vthe hand predicted state in moment covariance matrix

$P_{t_v}^{\′}=A_{t_v}{P}_{t_{v-1}}{A}_{t_v}^T+B_{t_{v-1}}---\left(8\right)$

In formula (8), for hand state-transition matrix transposition; represent dynamic noise covariance matrix, and obey standardized normal distribution

5), utilize formula (9) to upgrade t _vhand state in the boundary rectangle frame in moment

$X_{t_v}=X_{t_v}^{\′}+K_{t_v}{H}_{t_v}{X}_{t_v}^{\′}---\left(9\right)$

In formula (9), $H_{t_v}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right]$ For hand state observation matrix, be t _vmoment Kalman filtering gain coefficient;

6), utilize formula (10) computer card Kalman Filtering gain coefficient

$K_{t_v}=P_{t_v}^{\′}{H}_{t_v}^T{(H_{t_v}{P}_{t_v}^{\′}{H_{t_v}}^T+R_{t_v})}^{-1}---\left(10\right)$

In formula (10), for hand state observation matrix transposition, represent noise covariance matrix, and obey standardized normal distribution R _k～N (0,1);

7), repeating step 1)-6), constantly update t _vhand state Xt in the boundary rectangle frame in moment _vthereby, realize the tracking of hand region in boundary rectangle frame, obtained the center of external rectangle frame in described sequence depth image by described hand region.

Three-dimensional rotation matrix R in described step c _o ^k+1with translation vector t ^k+1obtain as follows:

1) coordinate of pixel in the three-dimensional point cloud to be matched while, utilizing formula (11) and (12) to obtain the k+1 time iteration barycenter μ _pwith the coordinate with reference to pixel in three-dimensional point cloud barycenter μ _q:

${\mathrm{\μ}}_P=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{P}_i^{k+1}---\left(11\right)$

${\mathrm{\μ}}_Q=\frac{1}{N_T}\underset{i=1}{\overset{N_Q}{\mathrm{\Σ}}}{Q}_i^{k+1}---\left(12\right)$

2), utilize formula (13) and (14) to obtain respectively the coordinate of pixel in described three-dimensional point cloud to be matched with respect to barycenter μ _ptranslation with the described coordinate with reference to pixel in three-dimensional point cloud with respect to barycenter μ _qtranslation

$m_i^P=P_i^{k+1}-{\mathrm{\μ}}_P---\left(13\right)$

$m_i^Q=Q_i^{k+1}-{\mathrm{\μ}}_Q---\left(14\right)$

3), utilize formula (15) to obtain described translation and translation between correlation matrix K _{α β}:

$K_{\mathrm{\α\β}}=\frac{1}{N}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{m}_i^P{\left(m_i^Q\right)}^T---\left(15\right)$

In formula (15), α, β=1,2,3;

4), utilize formula (16) to obtain correlation matrix K _{α β}the four-dimensional symmetric matrix K constructing:

$K=\left[\begin{array}{cccc}{K}_{11}+K_{12}+K_{13}& K_{32}-K_{23}& K_{13}-K_{31}& K_{21}-K_{12}\\ K_{32}-K_{23}& K_{11}-K_{22}-K_{33}& K_{12}+K_{21}& K_{13}+K_{31}\\ K_{13}-K_{31}& K_{12}+K_{21}& -K_{11}+K_{22}-K_{33}& K_{23}+K_{32}\\ K_{21}-K_{12}& K_{31}+K_{13}& K_{32}+K_{23}& -K_{11}-K_{22}+K_{33}\end{array}\right]---\left(16\right)$

5), utilize described four-dimensional symmetric matrix K to obtain maximum characteristic root; And obtain unit character vector according to described maximum characteristic root _q=[ _q0, _q1, _q2, _q3] ^t;

6), utilize formula (17) to obtain antisymmetric matrix K (q):

$K\left(q\right)=\left[\begin{array}{ccc}0& -q_2& q_1\\ q_2& 0& -q_0\\ -q_1& q_0& 0\end{array}\right]---\left(17\right)$

7), utilize formula (18) to obtain three-dimensional rotation matrix

$R_O^{k+1}=(q_3^2+q^{T}q+2q_{3}K\left(q\right))---\left(18\right)$

8), utilize formula (19) to obtain translation vector t ^k+1:

$t^{k+1}={\mathrm{\μ}}_Q-R_O^{k+1}{\mathrm{\μ}}_P---\left(19\right).$

Compared with prior art, beneficial effect of the present invention is embodied in:

1, the present invention has realized the tracking of driver's head detection and hand state by AdaBoost algorithm and Kalman filtering, and complete the three-dimensional point cloud registration of current depth map and reference depth figure with ICP algorithm, having solved 2D camera of the prior art is easily subject to illumination, shade, human body complexion and driver to wear the problem that dress ornament color etc. affects, thereby the accuracy of detection that has improved bad steering behavior, has reduced accident rate;

2, the present invention utilizes driver's recognition methods of TOF range information, and because of TOF camera, to obtain depth image speed fast, and frame frequency can reach 40fps, has the advantages that real-time is good;

3, AdaBoost algorithm of the present invention is a kind of high-precision sorter, the detection error that can not bring because of driver's head movement and to subsequent detection result produce tremendous influence, and because of the framework of algorithm simple, it goes without doing Feature Selection, thereby solve the speed issue detecting, there is good recognition effect.

4, ICP algorithm of the present invention needn't be cut apart and feature extraction pending point set, and in the situation that selecting accurately initial position, can obtain good Algorithm Convergence, aspect the registration of three-dimensional point cloud, can obtain point-device registration effect.

5, Kalman filtering algorithm of the present invention is the optimum criterion taking least mean-square error as estimating, fairly simple on mathematics framework, and be optimum linearity recursive filtering method, can effectively solve the target following based on irregular hand continuity chart picture, have advantages of that calculated amount is little.

Brief description of the drawings

Fig. 1 detection system schematic diagram of the present invention;

Fig. 2 detection method process flow diagram of the present invention;

Number in the figure: 1 head rectangle frame; 2 heads expand rectangle frame; 3 upper part of the body rectangle frames; 4 boundary rectangle frames; 5TOF camera.

Embodiment

In the present embodiment, a kind of bad steering behavior detection method based on TOF camera is the depth image of the TOF camera Real-time Obtaining driver by being arranged on driver's oblique upper in car, by virtual face rectangle frame, utilize ICP (iterative closest point) algorithm that present frame depth image and reference picture are carried out to three-dimensional registration, judge whether to have occurred bad steering behavior according to iterations; Adopt background subtraction to detect the hand line trace of going forward side by side, according to hand position and virtual face rectangle frame position judgment, whether bad steering behavior has occurred, concrete is to carry out according to the following steps:

Step 1, acquisition depth image:

TOF camera is arranged on to the oblique upper of driver in pilothouse, and connects the microprocessor that can report to the police and be arranged on vehicle console, utilize TOF camera to obtain the (t at time period T= ₁, t ₂... t _v..., t _m) interior driver's the depth image of driving behavior, choose t ₁the driving behavior depth image in moment is initial depth image; t ₂moment is to t _mthe driving behavior depth image in moment is sequence depth image;

The detection of step 2, head zone and coupling:

2.1, utilize AdaBoost algorithm to detect the human face region in initial depth image, and utilize head rectangle frame to mark detected human face region, and obtain the center of head rectangle frame;

Concrete, AdaBoost algorithm carries out as follows:

A), defined nucleotide sequence depth image is training sample I={ (x ₁, y ₁), (x ₂, y ₂) ..., (x _i, y _i) ..., (x _n, y _n), 1≤i≤N, x _irepresent i training sample, y _i∈ { 0,1}, y _i=0 represents positive sample; y _i=1 negative sample;

B), the each training sample in training sample I is carried out to the extraction of Harr feature, acquisition Harr characteristic set F={f ₁, f ₂..., f _j..., f _m, f _jrepresent j Harr feature, 1≤j≤M;

C), utilize formula (1) to obtain j Harr feature f of i training sample _jsorter h _j(x _i):

In formula (1), p _jfor sorter h _j(x _i) direction parameter, p _j=± 1, θ is threshold value, is taken as in this example 0.15;

D), repeating step c), thereby obtain M sorter set F={h ₁(x _i), h ₂(x _i) ..., h _j(x _i) ..., h _m(x _i);

E), utilize formula (2) to obtain j sorter h of i training sample _j(x _i) weighting error in classification ε _j:

${\mathrm{\ϵ}}_j=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\ω}}_i|h_j\left(x_i\right)-y_i|---\left(2\right)$

In formula (2), ω _irepresent i the weight after training sample normalization;

F), repeating step e), thereby obtain M weighting error in classification set E={ ε ₁, ε ₂..., ε _j..., ε _m;

G), choose the sorter of weighting error in classification minimum in weighting error in classification set E, with the corresponding Harr feature of sorter of weighting error in classification minimum, the human face region in initial depth image is detected.

2.2, header portion of institute rectangle frame is outwards expanded to A%, here expand reason be driver in normal driving, have small head movement, within the scope of this, not belong to bad steering behavior, A gets 10 in this example, obtain head expand rectangle frame;

2.3, formed with reference to three-dimensional point cloud by the pixel of human face region in head rectangle frame; Wherein, three-dimensional point cloud is the set of one group of pixel, the pixel that is expanded human face region in rectangle frame by head forms three-dimensional point cloud to be matched, utilize ICP algorithm, ICP algorithm is closest approach iterative algorithm, to three-dimensional point cloud to be matched with carry out three-dimensional registration with reference to three-dimensional point cloud, obtain the final iterations of ICP algorithm, compare according to the threshold value of final iterations and setting, if the threshold value of final iterations >=set, the threshold value setting is in this example 50, is judged as bad steering behavior, otherwise is judged as normal driving behavior; Here judge according to iterations according to being, ICP algorithm is a kind of Iterative matching algorithm, in matching process, need to carry out ceaselessly iteration, if the matching degree height of two width driving behavior images final iterations is few, it is now normal driving, if the low final iterations of the large matching degree of two width driving behavior image difference is many, is now bad steering, so select to judge according to the coupling iterations of driver's driving behavior image whether driver bad steering has occurred.

Concrete, ICP algorithm is to carry out as follows:

H), definition three dimensions is R ³;

The initial coordinate that defines pixel in three-dimensional point cloud to be matched is { P _i| P _i∈ R ³, i=1 ...., N _p, N _prepresent pixel sum in three-dimensional point cloud to be matched;

Definition is { Q with reference to the initial coordinate of pixel in three-dimensional point cloud _i| Q _i∈ R ³, i=1 ...., N _q; N _qrepresent to state the sum with reference to pixel in three-dimensional point cloud;

I), definition iteration total degree is W;

While defining the k time iteration, k ∈ W; To the initial coordinate { Q with reference to pixel in three-dimensional point cloud _i| Q _i∈ R ³, i=1 ...., N _qin each coordinate points, the coordinate of pixel in three-dimensional point cloud to be matched in all find out apart from { Q _i| Q _i∈ R ³, i=1 ...., N _qin the nearest point of each coordinate points, these nearest points form and with reference to the coordinate of pixel in three-dimensional point cloud are $\left\{Q_i^k\right|Q_i^k\∈R^3,i=1,....,N_Q\};$

J), definition is when the k+1 time iteration, the coordinate that utilizes formula (3) and formula (4) to obtain respectively pixel in three-dimensional point cloud to be matched is $\left\{P_i^{k+1}\right|P_i^{k+1}\∈R^3,i=1,....,N_P\}$ With the coordinate with reference to pixel in three-dimensional point cloud be $\left\{Q_i^{k+1}\right|Q_i^{k+1}\∈R^3,i=1,....,N_Q\};$

$P_i^{k+1}=R_O^k{P}_i^k+t^k---\left(3\right)$

$Q_i^{k+1}=R_O^k{Q}_i^k+t^k---\left(4\right)$

In formula (3) and formula (4), R _o ^k+1represent three-dimensional rotation matrix; t ^k+1represent translation vector;

Concrete, three-dimensional rotation matrix R _o ^k+1with translation vector t ^k+1obtain as follows:

J1) coordinate of pixel in the three-dimensional point cloud to be matched while, utilizing formula (5) and (6) to obtain the k+1 time iteration barycenter μ _pwith the coordinate with reference to pixel in three-dimensional point cloud barycenter μ _q:

${\mathrm{\μ}}_P=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{P}_i^{k+1}---\left(5\right)$

${\mathrm{\μ}}_Q=\frac{1}{N_T}\underset{i=1}{\overset{N_Q}{\mathrm{\Σ}}}{Q}_i^{k+1}---\left(6\right)$

J2), utilize formula (7) and (8) to obtain respectively the coordinate of pixel in three-dimensional point cloud to be matched with respect to barycenter μ _ptranslation with the coordinate with reference to pixel in three-dimensional point cloud with respect to barycenter μ _qtranslation

$m_i^P=P_i^{k+1}-{\mathrm{\μ}}_P---\left(7\right)$

$m_i^Q=Q_i^{k+1}-{\mathrm{\μ}}_Q---\left(8\right)$

J3), utilize formula (9) to obtain translation and translation between correlation matrix K _{α β}:

$K_{\mathrm{\α\β}}=\frac{1}{N}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{m}_i^P{\left(m_i^Q\right)}^T---\left(9\right)$

In formula (9), α, β=1,2,3;

J4), utilize formula (10) to obtain correlation matrix K _{α β}the four-dimensional symmetric matrix K constructing:

$K=\left[\begin{array}{cccc}{K}_{11}+K_{12}+K_{13}& K_{32}-K_{23}& K_{13}-K_{31}& K_{21}-K_{12}\\ K_{32}-K_{23}& K_{11}-K_{22}-K_{33}& K_{12}+K_{21}& K_{13}+K_{31}\\ K_{13}-K_{31}& K_{12}+K_{21}& -K_{11}+K_{22}-K_{33}& K_{23}+K_{32}\\ K_{21}-K_{12}& K_{31}+K_{13}& K_{32}+K_{23}& -K_{11}-K_{22}+K_{33}\end{array}\right]---\left(10\right)$

J5), utilize four-dimensional symmetric matrix K to obtain maximum characteristic root; And obtain unit character vector q=[q according to maximum characteristic root ₀, q ₁, q ₂, q ₃] ^t;

J6), utilize formula (11) to obtain antisymmetric matrix K (q):

$K\left(q\right)=\left[\begin{array}{ccc}0& -q_2& q_1\\ q_2& 0& -q_0\\ -q_1& q_0& 0\end{array}\right]---\left(11\right)$

J7), utilize formula (12) to obtain three-dimensional rotation matrix

$R_O^{k+1}=(q_3^2+q^{T}q+2q_{3}K\left(q\right))---\left(12\right)$

J8), utilize formula (13) to obtain translation vector t ^k+1:

$t^{k+1}={\mathrm{\μ}}_Q-R_O^{k+1}{\mathrm{\μ}}_P---\left(13\right)$

K), utilize formula (14) to obtain the coordinate of pixel in the three-dimensional point cloud to be matched of the k+1 time iteration coordinate with pixel in the reference three-dimensional point cloud of the k time iteration between Euclidean distance d ^k+1:

$d^{k+1}=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{\left|\right|P_i^{k+1}-Q_i^k\left|\right|}^2---\left(14\right)$

L), repeating step i)-step k), obtain Euclidean distance d ^k+1minimum value time, stop iteration and obtain final iterations W'.

The detection and tracking of step 3, hand region:

3.1, the whole upper part of the body of driver in initial depth image is marked, obtain rectangle frame above the waist;

3.2, using the region of depth value minimum in upper part of the body rectangle frame as hand region, because hand is near apart from TOF camera, so its depth value minimum, and hand region is marked with boundary rectangle frame, the wide of boundary rectangle frame is the wide of head rectangle frame, and the length of boundary rectangle frame is the twice of the length of head rectangle frame; Because in general, people's hand and head size are about the same, while considering normal driving, hand has certain scope of activities, so boundary rectangle frame wide is made as to the wide of head rectangle frame here, the length of boundary rectangle frame is made as the twice of the length of head rectangle frame.

3.3, utilize Kalman filtering algorithm to follow the tracks of the hand region in external rectangle frame in sequence depth image, obtain the center of external rectangle frame in sequence depth image; Obtain the center of boundary rectangle frame according to geometric relationship.

Concrete, Kalman filtering algorithm is to carry out as follows:

M), make t ₂the hand state in moment is initialization

N), utilize formula (15) to obtain t in the boundary rectangle frame of sequence depth image _vthe hand predicted state in moment

$X_{t_v}^{\′}=A_{t_v}{X}_{t_{v-1}}---\left(15\right)$

In formula (15), represent hand state-transition matrix, $A_{t_v}=\left[\begin{array}{cccc}1& 0& t& 0\\ 0& 1& 0& t\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ T is the interval between adjacent two moment; T gets 0.2s in this example, represent the interior t of external rectangle frame in sequence depth image _v-1the hand state in moment;

O), utilize formula (16) to obtain t _v-1hand state in the boundary rectangle frame in moment covariance matrix

$P_{t_{v-1}}=\mathrm{cov}(X_{t_{v-1}}^{\′}-X_{t_{v-2}})---\left(16\right)$

In formula (16), t in the boundary rectangle frame of expression sequence depth image _v-1the hand predicted state in moment; t in the boundary rectangle frame of expression sequence depth image _v-2the hand state in moment;

P), utilize formula (17) to obtain t _vthe hand predicted state in moment covariance matrix

$P_{t_v}^{\′}=A_{t_v}{P}_{t_{v-1}}{A}_{t_v}^T+B_{t_{v-1}}---\left(17\right)$

In formula (17), for hand state-transition matrix transposition; represent dynamic noise covariance matrix, and obey standardized normal distribution

Q), utilize formula (18) to upgrade t _vhand state in the boundary rectangle frame in moment

$X_{t_v}=X_{t_v}^{\′}+K_{t_v}{H}_{t_v}{X}_{t_v}^{\′}---\left(18\right)$

In formula (18), $H_{t_v}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right]$ For hand state observation matrix, be t _vmoment Kalman filtering gain coefficient;

R), utilize formula (19) computer card Kalman Filtering gain coefficient

$K_{t_v}=P_{t_v}^{\′}{H}_{t_v}^T{(H_{t_v}{P}_{t_v}^{\′}{H_{t_v}}^T+R_{t_v})}^{-1}---\left(19\right)$

In formula (19), for hand state observation matrix transposition, represent noise covariance matrix, and obey standardized normal distribution R _k～N (0,1);

S), repeating step m)-q), constantly update t _vhand state in the boundary rectangle frame in moment thereby realize the tracking of hand region in boundary rectangle frame, obtained the center of external rectangle frame in sequence depth image by hand region.

3.4, the Euclidean distance between center and the center of head rectangle frame of acquisition boundary rectangle frame;

If the distance threshold of 3.5 Euclidean distances >=set, the distance threshold setting is in this example 40cm, is judged as bad steering behavior, otherwise is judged as normal driving behavior.

Claims (5)

1. the bad steering behavior detection method based on TOF camera, is characterized in that carrying out as follows:

Step 1, acquisition depth image:

Utilize TOF camera to obtain the (t at time period T= ₁, t ₂... t _v..., t _m) interior driver's the depth image of driving behavior, choose t ₁the driving behavior depth image in moment is initial depth image; t ₂moment is to t _mthe driving behavior depth image in moment is sequence depth image;

The detection of step 2, head zone and coupling:

2.1, utilize AdaBoost algorithm to detect the human face region in described initial depth image, and utilize head rectangle frame to mark detected human face region, and obtain the center of described head rectangle frame;

2.2, header portion of institute rectangle frame is outwards expanded to A%, obtain head and expand rectangle frame;

2.3, formed with reference to three-dimensional point cloud by the pixel of human face region in described head rectangle frame; The pixel that is expanded human face region in rectangle frame by described head forms three-dimensional point cloud to be matched, utilize ICP algorithm to described three-dimensional point cloud to be matched and describedly carry out three-dimensional registration with reference to three-dimensional point cloud, obtain the final iterations of ICP algorithm, compare according to the threshold value of described final iterations and setting, if the threshold value of described final iterations >=set, be judged as bad steering behavior, otherwise be judged as normal driving behavior;

The detection and tracking of step 3, hand region:

3.1, the whole upper part of the body of driver in described initial depth image is marked, obtain rectangle frame above the waist;

3.2, using the region of depth value minimum in described upper part of the body rectangle frame as hand region, and described hand region is marked with boundary rectangle frame, the wide of described boundary rectangle frame is the wide of described head rectangle frame, and the length of described boundary rectangle frame is the twice of the length of described head rectangle frame;

3.3, utilize Kalman filtering algorithm to follow the tracks of the hand region in external rectangle frame in described sequence depth image, obtain the center of external rectangle frame in described sequence depth image;

3.4, obtain the Euclidean distance between the center of described boundary rectangle frame and the center of described head rectangle frame;

If the distance threshold of 3.5 described Euclidean distances >=set, is judged as bad steering behavior, otherwise is judged as normal driving behavior.

2. the bad steering behavior detection method based on TOF camera according to claim 1, is characterized in that: in described step 2.1, AdaBoost algorithm is that following steps are carried out:

Step a, to define described sequence depth image be training sample I={ (x ₁, y ₁), (x ₂, y ₂) ..., (x _i, y _i) ..., (x _n, y _n), 1≤i≤N, x _irepresent i training sample, y _i∈ { 0,1}, y _i=0 represents positive sample; y _i=1 negative sample;

Step b, the each training sample in described training sample I is carried out to the extraction of Harr feature, obtain Harr characteristic set F={f ₁, f ₂..., f _j..., f _m, f _jrepresent j Harr feature, 1≤j≤M;

Step c, utilize formula (1) to obtain j Harr feature f of i training sample _jsorter h _j(x _i):

In formula (1), p _jfor described sorter h _j(x _i) direction parameter, p _j=± 1, θ is threshold value;

Steps d, repeating step c, thus M sorter set F={h obtained ₁(x _i), h ₂(x _i) ..., h _j(x _i) ..., h _m(x _i);

Step e, utilize formula (2) to obtain j sorter h of described i training sample _j(x _i) weighting error in classification ε _j:

${\mathrm{\ϵ}}_j=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\ω}}_i|h_j\left(x_i\right)-y_i|---\left(2\right)$

In formula (2), ω _irepresent i the weight after training sample normalization;

Step f, repeating step e, thus M weighting error in classification set E={ ε obtained ₁, ε ₂..., ε _j..., ε _m;

Step g, choose the sorter of weighting error in classification minimum in described weighting error in classification set E, with the corresponding Harr feature of sorter of described weighting error in classification minimum, the human face region in described initial depth image is detected.

3. the bad steering behavior detection method based on TOF camera according to claim 1, is characterized in that: the ICP algorithm in described step 2.3 is to carry out as follows:

Step a, definition three dimensions are R ³;

The initial coordinate that defines pixel in described three-dimensional point cloud to be matched is { P _i| P _i∈ R ³, i=1 ...., N _p, N _prepresent pixel sum in described three-dimensional point cloud to be matched;

Defining the described initial coordinate with reference to pixel in three-dimensional point cloud is { Q _i| Q _i∈ R ³, i=1 ...., N _q; N _qrepresent to state the sum with reference to pixel in three-dimensional point cloud;

Step b, definition iteration total degree are W;

While defining the k time iteration, k ∈ W; To the described initial coordinate { Q with reference to pixel in three-dimensional point cloud _i| Q _i∈ R ³, i=1 ...., N _qin each coordinate points, the coordinate of pixel in three-dimensional point cloud to be matched in all find out distance { Q _i| Q _i∈ R ³, i=1 ...., N _qin the nearest point of each coordinate points, formed with reference to the coordinate of pixel in three-dimensional point cloud by described nearest point and be

When step c, the k+1 time iteration of definition, the coordinate that utilizes formula (3) and formula (4) to obtain respectively pixel in three-dimensional point cloud to be matched is with the coordinate with reference to pixel in three-dimensional point cloud be $\left\{Q_i^{k+1}\right|Q_i^{k+1}\∈R^3,i=1,....,N_Q\};$

$P_i^{k+1}=R_O^k{P}_i^k+t^k---\left(3\right)$

$Q_i^{k+1}=R_O^k{Q}_i^k+t^k---\left(4\right)$

In formula (3) and formula (4), R _o ^k+1represent three-dimensional rotation matrix; t ^k+1represent translation vector;

Steps d, utilize formula (5) to obtain the coordinate of pixel in the three-dimensional point cloud to be matched of the k+1 time iteration coordinate with pixel in the reference three-dimensional point cloud of the k time iteration between Euclidean distance d ^k+1:

$d^{k+1}=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{\left|\right|P_i^{k+1}-Q_i^k\left|\right|}^2---\left(5\right)$

Step e, repeating step b-steps d, obtain described Euclidean distance d ^k+1minimum value time, stop iteration and obtain final iterations W'.

4. the bad steering behavior detection method based on TOF camera according to claim 1, is characterized in that: the Kalman filtering algorithm in described step 3.3 is to carry out as follows:

1), make t ₂the hand state in moment is initialization

2), utilize formula (6) to obtain t in the boundary rectangle frame of described sequence depth image _vthe hand predicted state in moment

$X_{t_v}^{\′}=A_{t_v}{X}_{t_{v-1}}---\left(6\right)$

In formula (6), represent hand state-transition matrix, $A_{t_v}=\left[\begin{array}{cccc}1& 0& t& 0\\ 0& 1& 0& t\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ T is the interval between adjacent two moment; represent the interior t of external rectangle frame in described sequence depth image _v-1the hand state in moment;

3), utilize formula (7) to obtain t _v-1hand state in the boundary rectangle frame in moment covariance matrix

$P_{t_{v-1}}=\mathrm{cov}(X_{t_{v-1}}^{\′}-X_{t_{v-2}})---\left(7\right)$

In formula (7), represent t in the boundary rectangle frame of described sequence depth image _v-1the hand predicted state in moment; represent t in the boundary rectangle frame of described sequence depth image _v-2the hand state in moment;

4), utilize formula (8) to obtain described t _vthe hand predicted state in moment covariance matrix

$P_{t_v}^{\′}=A_{t_v}{P}_{t_{v-1}}{A}_{t_v}^T+B_{t_{v-1}}---\left(8\right)$

In formula (8), for hand state-transition matrix transposition; represent dynamic noise covariance matrix, and obey standardized normal distribution

5), utilize formula (9) to upgrade t _vhand state in the boundary rectangle frame in moment

$X_{t_v}=X_{t_v}^{\′}+K_{t_v}{H}_{t_v}{X}_{t_v}^{\′}---\left(9\right)$

In formula (9), $H_{t_v}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right]$ For hand state observation matrix, be t _vmoment Kalman filtering gain coefficient;

6), utilize formula (10) computer card Kalman Filtering gain coefficient

$K_{t_v}=P_{t_v}^{\′}{H}_{t_v}^T{(H_{t_v}{P}_{t_v}^{\′}{H_{t_v}}^T+R_{t_v})}^{-1}---\left(10\right)$

In formula (10), for hand state observation matrix transposition, represent noise covariance matrix, and obey standardized normal distribution R _k～N (0,1);

7), repeating step 1)-6), constantly update t _vhand state in the boundary rectangle frame in moment thereby realize the tracking of hand region in boundary rectangle frame, obtained the center of external rectangle frame in described sequence depth image by described hand region.

5. the bad steering behavior detection method based on TOF camera according to claim 3, is characterized in that: three-dimensional rotation matrix R in described step c _o ^k+1with translation vector t ^k+1obtain as follows:

1) coordinate of pixel in the three-dimensional point cloud to be matched while, utilizing formula (11) and (12) to obtain the k+1 time iteration barycenter μ _pwith the coordinate with reference to pixel in three-dimensional point cloud barycenter μ _q:

${\mathrm{\μ}}_P=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{P}_i^{k+1}---\left(11\right)$

${\mathrm{\μ}}_Q=\frac{1}{N_T}\underset{i=1}{\overset{N_Q}{\mathrm{\Σ}}}{Q}_i^{k+1}---\left(12\right)$

2), utilize formula (13) and (14) to obtain respectively the coordinate of pixel in described three-dimensional point cloud to be matched with respect to barycenter μ _ptranslation with the described coordinate with reference to pixel in three-dimensional point cloud with respect to barycenter μ _qtranslation

$m_i^P=P_i^{k+1}-{\mathrm{\μ}}_P---\left(13\right)$

$m_i^Q=Q_i^{k+1}-{\mathrm{\μ}}_Q---\left(14\right)$

3), utilize formula (15) to obtain described translation and translation between correlation matrix K _{α β}:

$K_{\mathrm{\α\β}}=\frac{1}{N}\underset{i=1}{\overset{N_P}{\mathrm{\Σ}}}{m}_i^P{\left(m_i^Q\right)}^T---\left(15\right)$

In formula (15), α, β=1,2,3;

4), utilize formula (16) to obtain correlation matrix K _{α β}the four-dimensional symmetric matrix K constructing:

$K=\left[\begin{array}{cccc}{K}_{11}+K_{12}+K_{13}& K_{32}-K_{23}& K_{13}-K_{31}& K_{21}-K_{12}\\ K_{32}-K_{23}& K_{11}-K_{22}-K_{33}& K_{12}+K_{21}& K_{13}+K_{31}\\ K_{13}-K_{31}& K_{12}+K_{21}& -K_{11}+K_{22}-K_{33}& K_{23}+K_{32}\\ K_{21}-K_{12}& K_{31}+K_{13}& K_{32}+K_{23}& -K_{11}-K_{22}+K_{33}\end{array}\right]---\left(16\right)$

5), utilize described four-dimensional symmetric matrix K to obtain maximum characteristic root; And obtain unit character vector q=[q according to described maximum characteristic root ₀, q ₁, q ₂, q ₃] ^t;

6), utilize formula (17) to obtain antisymmetric matrix K (q):

$K\left(q\right)=\left[\begin{array}{ccc}0& -q_2& q_1\\ q_2& 0& -q_0\\ -q_1& q_0& 0\end{array}\right]---\left(17\right)$

7), utilize formula (18) to obtain three-dimensional rotation matrix

$R_O^{k+1}=(q_3^2+q^{T}q+2q_{3}K\left(q\right))---\left(18\right)$

8), utilize formula (19) to obtain translation vector t ^k+1:

$t^{k+1}={\mathrm{\μ}}_Q-R_O^{k+1}{\mathrm{\μ}}_P---\left(19\right).$