CN106503615A - Indoor human body detecting and tracking and identification system based on multisensor - Google Patents

Abstract

Indoor human body detecting and tracking and identification system based on multisensor, the system completes the Primary Location of human body by pyroelectric infrared sensor, the scope that camera is moved to human body appearance by steering wheel, image information in the range of this is gathered by camera and image information is transferred in computer, computer completes the correlation computations of human testing and control steering wheel to become to making camera and mobile platform tracking human body.Computer is mated so that it is determined that being detected the identity of people by collecting image information and background information.Mainly for detection of under indoor environment, whether someone invades and helps indoor moving service robot to determine destination service people.System mainly controls camera steering wheel head by pyroelectric sensor makes which turn to the scope of physical activity, using visual information detect and track human body.Identification is carried out with the method for Adaboost and Principle components analysis after detecting human body.

Description

Indoor human body detecting and tracking and identification system based on multisensor

Technical field

The present invention relates to the technical field of indoor human body detection, specifically a kind of by pyroelectric infrared sensor and Monocular cam detects and tracks indoor sport human body and complete the system of identification.

Background technology

With the development of computer vision technique, human motion analysis problem has been a great concern, and human testing Tracking and the important component part that identification is the human motion analysis based on computer vision again.It in man-machine interaction, regard Frequency monitoring, the field such as intelligent vehicle system, virtual reality have a wide range of applications background and economic worth.And indoor human body detection with Track is even more one of which important application with technology of identification, such as, indoors in interactive environment, the technology can help Information Mobile Service Robot determines destination service people, while children, the elderly and patient with disabilities can also be guarded.On the other hand, In security protection and monitoring field, indoor human body detecting and tracking has huge use value with technology of identification, by movement human Detection can be monitored to indoor environment, enable monitoring system automatic identification people and risk object is reported to the police.

Traditional human body detecting method is broadly divided into Stereo Vision, profile testing method, template matching method, human body The method of model method, the method for Gait Recognition, wavelet analysis and neutral net, is more by two of which or more kinds of The method for combining.But as the human body for moving is under the influence of the change of angle and background, traditional human body detecting method exists Intended effect is not reached in accuracy.Human testing tracking in recent years based on machine learning and technology of identification are with good Real-time and robustness be target, obtained developing faster.The correlation theory of machine learning and Technology application are arrived each In link, traditional detecting and tracking and technology of identification is preferably instead of, its accuracy of detection is high, while can also meet part in real time The demand of property.So the human body detecting method with machine learning is widely applied, cognitive phase is mainly from target Human body is distinguished, is mainly mated by the special characteristic of human body, or by neutral net, SVM (SVMs), many Layer perceptron etc. method is judging whether have pedestrian to occur in target area.

On the other hand, in order to obtain the information of human body target, current study hotspot is in the static feelings of monocular cam Human testing and the tracking that moves is completed under condition.But in practical situations both, as monocular cam wide angular range is limited, if fortune Dynamic human body removes the angular field of view of camera, and system cannot then detect the human body of motion.Therefore how by controlling monocular It is still a problem demanding prompt solution that the motion of camera comes real-time detection and the human body of tracking indoor sport.When detecting human body After target, how to distinguish that identity is a highly important task, traditional method carries out face go-on-go to human body image Survey, the part for detecting is carried out identification, but traditional method is not referred to if do not detected in human body image How face go-on-go examining system recognizes human body identity.

Content of the invention

The main object of the present invention is to propose a kind of indoor human body detection and identification system, and the system can be with sensing chamber Under interior environment, whether someone invades and helps indoor moving service robot to determine destination service people, so which acts predominantly on inspection Survey under one or the indoor environment of a few peoples.Which mainly solves three technical problems：

1. static camera cannot determine mobile human body place room area, only when human motion is to camera visual angle In the range of, camera could catch human body information.

2., when human motion goes out in camera angular field of view, the image information that camera is obtained cannot continue to track indoor shifting Dynamic human body.

3. when information behaviour body side surface in human body picture, when system does not detect human face region from image information such as What confirms target identity information.

Indoor human body detecting and tracking and identification system based on multisensor, the system are sensed by rpyroelectric infrared Device completes the Primary Location of human body, the scope that camera is moved to human body appearance by steering wheel, gathers the model by camera Enclose interior image information and image information is transferred in computer, computer completes the correlation computations of human testing control flaps Machine becomes to making camera and mobile platform tracking human body.Computer by collect image information and background information mated so as to Determine the identity of detection people.Its system architecture diagram is as shown in Figure 1.

The workflow diagram of the inventive method is as shown in Fig. 2 specific workflow is as follows：

The Preliminary detection of S1 human bodies and the Primary Location of camera

The system catches human infrared radiation using pyroelectric infrared sensor and is converted into faint voltage signal Principle come detect interior whether have human body, the pyroelectric infrared sensor induction range that system is adopted for 6 meters, sense angle be 100 degree.Four pyroelectric sensors are distributed in the due west of camera, due south, due east, positive north four direction, detection angles will Indoor 360 degree scope is covered, when pyroelectric sensor detects human body signal, camera steering wheel head is just turned to In the range of being somebody's turn to do, its structure is as shown in Figure 3.

S2 completes human testing using image

The system using HOG+SVM method to video sequence in human body detect that step is as follows：

The making of S2.1 Sample Storehouses

Used as positive sample, the system is adopted in INRIA as shown in Figure 4 static state pedestrian's Test databases collection human sample Picture as training positive sample, using the negative sample in INRIA databases as shown in Figure 5 and treated indoor unmanned environment Under picture as training negative sample.

The selection of the parameter of S2.2HOG feature extractions

Using the HOG that OpenCv is carried, this method detects that built-in function, corresponding parameter are set to：Detection window is 64*64, The size of cell factory is 16*16 pixels, and block sliding step is 16, and gradient direction is quantified as 9, therefore the HOG of piece image is special Dimension is levied for 4*9*3*3=324.The standardized method of characteristic block selects L2-Hys, threshold value to be 0.2 and carry out Gamma corrections.

S2.3 carries out SVM training

The HOG features of all positive negative samples are extracted, and aligns negative sample and give label, positive sample is labeled as 1, by negative sample Originally 0 is labeled as, then the HOG features of positive negative sample and label is all input in SVM training aids and is trained, just obtain one Human body grader.

S3 control cameras rotate tracking human body

As shown in fig. 6, human body is marked its in-scope with square frame in image by system, every 25 frame is detected once, then The angle of steering wheel left-right rotation is calculated by the difference of center pixel coordinate in calculation block and whole sub-picture center pixel coordinate Degree and the angle for rotating upwardly and downwardly, so that ensure that human body is occurred in camera angular field of view all the time.On the other hand, by video camera The intrinsic parameter and distortion parameter for obtaining camera is demarcated, so that human body reality is extrapolated according to the coordinate of human body block diagram center pixel Physical coordinates, make camera be maintained in given area with human body so as to adjust base plate electric machine.

S4 identifications

It is secondary independent that the identification procedure of the system out becomes independent for the human body block diagram that detects cutting into one first Image, carry out Face datection using VIola-Jones graders on the sub-picture, its process approximately as：

S4.1 accelerates the calculating of the value of 45 degree of rotations of rectangular image area or rectangular area, the image using integral image Structure is used to the calculating for accelerating class Haar input feature vector, its feature that fully enters as Adaboost graders, class Harr Feature is as shown in Figure 7.

S4.2 creates face with non-face grader node using Adaboost algorithm.

S4.3 constitutes Weak Classifier node one node of screening type cascade, and which is as shown in Figure 8.Each node D in figure_j Comprising one group of decision tree that is trained using Like-Fenton Oxidation either with or without face.Node is arranged from simple to complex, so can be most Littleization refuses the amount of calculation during simple region of image.First classifiers be optimum, can by the image-region comprising face, Some images not comprising face are allowed to pass through simultaneously；Second classifiers suboptimum grader, and have relatively low reject rate；So Afterwards by that analogy, as long as image-region has passed through whole cascade, then it is assumed that there is face the inside, and face block diagram is rectified out.

Due to the limited amount system of house person, the quantity of the Sample Storehouse of old friend's face is also limited, using principal component analysis Method PCA carries out identification, and all of training data is projected to PCA subspaces, and image projection to be identified is empty to PCA Between, find training data projection after vector sum images to be recognized projection after vectorial immediate that, training data such as Fig. 9 Shown.

The system using the method for human testing be not the information that can capture face front every time, if cutting Human body block diagram out only has the side-information of head, back side information or the information only below head, then skip The process of Adaboost Face datections, directly carries out identity differentiation using the method for PCA, only need to change data set, number According to collection as shown in Figure 10.

Description of the drawings

Fig. 1 system architecture diagrams.

Fig. 2 working-flow figures.

Fig. 3 pyroelectric sensor distributed architecture figures.

Fig. 4 human testing positive sample examples.

Fig. 5 human testing negative sample examples.

The human body block diagram example that Fig. 6 is detected.

Fig. 7 class Harr features.

The screening type cascade that uses in Fig. 8 Viola-Jones graders.

Fig. 9 recognition of face PCA training dataset examples.

Figure 10 non-face part PCA training dataset examples.

Specific embodiment

In conjunction with Fig. 1 to Fig. 9, the system is described in detail.

Fig. 1 illustrates system architecture diagram, and Arduino reads pyroelectric sensor signal to be responsible for completing initial alignment, will Camera moves to the scope of physical activity, then all visual informations in camera collection current visual angle, by video sequence Pass in the computer of system, computer detects the human body in video by HOG+SVM algorithms, then adjust camera steering wheel and Base plate electric machine makes human body be occurred in the angular field of view of camera all the time, the picture profit of the human body block diagram that cutting is got off by computer Face datection is carried out with Adaboost, if face is detected, identification is carried out using PCA, if being not detected by face Then directly identification is carried out using human body PCA, why directly do not carry out identification using PCA, be because straight using PCA To connect be identified face and will compare whole body and be identified that intrinsic dimensionality is few, amount of calculation is low, and discrimination is higher.

Fig. 3 illustrates branch schematic diagram of the pyroelectric sensor on camera steering engine flat, 0 ° of camera, 90 °, It is 1,2,3,4 that the pyroelectric sensor of 180 °, 270 ° four directions is numbered respectively, if only one of which pyroelectric sensor, Camera steering wheel moves to the direction that the pyroelectric sensor is located.When two pyroelectric sensors capture signal, Camera is moved to the position in the middle of two pyroelectric sensor angles then, for example numbering is that 1,2 sensors capture signal, Camera is then moved to the position of its 45° angle.After camera initial position is fixed to be finished, then close pyroelectric sensor Signal capture, detects human body in the video information that cannot be obtained from camera and then restarts the first of pyroelectric sensor Begin positioning.

HOG+SVM human testing steps：

The HOG features of system extraction human body are put into, and the HOG of characteristics of human body is special Levy extraction process as follows：

Collect the positive example and negative example of human sample, human body of the positive sample from 96*160 sizes in data set as shown in Figure 4 Picture, will all remove 16 pixels up and down during use, intercept the human body of middle 64*128 sizes.Negative sample is from such as Fig. 5 Cut in the shown indoor picture not comprising human body at random, size is equally 64*128, and the quantity of positive negative sample is The more the better, 10000 or so are preferably attained by.Sample label is given to all of positive negative sample, and positive sample is labeled as 1, and negative sample is marked For 0, HOG description of positive and negative sample image are then extracted.

The normalization that color space is carried out to input picture using Gamma correction methods, not only can so suppress noise Interference, can also reduce the impact caused by illumination variation.

The gradient of pixel (x, y) is calculated, its formula is as follows：

G_x(x, y)=H (x+1, y)-H (x-1, y)

G_y(x, y)=H (x, y+1)-H (x, y-1)

G in formula_x(x, y) represents pixel (x, y) horizontal direction gradient, G_y(x, y) represents pixel (x, y) vertical direction On gradient, (x+1 y) represents that ((x-1 y) represents point (x-1, pixel value y), H (x, y+1) to H to point for x+1, pixel value y) to H Represent that the pixel value of point (x, y+1), H (x, y-1) represent the pixel value of point (x, y-1)；The gradient magnitude G at pixel (x, y) place (x, y) and gradient direction α (x, y) are respectively：

Gradient orientation histogram is built, the detection window of a 64*64 pixel is divided into 4*4 cell factory, each Cell factory size is 16*16 pixels, and gradient direction is quantified as 9, and each pixel in cell factory is based on certain The histogram passage in individual direction takes Nearest Neighbor with Weighted Voting, weights to be calculated according to the gradient magnitude of the pixel.Will be thin per adjacent 4 Born of the same parents constitute a block, so a total of 9 blocks, so the dimension of the HOG characteristic vectors for obtaining is 4*9**3*3=324.

As the change of illumination causes the excursion of gradient intensity very big, it is therefore desirable to carry out normalizing to gradient intensity Change.Using the method for normalizing of following formula, formula is as follows：

In formula, ν is represented also without normalized vector, | | ν_k| | represent that the k rank norms of ν, ε represent constant, the value very little of ε. Then normalization after eigenvectors matrix and label put in SVM and be trained.

The Generalized optimal interface solved by SVM changes into the problem of the quadratic function optimizing under an inequality constraints：

Corresponding constraints is y_i(ω^Tx_i-b)≥1-ξ_iAnd ξ_i>=0, i=1,2 ... m, m are positive integer, and in formula, C is Penalty, ξ_iFor the slack variable for introducing, x_iFor input sample, ω is its parameter value, and b is threshold value.Then optimal classification function phase Ground is answered to be changed into：

B in formula^*For classification thresholds,For y_iSlope, φ^T(x_i) φ (x) be kernel function K (x, x_i)=φ^T(x_i)φ X (), kernel function adopt RBF：

In formula, σ is smoothness parameter, then carries out human testing in negative sample artwork using the grader for training, so The rectangle frame of error detection is put into SVM again as difficult example afterwards it is trained and obtains final grader.Finally detect The human body block diagram for coming is as shown in Figure 5.

Camera location human body and the calibration process of camera：

Human testing is carried out to video sequence by the final grader for training, is detected once per 25 frames, when After detecting human body, the center pixel coordinate G of human body rectangle frame is calculated₁(x₁,y₁), then obtain the middle imago of whole two field picture Plain coordinate G₂(x₂,y₂), according to formula：

In formula, X, Y are that camera shoots the maximum transverse and longitudinal pixel value of photo, and θ is camera wide-angle.

According further to national forest park in Xiaokeng：

Focal lengths of the f for camera in formula, Z is distance of the human body to camera, and X is the height of human body, and x is on the plane of delineation The height of human body.The distance of human body and camera can be calculated, the then regulation by system chassis steering wheel makes human body and takes the photograph As the distance of head remains definite value, so can ensure that camera shoots the definition of video.In fact, due to camera chip Center is not generally on optical axis and single pixel point is rectangular rather than foursquare reason on low order imager, it is impossible to make Go to calculate with preferable national forest park in Xiaokeng, be therefore reintroduced back to camera inner parameter C_x、C_y、f_x、f_y.Wherein C_xAnd C_yFor light The possible skew of axle, f_xPhysics focal length F and length for lens and each unit size s of imager_xProduct, f_yFor lens Physics focal length F and length and each unit size s of imager_yProduct because s_x、s_yCan not be in camera calibration process with F In be unable to direct measurement, only combined amount f_x=Fs_xAnd f_y=Fs_yCan directly calculate and go directly without removing camera Connect and measure its part.So can will be represented with following formula on the spot projection in the world to camera：

In formula, M is parameter matrix, and q is the pixel coordinate value of the point on projector space, and which is that three-dimensional vector represents two-dimentional throwing The homogeneous coordinates in shadow space, launch above formula and can be found that w=Z, can just recover former definition divided by w, and Q is sat for actual physics Mark.

Video camera, as a result of lens, is not introduce any distortion without lens, it is therefore desirable to camera Radial distortion and tangential distortion are modeled and try to achieve accurate coordinate value.For radial distortion, the distortion at imager center is 0, with moving to edge, distort also increasingly severe.Taylor series exhibition r=0 position around can be used in a practical situation Being quantitatively described, the radial position of imager point is adjusted the first few items that opens as the following formula.

x_co=x (1+k₁r²+k₂r⁴)

y_co=y (1+k₁r²+k₂r⁴)

In formula, (x, y) is home position of the distortion point on imager, (x_co,y_co) be correction after new position, k₁And k₂ Parameter for the Section 1 and Section 2 of Taylor series expansion.For tangential distortion, with two extra parameters p₁And p₂To retouch State, its formula is as follows：

x_co=x+ [2p₁y+p₂(r²+2x²)]

y_co=y+ [p₁(r²+2y²)+2p₂x]

Therefore a total of four distortion parameter k₁、k₂、p₁、p₂.It is that camera internal reference number and distortion parameter carry out video camera mark Fixed so as to obtaining 4 camera intrinsic parameter (f_x,f_y,c_x,c_y) and 4 distortion parameters, two radial direction (k₁,k₂) and two cut To (p₁,p₂).After all parameters are tried to achieve, just human body actual physics coordinate can be tried to achieve according to human body pixel coordinate in the picture, Human body is made to keep appropriately distance with video camera by regulating system base plate electric machine.

System is using 8*6 black and white interval chessboard as calibrated reference, it is assumed that when calibrating parameters are solved, camera does not have Distortion.For each chessboard visual field, a homography matrix H, the form that H is written as column vector, H=[h is obtained₁h₂h₃], Each h is 3*1 vectors.H is multiplied by the first two spin matrix r equal to camera intrinsic parameter matrix M₁And r₂With combining for translation vector t Matrix, along with zoom factor s, i.e.,：

H=[h₁h₂h₃]=sM [r₁r₂t]

Decompose equation to obtain：

r₁=λ M^-1h₁

r₂=λ M^-1h₂

T=λ M^-1h₃

λ=1/s in formula, rotating vector are mutually orthogonal in construction, so r₁And r₂Mutually orthogonal, then have：

r₁ ^Tr₂=0

For any vectorial a and b, there is (ab)^T=b^Ta^T, therefore obtain first constraint：

h₁ ^T(M^-1)^TM^-1h₂=0

The equal length of known rotating vector, i.e., | | r₁| |=| | r₂| |, obtain second constraint：

h₁ ^T(M^-1)^TM^-1h₁=h₂ ^T(M^-1)^TM^-1h₂

If B=is (M^-1)^TM^-1, launch to have：

The common version of matrix B closes solution：

Because B is symmetrical, it can be write as the dot product of 6 element vectors.Rearrange B element obtain one new to Amount b, then have：

Then two constraints can be written as：

Using different angle K checkerboard images, stacking these equations has：

Vb=0

In formula, V is the matrix of 2K*6, and camera intrinsic parameter can be directly obtained from the closing solution of B matrixes：

c_x=-B₁₃f_x ²/λ

In formula,, outer parameter can be calculated by homography condition：

r₁=λ M^-1h₁

r₂=λ M^-1h₂

r₃=r₁*r₂

T=λ M^-1h₃

λ=1/ is determined by orthogonality condition in formula | | M^-1h₁| |, the position of the perception point obtained on image due to distortion It is false.If pin-hole model is perfect, (x is made_p,y_p) it is the position that puts, make (x_d,y_d) it is the position that distorts, then Have：

By following replacement, the calibration result that can not distorted：

So after inside and outside parameter is reevaluated, these a large amount of equations for obtaining can find distortion parameter, take the photograph when obtaining As after all parameters, it is possible to according to the position that its pixel coordinate tries to achieve human body physical coordinates, so as to pass through to adjust chassis Motor makes video camera and human body be maintained at certain distance.

Face datection is completed using Adaboost algorithm：

The system is by the use of Harr features as shown in Figure 7 as the key feature for judging face, the calculating institute of Harr features The feature templates for using are combined using simple rectangle and are made up of the rectangle of two or more congruences, are wherein had in feature templates black Color and white two kinds of rectangles.Method of the calculating process using integrogram, for image in 1 point of A (x, y), the gray value is I (x, y), (x ', y ') is that the computational methods of its integrogram ii (x, y) are less than the point of (x, y)：

Ii (x, y)=∑_{x′≤x,y′≤y}I(x′,y′)

After calculating the Harr features of sample image, need to screen required feature, each Harr feature corresponding Individual grader, picks out optimum Harr features to construct Face datection grader from substantial amounts of characteristic value, Weak Classifier Construction method is as follows：

Give a series of face sample (x₁,y₁),(x₂,y₂),...,(x_i,y_i),...,(x_n,y_n) wherein x_iIt is i-th sample This, y_iWork as y as sample labeling_iFace sample, y is represented when=1_iNon-face sample is represented when=0.Then with equation below at the beginning of Beginningization weights：

D in formula_tI () represents the weights of i-th sample in the t time circulation, m, n represent positive and negative number of samples, the system Using ORL face databases as sample.Weights q after after trying to achieve weights normalized weights_t(i)：

The Harr characteristic values of all samples are calculated after weights normalization, the order by ascending order arrayed feature value and before recording I, the weights and S of portion's face sample of demanding perfection⁺, whole non-face sample weights S^-And face sample weights T before currentElement⁺, non-face sample weights T before currentElement^-.And calculate two parameter values J, K, J=T⁺+(S^--T^-), K=T^-+(S⁺-T⁺), E=min (J, K) is then made, the element corresponding to min (E) characteristic value f according to corresponding to I finds the element is found X () is required preliminary examination threshold value f '_θ, and direction vector p_jWorthwhile J=K when be 1, otherwise be -1.For each Harr spy Levying f can all have a grader h (x, f, p, θ) corresponding therewith, and angles of the θ for direction vector in formula, with preliminary examination threshold value f '_θAccording to Weak Classifier formula：

The weighting fault rate of each grader is trained and is calculated to sample：

ξ_i=∑_iq_i|h(x_i,f,p,θ)-y_i|

It is the optimum classifier h obtained by current signature to find the minimum grader of weighting fault rate_tAnd grader is most Excellent threshold value f_θ, q_iWeights after for normalization, all parametric solutions of such Weak Classifier are finished.

Corresponding Harr features and threshold value f are extracted with the Weak Classifier for training_θPositive sample, classification is traveled through to sample space The sample weights of mistake are constant, and correct weights of classifying diminish, and the weights of classification error is increased with respect to proportion, and weights are more New Policy is：

ξ in formula_iFor weighting fault rate, D_t+1I () is the weights after updating.

The sample after weights will be updated and build new grader, the weak of T weighting rate minimum by mistake will be had after T this circulation Grader is generated, and these graders is carried out cascade and obtains strong classifier：

A in formula_t=log [(1- ξ_i)/ξ_i] for h (t) coefficient, ξ_iFor weighting fault rate.

System is 20*20 using detection window size, and amplification coefficient is 1.2 to be detected using multiple dimensioned method, and With the number N ＞ 4 of detection window, center of circle radius R ＜ 5 carries out multiple dimensioned merging for merging the condition of window.

Identification is completed using principal component analysis PCA：

PCA PCA is for random character χ={ χ₁,χ₂,...χ_n, wherein χ_i∈R^d, seek its mean vector first μ, formula are as follows：

In formula, n is characterized quantity, then calculates covariance matrix S, and formula is as follows：

Calculate eigenvalue λ_iWith corresponding characteristic vector ν_i, formula is as follows：

Sν_i=λ_iν_i, i=1,2 ..., n

Arrangement of successively decreasing, characteristic vector and its sequence consensus are carried out to characteristic value, and K main component is also K maximum spy The corresponding characteristic vector of value indicative, the K principal component of χ：

Y=ω^T(χ-μ)

ω=(ν in formula₁,ν₂,...,ν_k)

All of sample is added label, the sample of member A is collectively labeled as A, and the sample of member B is collectively labeled as B, with such Push away.Retain 15 main components, then initialized with confidence threshold value, data set as shown in Figure 9 is put into training The characteristic vector of each member's face being obtained in device and mean vector being preserved, data set as shown in Figure 10 is put into training Interim obtain the non-face characteristic vector of each member and mean vector and preserve.When detecting face using Adaboost algorithm When, the characteristic vector that is deposited with face code insurance that human face photo cutting is got off is compared so as to verify identity, without inspection When measuring face, the characteristic vector that the picture of whole human body parts is preserved with non-face part is compared so as to verifying Identity.

Claims (6)

1. the indoor human body detecting and tracking and identification system based on multisensor, it is characterised in that：The system is released by heat Electric infrared sensor completes the Primary Location of human body, the scope that camera is moved to human body appearance by steering wheel, by shooting Head gathers the image information in the range of this and image information is transferred in computer, and computer completes the correlometer of human testing Calculate and control steering wheel and become to making camera and mobile platform tracking human body；Computer is entered by collecting image information and background information Row coupling is so that it is determined that the identity of detection people；

Specific workflow is as follows：

The Preliminary detection of S1 human bodies and the Primary Location of camera

The system catches human infrared radiation using pyroelectric infrared sensor and is converted into the original of faint voltage signal Manage to detect whether interior has human body, for 6 meters, sensing angle is 100 to the pyroelectric infrared sensor induction range that system is adopted Degree；Four pyroelectric sensors are distributed in the due west of camera, due south, due east, positive north four direction, detection angles will be covered Indoor 360 degree scope, when pyroelectric sensor detects human body signal, camera steering wheel head just turns to the model In enclosing；

S2 completes human testing using image

The system using HOG+SVM method to video sequence in human body detect that step is as follows：

The making of S2.1 Sample Storehouses

, used as positive sample, the system is using the picture in INRIA static state pedestrian's Test databases as training for collection human sample Positive sample, using the picture under the negative sample in INRIA databases and treated indoor unmanned environment as the negative sample of training This；

The selection of the parameter of S2.2HOG feature extractions

Using the HOG that OpenCv is carried, this method detects that built-in function, corresponding parameter are set to：Detection window is 64*64, cell The size of unit is 16*16 pixels, and block sliding step is 16, and gradient direction is quantified as 9, therefore the HOG feature dimensions of piece image Number is 4*9*3*3=324；The standardized method of characteristic block selects L2-Hys, threshold value to be 0.2 and carry out Gamma corrections；

S2.3 carries out SVM training

The HOG features of all positive negative samples are extracted, and aligns negative sample and give label, positive sample is labeled as 1, by negative sample mark 0 is designated as, then the HOG features of positive negative sample and label is all input in SVM training aids and is trained, just obtain a human body Grader；

S3 control cameras rotate tracking human body

Human body is marked its in-scope with square frame in image by system, and every 25 frame is detected once, then by calculation block The difference of center pixel coordinate and whole sub-picture center pixel coordinate is calculating the angle of steering wheel left-right rotation and rotate upwardly and downwardly Angle, so that ensure that human body is occurred in camera angular field of view all the time；On the other hand, camera is obtained by camera calibration Intrinsic parameter and distortion parameter, so as to extrapolate human body actual physics coordinate according to the coordinate of human body block diagram center pixel, so as to Adjusting base plate electric machine makes camera be maintained in given area with human body；

S4 identifications

The identification procedure of the system out becomes independent for the human body block diagram that detects cutting into a secondary independent figure first Picture, carries out Face datection using VIola-Jones graders on the sub-picture, its process approximately as：

S4.1 accelerates the calculating of the value of 45 degree of rotations of rectangular image area or rectangular area, the picture structure using integral image The calculating for accelerating class Haar input feature vector is used to, which fully enters feature as Adaboost graders；

S4.2 creates face with non-face grader node using Adaboost algorithm；

S4.3 constitutes Weak Classifier node one node of screening type cascade；Each node D_jSpecial using class Haar comprising one group Levy the decision tree that trains either with or without face；Node is arranged from simple to complex, can so minimize the simple zones of refusal image Amount of calculation during domain；First classifiers are optimums, can be by the image-region comprising face, while allowing some not include people The image of face passes through；Second classifiers suboptimum grader, and have relatively low reject rate；Then by that analogy, as long as image Region has passed through whole cascade, then it is assumed that there is face the inside, and face block diagram is rectified out.

2. the indoor human body detecting and tracking and identification system based on multisensor according to claim 1, its feature It is：Due to the limited amount system of house person, the quantity of the Sample Storehouse of old friend's face is also limited, using PCA PCA carries out identification, and all of training data is projected to PCA subspaces, and image projection to be identified is empty to PCA Between, find training data projection after vector sum images to be recognized projection after vectorial immediate that.

3. the indoor human body detecting and tracking and identification system based on multisensor according to claim 1, its feature It is：The system using the method for human testing be not the information that can capture face front every time, if cut out The human body block diagram for coming only has the side-information of head, back side information or the information only below head, then skip Adaboost The process of Face datection, directly carries out identity differentiation using the method for PCA, only need to change data set.

4. the indoor human body detecting and tracking and identification system based on multisensor according to claim 1, its feature It is：Arduino reads pyroelectric sensor signal to be responsible for completing initial alignment, the model that camera is moved to physical activity Enclose, then all visual informations in camera collection current visual angle, video sequence are passed in the computer of system, computer Human body in video is detected by HOG+SVM algorithms, then adjusting camera steering wheel and base plate electric machine makes human body occur in all the time In the angular field of view of camera, the picture of the human body block diagram that cutting is got off by computer carries out Face datection using Adaboost, Identification is carried out using PCA if face is detected, if face is not detected by, directly body is carried out using human body PCA Why part identification, directly do not carry out identification using PCA, be because directly face being identified to compare using PCA Whole body is identified that intrinsic dimensionality is few, and amount of calculation is low, and discrimination is higher.

5. the indoor human body detecting and tracking and identification system based on multisensor according to claim 1, its feature It is：It is 1,2,3,4 that the pyroelectric sensor of 0 °, 90 °, 180 °, 270 ° four direction of camera is numbered respectively, if only one Individual pyroelectric sensor, then camera steering wheel move to the direction that the pyroelectric sensor is located；When two pyroelectric sensors When capturing signal, then camera is moved to the position in the middle of two pyroelectric sensor angles, for example numbering is 1,2 Sensor captures signal, then camera is moved to the position of its 45° angle；After camera initial position is fixed to be finished, then The signal capture of pyroelectric sensor is closed, and human body is detected in the video information that cannot be obtained from camera and is then opened again The initial alignment of dynamic pyroelectric sensor.

6. the indoor human body detecting and tracking and identification system based on multisensor according to claim 1, its feature It is：HOG+SVM human testing steps：

The HOG features of system extraction human body are put into, and the HOG features of characteristics of human body are carried Take process as follows：

The positive example and negative example of human sample is collected, human body picture of the positive sample from 96*160 sizes in data set will during use All remove 16 pixels up and down, intercept the human body of middle 64*128 sizes；Negative sample is from the indoor figure not comprising human body Cut in piece at random, size is equally 64*128, and the quantity of positive negative sample is The more the better, be preferably attained by 10000 left sides Right；Sample label is given to all of positive negative sample, positive sample is labeled as 1, and negative sample is labeled as 0, then extracts positive negative sample HOG description of image；

The normalization that color space is carried out to input picture using Gamma correction methods, not only can so suppress the interference of noise, The impact caused by illumination variation can also be reduced；

The gradient of pixel (x, y) is calculated, its formula is as follows：

G_x(x, y)=H (x+1, y)-H (x-1, y)

G_y(x, y)=H (x, y+1)-H (x, y-1)

G in formula_x(x, y) represents pixel (x, y) horizontal direction gradient, G_y(x, y) is represented in pixel (x, y) vertical direction Gradient, (x+1 y) represents that ((x-1 y) represents that (x-1, pixel value y), H (x, y+1) are represented point to H to point for x+1, pixel value y) to H The pixel value of point (x, y+1), H (x, y-1) represent the pixel value of point (x, y-1)；Gradient magnitude G (x, y) at pixel (x, y) place It is respectively with gradient direction α (x, y)：

$G(x,y)=\sqrt{G_x{(x,y)}^2+G_y{(x,y)}^2}$

$\mathrm{\α}(x,y)={\arctan }^{-1}\left(\frac{G_x(x,y)}{G_y(x,y)}\right)$

Gradient orientation histogram is built, the detection window of a 64*64 pixel 4*4 cell factory is divided into, each cell Cell size is 16*16 pixels, and gradient direction is quantified as 9, and each pixel in cell factory is based on certain side To histogram passage take Nearest Neighbor with Weighted Voting, weights are calculated according to the gradient magnitude of the pixel；By per 4 adjacent groups of cells Into a block, so a total of 9 blocks, so the dimension of the HOG characteristic vectors for obtaining is 4*9**3*3=324；

As the change of illumination causes the excursion of gradient intensity very big, it is therefore desirable to which gradient intensity is normalized； Using the method for normalizing of following formula, formula is as follows：

$L_2-Hys:\stackrel{\→}{\mathrm{\ν}}\←\stackrel{\→}{\mathrm{\ν}}/\sqrt{\left|\right|\stackrel{\→}{\mathrm{\ν}}|{|}_2^2+{\mathrm{\ϵ}}^2}$

In formula, ν is represented also without normalized vector, | | ν_k| | represent that the k rank norms of ν, ε represent constant, the value very little of ε；Then Eigenvectors matrix and label after normalization is put in SVM and is trained；

The Generalized optimal interface solved by SVM changes into the problem of the quadratic function optimizing under an inequality constraints：

$min\frac{1}{2}\left|\right|\mathrm{\ω}|{|}^2+C\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\ξ}}_i$

Corresponding constraints is y_i(ω^Tx_i-b)≥1-ξ_iAnd ξ_i>=0, i=1,2 ... m, m are positive integer, and in formula, C is punishment Function, ξ_iFor the slack variable for introducing, x_iFor input sample, ω is its parameter value, and b is threshold value；Then optimal classification function is correspondingly It is changed into：

$g\left(x\right)=sign\{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{{\mathrm{\α}}_i}^{*}{y}_i\[{\mathrm{\φ}}^T\left(x_i\right)*\mathrm{\φ}\left(x\right)\]+b^{*}\}$

B in formula^*For classification thresholds,For y_iSlope, φ^T(x_i) φ (x) be kernel function K (x, x_i)=φ^T(x_i) φ (x), core letter Number adopts RBF：

In formula, σ is smoothness parameter, then carries out human testing in negative sample artwork using the grader for training, and then will The rectangle frame of error detection is put into again SVM as difficult example and is trained and obtains final grader；

Camera location human body and the calibration process of camera：

Human testing is carried out to video sequence by the final grader for training, detection once, works as detection per 25 frames To after human body, the center pixel coordinate G of human body rectangle frame is calculated₁(x₁,y₁), the center pixel for then obtaining whole two field picture is sat Mark G₂(x₂,y₂), according to formula：

${\mathrm{\θ}}_1=\frac{x_1-x_2}{X}\mathrm{\θ}$

${\mathrm{\θ}}_1=\frac{x_1-x_2}{X}\mathrm{\θ}$

In formula, X, Y are that camera shoots the maximum transverse and longitudinal pixel value of photo, and θ is camera wide-angle；

According further to national forest park in Xiaokeng：

${\mathrm{\θ}}_1=\frac{y_1-y_2}{Y}\mathrm{\θ}$

Focal lengths of the f for camera in formula, Z is distance of the human body to camera, and X is the height of human body, and x is human body on the plane of delineation Height；The distance of human body and camera can be calculated, the then regulation by system chassis steering wheel makes human body and camera Distance remain definite value, so can ensure that camera shoot video definition；In fact, in due to camera chip The heart is not generally on optical axis and single pixel point is rectangular rather than foursquare reason on low order imager, it is impossible to using reason The national forest park in Xiaokeng that thinks goes to calculate, and is therefore reintroduced back to camera inner parameter C_x、C_y、f_x、f_y；Wherein C_xAnd C_yCan for optical axis The skew of energy, f_xPhysics focal length F and length for lens and each unit size s of imager_xProduct, f_yPhysics for lens Focal length F and length and each unit size s of imager_yProduct because s_x、s_yCan not be during camera calibration not with F Can direct measurement, only combined amount f_x=Fs_xAnd f_y=Fs_yCan directly calculate directly survey is gone without removing camera Measure its part；So can will be represented with following formula on the spot projection in the world to camera：

$q=MQ,q=\left[\begin{array}{c}x\\ y\\ w\end{array}\right],M=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right],Q=\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]$

In formula, M is parameter matrix, and q is the pixel coordinate value of the point on projector space, and which is that three-dimensional vector represents that two-dimensional projection is empty Between homogeneous coordinates, launch above formula and can be found that w=Z, can just recover former definition divided by w, Q is actual physics coordinate；

Video camera, as a result of lens, is not introduce any distortion without lens, it is therefore desirable to the radial direction to camera Distortion and tangential distortion are modeled and try to achieve accurate coordinate value；For radial distortion, the distortion at imager center is 0, with To edge and move, distort also increasingly severe；In a practical situation can be with the Taylor series expansion around r=0 positions Being quantitatively described, the radial position of imager point is adjusted first few items as the following formula；

x_co=x (1+k₁r²+k₂r⁴)

y_co=y (1+k₁r²+k₂r⁴)

In formula, (x, y) is home position of the distortion point on imager, (x_co,y_co) be correction after new position, k₁And k₂For Thailand Strangle the parameter of the Section 1 and Section 2 of series expansion；For tangential distortion, with two extra parameters p₁And p₂To describe, its Formula is as follows：

Therefore a total of four distortion parameter k₁、k₂、p₁、p₂；For camera internal reference number and distortion parameter carry out camera calibration from And obtain 4 camera intrinsic parameter (f_x,f_y,c_x,c_y) and 4 distortion parameters, two radial direction (k₁,k₂) and two tangential (p₁,p₂)；After all parameters are tried to achieve, just human body actual physics coordinate can be tried to achieve according to human body pixel coordinate in the picture, be led to Overregulating system chassis motor makes human body keep appropriately distance with video camera；

System is using 8*6 black and white interval chessboard as calibrated reference, it is assumed that when calibrating parameters are solved, camera does not distort； For each chessboard visual field, a homography matrix H, the form that H is written as column vector, H=[h is obtained₁h₂h₃], each h It is 3*1 vectors；H is multiplied by the first two spin matrix r equal to camera intrinsic parameter matrix M₁And r₂With the combinatorial matrix of translation vector t, Zoom factor s is added, i.e.,：

H=[h₁h₂h₃]=sM [r₁r₂t]

Decompose equation to obtain：

r₁=λ M^-1h₁

r₂=λ M^-1h₂

T=λ M^-1h₃

λ=1/s in formula, rotating vector are mutually orthogonal in construction, so r₁And r₂Mutually orthogonal, then have：

r₁ ^Tr₂=0

For any vectorial a and b, there is (ab)^T=b^Ta^T, therefore obtain first constraint：

h₁ ^T(M^-1)^TM^-1h₂=0

The equal length of known rotating vector, i.e., | | r₁| |=| | r₂| |, obtain second constraint：

h₁ ^T(M^-1)^TM^-1h₁=h₂ ^T(M^-1)^TM^-1h₂

If B=is (M^-1)^TM^-1, launch to have：

$B\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{12}& B_{22}& B_{23}\\ B_{13}& B_{23}& B_{33}\end{array}\right]$

The common version of matrix B closes solution：

$B=\left[\begin{array}{ccc}\frac{1}{{f_x}^2}& 0& \frac{-c_x}{{f_x}^2}\\ 0& \frac{1}{{f_y}^2}& \frac{-c_y}{{f_y}^2}\\ \frac{-c_x}{{f_x}^2}& \frac{-c_y}{{f_y}^2}& \frac{c_x}{{f_x}^2}+\frac{c_y}{{f_y}^2}+1\end{array}\right]$

Because B is symmetrical, it can be write as the dot product of 6 element vectors；The element for rearranging B obtains a new vector b, Then have：

${h_i}^T{\mathrm{Bh}}_j={{\mathrm{\ν}}_{ij}}^{T}b={\left[\begin{array}{c}{h}_{i1}{h}_{j1}\\ h_{i1}{h}_{j2}+h_{i2}{h}_{j1}\\ h_{i2}{h}_{j2}\\ h_{i3}{h}_{j1}+h_{i1}{h}_{j3}\\ h_{i3}{h}_{j2}+h_{i2}{h}_{j3}\\ h_{i3}{h}_{j3}\end{array}\right]}^T{\left[\begin{array}{c}{B}_{11}\\ B_{12}\\ B_{22}\\ B_{13}\\ B_{23}\\ B_{33}\end{array}\right]}^T$

Then two constraints can be written as：

$\left[\begin{array}{c}{{\mathrm{\ν}}_{12}}^T\\ {({\mathrm{\ν}}_{11}-{\mathrm{\ν}}_{22})}^T\end{array}\right]b=0$

Using different angle K checkerboard images, stacking these equations has：

Vb=0

In formula, V is the matrix of 2K*6, and camera intrinsic parameter can be directly obtained from the closing solution of B matrixes：

$f_x=\sqrt{\mathrm{\λ}/B_{11}}$

$f_y=\sqrt{{\mathrm{\λB}}_{11}/(B_{11}{B}_{22}-B_{12}^2)}$

c_x=-B₁₃f_x ²/λ

$c_y=(B_{12}{B}_{13}-B_{11}{B}_{23})/(B_{11}{B}_{22}-B_{12}^2)$

In formula,Outer parameter can be calculated by homography condition：

r₁=λ M^-1h₁

r₂=λ M^-1h₂

r₃=r₁*r₂

T=λ M^-1h₃

λ=1/ is determined by orthogonality condition in formula | | M^-1h₁| |, the position of the perception point obtained on image due to distortion is not Really；If pin-hole model is perfect, (x is made_p,y_p) it is the position that puts, make (x_d,y_d) it is the position that distorts, then have：

$\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right]=\left[\begin{array}{c}{f}_x{X}^W/Z^W+c_x\\ f_y{X}^W/Z^W+c_y\end{array}\right]$

By following replacement, the calibration result that can not distorted：

$\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right]=(1+k_1{r}^2+k_2{r}^4)\left[\begin{array}{c}{x}_d\\ y_d\end{array}\right]+\left[\begin{array}{c}2p_1{x}_d{y}_d+p_2(r^2+2x_d^2)\\ p_1(r^2+2y_d^2)+2p_2{x}_d{y}_d\end{array}\right]$

So after inside and outside parameter is reevaluated, these a large amount of equations for obtaining can find distortion parameter, when obtaining camera After all parameters, it is possible to according to the position that its pixel coordinate tries to achieve human body physical coordinates, so as to pass through to adjust base plate electric machine Video camera and human body is made to be maintained at certain distance；

Face datection is completed using Adaboost algorithm：

The system by the use of Harr features as the key feature for judging face, adopt by the feature templates used by the calculating of Harr features It is made up of the rectangle of two or more congruences with the combination of simple rectangle, wherein in feature templates, there are two kinds of squares of black and white Shape；Calculating process using integrogram method, for image in 1 point of A (x, y), the gray value be I (x, y), (x ', Y ') it is that the computational methods of its integrogram ii (x, y) are less than the point of (x, y)：

Ii (x, y)=∑_{x′≤x,y′≤y}I(x′,y′)

After calculating the Harr features of sample image, need to screen required feature, each Harr feature is corresponding one point Class device, picks out optimum Harr features to construct Face datection grader, the structure of Weak Classifier from substantial amounts of characteristic value Method is as follows：

Give a series of face sample (x₁,y₁),(x₂,y₂),...,(x_i,y_i),...,(x_n,y_n) wherein x_iIt is i-th sample, y_i Work as y as sample labeling_iFace sample, y is represented when=1_iNon-face sample is represented when=0；Then with equation below initialization power Value：

$D_t\left(i\right)=\left\{\begin{array}{c}1/\left(2m\right),y_i=1\\ 1/\left(2n\right),y_i=0\end{array}\right.$

D in formula_tI () represents that the weights of i-th sample in the t time circulation, m, n represent that positive and negative number of samples, the system are adopted ORL face databases are used as sample；Weights q after after trying to achieve weights normalized weights_t(i)：

$q_t\left(i\right)=D_t\left(i\right)/\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{D}_t\left(i\right)$

The Harr characteristic values of all samples are calculated after weights normalization, order I by ascending order arrayed feature value and before recording is asked The weights and S of whole face samples⁺, whole non-face sample weights S^-And face sample weights T before currentElement⁺, when Non-face sample weights T before front element^-；And calculate two parameter values J, K, J=T⁺+(S^--T^-), K=T^-+(S₊-T⁺), so After make E=min (J, K), find the element corresponding to min (E) characteristic value f (x) according to corresponding to I finds the element and be Required preliminary examination threshold value f_θ', and direction vector p_jWorthwhile J=K when be 1, otherwise be -1；For each Harr feature f can There is a grader h (x, f, p, θ) corresponding therewith, angles of the θ for direction vector in formula, with preliminary examination threshold value f_θ' according to weak typing Device formula：

$h_j\left(x\right)=\left\{\begin{array}{cc}1,& p_j{f}_j\left(x\right)\≤p_j{f}_{\mathrm{\θ}}\\ 0,& others\end{array}\right.$

The weighting fault rate of each grader is trained and is calculated to sample：

ξ_i=∑_iq_i|h(x_i,f,p,θ)-y_i|

It is the optimum classifier h obtained by current signature to find the minimum grader of weighting fault rate_tAnd the optimal threshold of grader f_θ, q_iWeights after for normalization, all parametric solutions of such Weak Classifier are finished；

Corresponding Harr features and threshold value f are extracted with the Weak Classifier for training_θPositive sample, classification error are traveled through to sample space Sample weights constant, the correct weights of classifying diminish, and so make the relative proportion increasing of weights of classification error, right value update plan Slightly：

$D_{t+1}\left(i\right)=\left\{\begin{array}{cc}{D}_t\left(i\right),& right\\ D_t\left(i\right)\frac{{\mathrm{\ξ}}_i}{1-{\mathrm{\ξ}}_i},& wrong\end{array}\right.$

ξ in formula_iFor weighting fault rate, D_t+1I () is the weights after updating；

The sample after weights will be updated and build new grader, this will have the T weighting minimum weak typing of rate by mistake after circulating through T Device is generated, and these graders is carried out cascade and obtains strong classifier：

$H\left(x\right)=\left\{\begin{array}{cc}1,& {\mathrm{\Σ}}_{t=1}^T{a}_{t}h\left(t\right)\≥\frac{1}{2}{\mathrm{\Σ}}_{t=1}^T{a}_t\\ 0,& others\end{array}\right.$

A in formula_t=log [(1- ξ_i)/ξ_i] for h (t) coefficient, ξ_iFor weighting fault rate；

System is 20*20 using detection window size, and amplification coefficient is 1.2 to be detected using multiple dimensioned method, and examining The number N ＞ 4 of window is surveyed, center of circle radius R ＜ 5 carries out multiple dimensioned merging for the condition for merging window；

Identification is completed using principal component analysis PCA：

PCA PCA is for random character χ={ χ₁,χ₂,...χ_n, wherein χ_i∈R^d, seek its mean vector μ first, public Formula is as follows：

$\mathrm{\μ}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\χ}}_i$

In formula, n is characterized quantity, then calculates covariance matrix S, and formula is as follows：

$S=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}({\mathrm{\χ}}_i-\mathrm{\μ}){({\mathrm{\χ}}_i-\mathrm{\μ})}^{T_i}$

Calculate eigenvalue λ_iWith corresponding characteristic vector ν_i, formula is as follows：

Sν_i=λ_iν_i, i=1,2 ..., n

Arrangement of successively decreasing, characteristic vector and its sequence consensus are carried out to characteristic value, and K main component is also K maximum characteristic value Corresponding characteristic vector, the K principal component of χ：

Y=ω^T(χ-μ)

ω=(ν in formula₁,ν₂,...,ν_k)

All of sample is added label, the sample of member A is collectively labeled as A, and the sample of member B is collectively labeled as B, by that analogy.