CN101320477B - Human body tracing method and equipment thereof - Google Patents

Abstract

The present invention relates to a human body tracking method under a complex background and a device thereof. The method comprises the steps as follows: a pixel point set is obtained in terms of a human body initial position (y0); the space characteristic and the color characteristic of pixel points in the set are calculated; the human body position (y1) at a current frame in terms of the space characteristic and the color characteristic is calculated. The present invention integrates space position information into color information and combines a human body detection technology to correct a human body tracking result, thereby improving the comprehensive tracking ability further.

Description

A kind of human body tracing method and equipment thereof

Technical field

The present invention relates to a kind of human body tracing method and equipment, relate in particular to a kind of method of under complex background, human body being followed the tracks of and equipment thereof.

Background technology

Moving object detection and tracking is an important topic of computer vision field, is widely used in all many-sides such as video monitoring, robot navigation, man-machine interactions.Present popular connection cradle head control video camera, the cloud platform control method of its core just are based on the human body tracking technical design.

In the actual monitored scene, because the influence of the factors such as variation of the motion of video camera, weather, background image has the variation of a lot of complexity.And existing method bad for the complicated big multiprocessing of situation that changes of this background, as mean shift algorithm commonly used.Because this method has only been considered the colouring information of candidate's human body target, has but ignored its spatial positional information,, cause following the tracks of failure easily so treatment effect is bad under near the background color the human body target of motion distributes the situation similar to color of object.And another kind of commonly used tracking--particle filter method needs to be provided with a large amount of particle points, and calculated amount is too big, is unfavorable for real-time processing.

Summary of the invention

The invention provides human body tracing method and equipment under a kind of complex background that can overcome the above problems.

In first aspect, the invention provides a kind of human body tracing method, comprising: according to the initial position (y of human body ₀) obtain the set of pixel; Calculate the space characteristics and the color characteristic of pixel in the described set; And calculate position of human body (y on the present frame according to described space characteristics and described color characteristic ₁).

In second aspect, the invention provides a kind of human body tracking equipment, comprising: video acquisition module is used to gather video streaming image, the output map picture frame; Initial position (y according to human body ₀) obtain the module of pixel set; Calculate the module of the space characteristics and the color characteristic of pixel in the described set; And calculate position of human body (y on the present frame according to described space characteristics and described color characteristic ₁) module.

In one embodiment of the invention, preferably, calculate position of human body (y ₁) step comprise: the similarity measurement that defines candidate region and human region according to described characteristics of human body; And calculate position of human body (y on the present frame according to described similarity measurement ₁).

In another embodiment of the present invention, preferably, described method also comprises: to the position of human body (y on the present frame ₁) detect correction, if testing result is a human region, then with described position of human body (y ₁) as need not the modifier body position.

In another embodiment of the present invention, preferably, described method also comprises: if testing result is non-human region, then with described position of human body (y ₁) carry out window scanning in the relevant zone, search out human region, according to the human region that searches out to position of human body (y ₁) revise, obtain revised position of human body (y ₂).

In yet another embodiment of the present invention, preferably, described detection comprises: the direction histogram that calculates the zone relevant with described position of human body; And; Judge according to an average with described direction histogram whether this zone is human region, wherein, described average is by collecting some human body images and drawing according to its direction histogram.

In another embodiment of the present invention, preferably, described method also comprises: export described need not and revise or revised position of human body; Revise or revised position of human body according to described need not, calculate human space feature and color characteristic and output in the present frame; With position of human body, space characteristics and the color characteristic of output as human body initial position, space characteristics and the color characteristic of next frame.

In another embodiment of the present invention, preferably, described color characteristic is the color histogram of the human region relevant with described initial position.

The present invention goes by spatial positional information is fused in the colouring information, and combines Human Detection the human body tracking result is revised, and has further improved the tracking power of entire system.

Description of drawings

Below with reference to accompanying drawings specific embodiments of the present invention is described in detail, in the accompanying drawings:

Fig. 1 is the block diagram according to human body tracking of the present invention system; And

Fig. 2 is the process flow diagram according to human body tracing method of the present invention.

Embodiment

Fig. 1 is the block diagram according to human body tracking of the present invention system.

As shown in Figure 1, this system comprises video acquisition module, human body tracking module and human detection correcting module.

Video acquisition module is used to obtain video streaming image, the output map picture frame.

The human body tracking module is used for finding the position with body templates similarity maximum on present image.

The human detection correcting module is used for the picture position that the human body tracking module finds is detected and revised.

The course of work according to human body tracking of the present invention system is divided into two stages, initial phase and tracking phases.Below this is elaborated respectively.

Initial phase:

At first need to determine the initial position y of human body ₀, can use various human body detecting methods to come human body, even can the directly manual position of demarcating human body.Then, the initial position of human body is sent into the human body tracking module.

Should be pointed out that described initialization process can be undertaken by an independent initialization module, also can directly finish by the human detection correcting module.Further, when target body disappears in current image frame, need carry out initialization again, promptly demarcate y again ₀Less if desired artificial participation can be used the method for human detection, otherwise, can get final product with artificial the demarcation simply.The method of existing human detection is a lot, and is not the emphasis that the present invention will set forth, and therefore repeats no more.

Tracking phase;

The video acquisition module collection comprises the video streaming image of human body, and wherein continuous multiple frames image respectively as input picture, is repeated following step 1,2 then:

Step 1: utilize the human body tracking module to calculate position of human body y in the current image frame ₁

The human body tracking module functions is the position of finding on current image frame with body templates similarity maximum.There is several different methods to realize human body tracking, and the present invention proposes a kind of improved mean shift method.Below this improved mean shift method is described in detail:

At first, calculate the color histogram in body templates zone:

If { x _i} _{I=1,2 ... N}Be that central point is y ₀, long is h, wide is the pixel set in the body templates zone of w, definition mapping b:R ²→ 1,2 ..., B} is for each x _i, b (x _i) the quantification sequence number of feature in the feature space that quantizes of representing this pixel.

For any quantization value u ∈ 1,2 ..., B} calculates its position average v _u:

$v_u=\frac{1}{{\mathrm{\Σ}}_{i=1}^N{\mathrm{\δ}}_{\mathrm{iu}}}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\frac{2(x_i-y)}{\sqrt{w^2+h^2}}{\mathrm{\δ}}_{\mathrm{iu}}---\left(1\right)$

Wherein, ${\mathrm{\δ}}_{\mathrm{iu}}=\left\{\begin{array}{cc}1& \mathrm{ifb}\left(x_i\right)=u\\ 0& \mathrm{else}\end{array}\right.$

v _uThe expression be the locus mean value of the pixel that all pixel values equal u in the target body region, reflection be the spatial positional information of target body, i.e. space characteristics.

The probability that u occurs in human region

Can be calculated as follows:

${\hat{q}}_u\left(y_0\right)=C_u\underset{i=1}{\overset{N}{\mathrm{\Σ}}}k\left(\right|\left|\frac{2(x_i-y_0)}{\sqrt{w^2+h^2}}\right|{|}^2)\mathrm{\δ}[b\left(x_i\right)-u]---\left(2\right)$

Wherein k () is the Epanechnikov kernel function, and δ () is the Kronecker function, and C is a normaliztion constant, makes $\underset{i=0}{\overset{H-1}{\mathrm{\Σ}}}{\hat{q}}_u\left(y_0\right)=1.$

What represent is the colouring information that all its pixel values equal the pixel of u in the target body region, i.e. color characteristic.

After finishing the calculating of above-mentioned space characteristics and color characteristic respectively, can obtain the feature of human body $\hat{q}={\{{\hat{q}}_u\left(y_0\right),v_u\left(y_0\right)\}}_{u=\mathrm{0,1},...,B-1}$

Next, definition similarity measurement.If with y ₀Candidate region { x for the center _i} _{I=1,2 ... N}Be characterized as

The similarity measurement of this candidate region and human region adopts following formula:

$\mathrm{\ρ}\left(y\right)=\underset{u=0}{\overset{B-1}{\mathrm{\Σ}}}{e}^{-{\left|\right|{\mathrm{\μ}}_u\left(y\right)-v_u\left(y_0\right)\left|\right|}^2}\sqrt{{\hat{p}}_u\left(y\right){\hat{q}}_u\left(y_0\right)}---\left(3\right)$

What formula (3) reacted is the similarity of target area and candidate region, wherein

First

Expression be similarity on the locus, more near first, its value is just big more on space distribution for target area and candidate region, thus similarity is big more.

Second

Expression be similarity on the colouring information, the color of target area and candidate region is more near second, its value is just big more, thereby similarity is big more.

Original mean shift algorithm has only been considered the similarity on color of target area and candidate region, has ignored the similarity on the locus fully.Therefore, under the background color of target distributes very complicated situation, failure appears possibly following the tracks of.And the present invention uses as formula 3 described similarity measurements, and colouring information and spatial information are combined consideration, and candidate region and target area must all very approachingly on color and space distribution could obtain maximum similarity.

Formula 3 also has other make, such as $\mathrm{\ρ}\left(y\right)=\underset{u=0}{\overset{B-1}{\mathrm{\Σ}}}(\mathrm{\α}{e}^{-{\left|\right|{\mathrm{\μ}}_u\left(y\right)-v_u\left(y_0\right)\left|\right|}^2}+(1-\mathrm{\α})\sqrt{{\hat{p}}_u\left(y\right){\hat{q}}_u\left(y_0\right)}),$ Wherein α is the constant between 0 to 1, expression be weighting coefficient, can regulate colouring information and spatial information shared proportion in similarity measurement by regulating this coefficient, the influence of the big more then spatial information of α is bigger, otherwise the influence of colouring information is bigger.

At last, use the gradient optimization method to calculate the position y of human body on present image ₁, y ₁Should satisfy $\frac{\∂\mathrm{\ρ}\left(y\right)}{\∂y}=0.$

Should be pointed out that and also can adopt other mean shift algorithm to finish human body tracking.

Step 2: the position of human body y that the human body tracking module is drawn with the human detection correcting module ₁Detect and revise.

At list of references: in " Qiang Zhu; Shai Avidan; Mei-chen Yeh; Kwang-TingCheng; Fast human detection using a cascade of Histograms of OrientedGradients.IEEE Computer Vision and Pattern Recognition 2006 ", proposed the most effective at present a kind of Human Detection.This technology is a kind of human body detecting method based on direction histogram (HoG) and Adaboost.But the operand of this method is bigger, obviously is unfavorable for real-time processing if each frame all carries out the human detection of such macrooperation amount.

The present invention improves the method in the above-mentioned list of references, can fast and effeciently detect correction to tracking results.

At first, be y according to the method that above-mentioned list of references proposed to central point ₁, long for h, wide for carrying out human detection in the zone of w.The key step of detection method comprises: extract this regional direction histogram (HoG), good Adaboost human body sorter is classified to this direction histogram to utilize precondition, promptly carry out the human region checking.

If classification results is a human region, then directly with y ₁As position of human body output correct in the present frame.Then, calculate y ₁The color characteristic of the human region at place and space characteristics (therefore computing method repeat no more as described in the step 1) are with itself and y ₁As initial human body color characteristic, space characteristics and the initial position of human body of next frame, in next frame, human body is followed the tracks of respectively so that continue.

Otherwise, with y ₁Be the center, wide is to carry out human detection according to the method for introducing in the above-mentioned list of references in the zone of H for W is high.Briefly, in this zone, carry out window scanning exactly, search out human region, then according to the human region that searches out to position of human body (y ₁) revise, obtain revised position of human body (y ₂).Then, calculate y ₂The color characteristic of the human region at place and space characteristics (therefore computing method repeat no more as described in the step 1) are with them and y ₂As initial human body color characteristic, space characteristics and the initial position of human body of next frame, in next frame, human body is followed the tracks of respectively so that continue.

Should be pointed out that to central point be y ₁The human region checking in zone needn't use the HoG human detection algorithm of standard.

The human region verification method that the present invention proposes is as follows:

Collect N width of cloth human body image in advance, calculate the direction histogram { Hist of these images ₁, Hist ₂..., Hist _N, and obtain the average of these direction histograms (HoG)

Then, computing center's point is y ₁The direction histogram Hist (y in zone ₁), and a threshold value T is set.If $|\mathrm{Hist}\left(y_1\right)-\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{Hist}}_i|<T$ Set up, think that then central point is y ₁The zone be human region, otherwise be non-human region.Threshold value T and quantity N can be provided with according to actual needs.

Fig. 2 is the process flow diagram according to human body tracing method of the present invention.

By process flow diagram shown in Figure 2, the human body tracing method that the present invention may be better understood.At first, in step S100, by human body detecting method or the method for demarcating by hand determine human body initial position y in the present frame ₀In step S200, obtain a pixel set according to the human body initial position.In step S300, calculate the space characteristics and the color characteristic of all pixels in this set.In step S400, calculate position of human body in the present frame according to the space characteristics that calculates and color characteristic.In step S500, the position of human body that draws is detected then, judge that whether it is correct position of human body (that is, carries out the human region checking about this position as described above, judge with this position to be whether the zone at center is human region).If, then directly with y ₁As position of human body output correct in the present frame.If not, then this position is revised, draw correct position of human body.Then in step S700, according to need not to revise or revised correct position of human body calculates the color characteristic and the space characteristics of the human region at its place, with these two features and correct position of human body respectively as initial human body color characteristic, space characteristics and the initial position of human body of next frame.Export this correct position of human body simultaneously.

In sum, by step 1 and step 2 that initialization process and repetition are described in Fig. 1, method of the present invention can successfully realize human body tracking, provides correct current position of human body in any frame of video flowing.Method of the present invention is fused to spatial positional information in the colouring information by improving existing mean shift algorithm, has further improved the human body tracking ability under the complex background situation.Human detection correcting module of the present invention has carried out further detection and correction to tracking results, has prevented the cumulative effect of tracking error, has further improved the robustness of system keeps track.

Obviously, under the prerequisite that does not depart from true spirit of the present invention and scope, the present invention described here can have many variations.Therefore, the change that all it will be apparent to those skilled in the art that all should be included within the scope that these claims contain.The present invention's scope required for protection is only limited by described claims.

Claims (10)

1. human body tracing method comprises:

Step a is according to the initial position (y of human body ₀) obtain the set of pixel;

Step b calculates the space characteristics and the color characteristic of pixel in the described set; Wherein, the space characteristics of this pixel obtains by the position average, and the color histogram of human region that will be relevant with this initial position is as described color characteristic;

Step c is according to the similarity measurement of described space characteristics and described color characteristic definition candidate region and human region;

Steps d is according to the position of human body (y on the described similarity measurement calculating present frame ₁).

2. according to the method for claim 1, also comprise:

To the position of human body (y on the present frame ₁) detect correction, if testing result is a human region, then with described position of human body (y ₁) as need not the modifier body position.

3. according to the method for claim 2, also comprise:

If testing result is non-human region, then with described position of human body (y ₁) carry out window scanning in the relevant zone, search out human region, according to the human region that searches out to position of human body (y ₁) revise, obtain revised position of human body (y ₂).

4. according to the method for claim 2, wherein, described detection comprises:

Calculate the direction histogram in the zone relevant with described position of human body; And;

Judge according to an average with described direction histogram whether this zone is human region, wherein, described average is by collecting some human body images and drawing according to its direction histogram.

5. according to the method for claim 2 or 3, also comprise:

Exporting described need not revises or revised position of human body;

Revise or revised position of human body according to described need not, calculate human space feature and color characteristic and output in the present frame;

With position of human body, space characteristics and the color characteristic of output as human body initial position, space characteristics and the color characteristic of next frame.

6. according to the process of claim 1 wherein, the calculating of described position average is:

If { x _i} _{I=1,2 ... N}Be that central point is y ₀, long is h, wide is the pixel set in the body templates zone of w, definition mapping b:R ²→ 1,2 ..., B}, b (x _i) the quantification sequence number of feature in the feature space that quantizes of representing each pixel,

For any quantization value u ∈ 1,2 ..., B} calculates its position average v _u:

$v_u=\frac{1}{{\mathrm{\&Sigma;}}_{i=1}^N{\mathrm{\&delta;}}_{\mathrm{iu}}}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\frac{2(x_i-y)}{\sqrt{w^2+h^2}}{\mathrm{\&delta;}}_{\mathrm{iu}}$

Wherein, ${\mathrm{\&delta;}}_{\mathrm{iu}}=\left\{\begin{array}{ccc}1& \mathrm{if}& b\left(x_i\right)=u\\ 0& & \mathrm{else}\end{array}\right.$

7. human body tracking equipment comprises:

Video acquisition module is used to gather video streaming image, the output map picture frame;

Initial position (y according to human body ₀) obtain the module of pixel set;

Calculate the module of the space characteristics and the color characteristic of pixel in the described set; Wherein, the space characteristics of this pixel obtains by the position average, and the color histogram of human region that will be relevant with this initial position is as described color characteristic;

Module according to the similarity measurement of described space characteristics and described color characteristic definition candidate region and human body;

According to the position of human body (y on the described similarity measurement calculating present frame ₁) module.

8. according to the equipment of claim 7, also comprise;

The human detection correcting module is used for described position of human body (y ₁) detect correction, wherein, if testing result is a human region, then with described position of human body (y ₁) as need not the modifier body position.

9. equipment according to Claim 8, wherein, described human detection correcting module also is used for:

If testing result is a human region, then with described position of human body (y ₁) carry out window scanning in the relevant zone, search out human region, according to the human region that searches out to position of human body (y ₁) revise, obtain revised position of human body (y ₂).

10. equipment according to Claim 8, wherein, described human detection correcting module comprises:

Calculate the module of the direction histogram in the zone relevant with described position of human body; And;

Judge according to an average with described direction histogram whether this zone is the module of human region, wherein, described average is by collecting some human body images and drawing according to its direction histogram.

Images