CN102682287B - Pedestrian detection method based on saliency information - Google Patents

Abstract

The invention provides a pedestrian detection method based on saliency information. The method comprises an offline training step and an online detection step; the online detection step comprises calculating a salient map of an image to be detected, extracting a detection child window from the image and calculating the corresponding saliency of the detection child window according to the salient map, calculating corresponding features in the detection child window, detecting the corresponding features in the detection child window through a cascade classifier, and simultaneously distributing adjustment coefficients for the cascade classifier according to the corresponding saliency of the detection child window. According to the method, the saliency information is introduced to be served as auxiliary information for pedestrian detection to participate in the process of image identification on the basis of the existing AdaBoost classifier. In most cases, pedestrians are different from the surrounding environment in terms of color, shape and profile, the saliency information of the child window is adopted for correcting detection results of the classifier, the detection rate can be effectively improved, and the false detecting rate can be reduced.

Description

Pedestrian detection method based on significance information

Technical field

The present invention relates to image detecting technique, particularly the pedestrian detection technology based on image.

Background technology

Pedestrian detection is because it is applied widely and has been subject to researchist and commercial company more and more payes attention in recent years, and the research of this aspect has also obtained significant progress.But due to pedestrian itself and surrounding environment intrinsic, accomplish in real time, detect exactly and still face two technological difficulties:

1. pedestrian is a nonrigid object, because angle (front, side, the back side etc.), the difference of situation such as wear, block clothes, has caused the complicacy of pedestrian detection.

2. the diversity of camera angle and attribute, lighting angle and intensity, periphery object etc. all can be given accurately and detect and bring certain difficulty.

Pedestrian detection can be seen two classification problems as, and classification problem is used the method for statistical learning the most effective.Mainly comprise following two aspects: the first, the extraction of different characteristic, features such as color, edge, Haar-like (AdaBoost people's face detects a kind of feature extracting method in training algorithm), profile, gradient; Second, the use of different sorters, as nearest neighbour method, neural network, Support Vector Machine (SVM) and AdaBoost, (Adaboost is a kind of iterative algorithm, its core concept is to train different sorter (Weak Classifier) for same training set, then these Weak Classifiers are gathered, form a stronger final sorter (strong classifier)), Bayes classifier etc.

Feature extraction aspect: color characteristic, when processing pedestrian's situation of different clothing or posture, has significant limitation; Haar-like feature has obtained immense success in the algorithm that people's face detects, but pedestrian is very different than people's face, only according to gray scale, can not intactly describe pedestrian information; Edge orientation histogram (EOH), gradient orientation histogram (HOG) can reflect the shape facility of object well, insensitive to the distortion of direction, yardstick, but its dimension is larger, and computing velocity is slower.

Sorter aspect: using sorter comparatively widely to mainly contain Support Vector Machine (SVM) and AdaBoost and their improvement algorithm at present.Than SVM, AdaBoost algorithm is even better on detection speed, particularly at the AdaBoost of cascade sorter, can make detection speed reach real-time.

The pedestrian detection method of obtaining important breakthrough of generally acknowledging is at present the pedestrian detection method based on gradient orientation histogram (HOG) and Support Vector Machine (SVM) that Dalal in 2005 and Triggs propose.On the basis of this algorithm, a lot of researchists improve extraction and being built with further of sorter of feature.Although pedestrian detection has had and significantly improved in detection speed and precision, still have many improved places that are worth.

On the other hand, the key areas that obvious object detects as computer vision also receives more and more researchists' concern in recent years.So-called obvious object detects, and extracts a width picture and the most easily causes the region that human eye is noted.Obvious object detects and has a wide range of applications, and it can be used as the pre-service of the problems such as object detection, object segmentation, picture reorientation.

Summary of the invention

Technical matters to be solved by this invention is that a kind of method of utilizing remarkable detection to improve pedestrian detection precision is provided.

The present invention solves the problems of the technologies described above adopted technical scheme to be, the pedestrian detection method based on significance information, comprises detecting step on training step under line, line;

Training step under line: collect the positive sample that comprises pedestrian and the negative sample that does not comprise pedestrian; From positive sample and negative sample, extract feature as training data respectively, construct some Weak Classifiers; According to cascade AdaBoost algorithm, some Weak Classifiers are formed to strong classifier again, a plurality of strong classifiers form cascade classifier;

It is characterized in that detecting step on line: the remarkable figure that calculates image to be detected; From image, extract detection subwindow, and calculate and detect significance corresponding to subwindow according to remarkable figure; Calculate and detect individual features in subwindow, utilize cascade classifier to detect detecting individual features in subwindow, by being that cascade classifier distributes adjustment coefficient according to significance corresponding to this detection subwindow, obtain classification results simultaneously; Finally classification results corresponding to all detection subwindows merged, obtain pedestrian detection result; When significance is higher, the probability that detection subwindow is identified as pedestrian's window is larger.

The present invention is different from the existing application in image is processed by remarkable information.In existing image processing and identification process, be first according to significance information, for after image recognition distribute different sampled points, i.e. the sweep limit of different subwindows and frequency.The application that is significance be only for after image recognition determine the use of scope, with image recognition be afterwards separate.The present invention, on the basis of existing AdaBoost sorter, has introduced the supplementary of significance information as pedestrian detection, participates in the process of image recognition.In most cases, pedestrian is being very different aspect CF, profile than surrounding environment, utilizes the testing result of the significance Information revision sorter of subwindow, can effectively improve verification and measurement ratio, reduce false drop rate.

In order to improve the accuracy of significance, the present invention further proposes a kind of new remarkable figure computing method, has introduced the characteristic information of histogram of gradients, is specially:

Testing image is divided into some regions; By calculating the color histogram in each region and other region, the diversity factor of histogram of gradients, draw the remarkable value in this region;

For arbitrary region r in image _i, it is significantly worth S (r _i) be:

$S\left(r_i\right)=\underset{r_k\≠r_i}{\mathrm{\Σ}}\exp (-D_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_k\right)(D_c(r_k,r_i)+D_g(r_k,r_i))$

Wherein, r _kfor removing region r after cutting apart in image _ioutside arbitrary region, D _s(r _i, r _k) be r _iand r _kthe Euclidean distance of regional center, D _c(r _i, r _k), D _g(r _i, r _k) be respectively region r _i, r _kcolor histogram and the Euclidean distance of histogram of gradients, σ _sfor space length weight, ω (r _k) be r _kthe number of pixels in region.Significantly value is larger, and significance is higher;

After the remarkable value of All Ranges is calculated in image, to all remarkable value normalizeds, obtain final remarkable figure I _sal.

Concrete, detect the significance that subwindow is corresponding and reflect by detecting remarkable coefficient corresponding to subwindow:

Detect remarkable coefficient corresponding to subwindow

s wherein _winfor subwindow significance, i.e. the remarkable mean value of all pixel significances in corresponding subwindow in figure, S _averagefor the average significance of picture, i.e. the remarkable mean value of all pixel significances in figure, β is deviation ratio.I _sal(x, y) is the remarkable value after the normalization of pixel (x, y); $S_{\mathrm{win}}=\frac{\underset{(x,y)\∈\mathrm{window}}{\mathrm{\Σ}}{I}_{\mathrm{sal}}(x,y)}{\mathrm{area}\left(\mathrm{window}\right)},$ Area (window) is subwindow number of pixels; $S_{\mathrm{average}}=\frac{\underset{(x,y)\∈\mathrm{image}}{\mathrm{\Σ}}{I}_{\mathrm{sal}}(x,y)}{\mathrm{area}\left(\mathrm{image}\right)},$ The number of pixels that area (image) is whole image.Significance is higher, detects the remarkable coefficient that subwindow is corresponding less.

Particularly, according to significance corresponding to this detection subwindow, be that cascade classifier distributes adjustment coefficient, the concrete mode that obtains classification results is:

$F_s=\mathrm{sign}(\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)-\frac{1}{2}E\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t)$

Wherein, F _sfor s level strong classifier in whole cascade classifier is to detecting the classification results of subwindow, h _tfor F _sin t Weak Classifier, α _tweak Classifier h _tcorresponding weights, T _sfor strong classifier F _sthe number of middle Weak Classifier, E is for detecting remarkable coefficient corresponding to subwindow; It is 1 that value in sign () function is more than or equal to 0, the s level strong classifier Output rusults, otherwise is 0; In whole cascade classifier, the Output rusults of all strong classifiers is 1, differentiate for pedestrian's subwindow, otherwise differentiate, be without pedestrian's subwindow.

Further, in order to have brought into play existing each category feature in the advantage aspect description gray scale and gradient, more fully embody the characteristic that pedestrian is different from other object, calculate and detect individual features in subwindow, in calculating AdaBoost sorter conventional Haar-like rectangular characteristic, also edge calculation direction histogram feature and gradient orientation histogram feature.Haar-like rectangular characteristic, edge orientation histogram feature and gradient orientation histogram feature, except the rectangle that comprises horizontal and vertical direction, also comprise the rectangular characteristic of 45 ° of rotations.

The invention has the beneficial effects as follows, more existing pedestrian's algorithm, is close under consistent condition at detection speed, and accuracy of detection is higher, and false drop rate is lower, and robustness is stronger.

Accompanying drawing explanation

Fig. 1 is overall flow figure.

Fig. 2 is edge orientation histogram, gradient orientation histogram feature extraction process flow diagram.

Fig. 3 is remarkable figure schematic diagram.

Fig. 4 is experimental result schematic diagram.

Embodiment

Implementing procedure figure as shown in Figure 1.

Training process under step 1, line:

The 1st step: the generation of sample: collect the picture that comprises pedestrian and do not comprise pedestrian, respectively as positive sample, negative sample.

The 2nd step: the calculating of feature: calculate its gray integration figure, gradient integrogram for samples pictures, calculate Haar-like rectangular characteristic, edge orientation histogram feature, the gradient orientation histogram feature of the rectangle of different scale in sample graph, position and size, while calculating rectangular characteristic, get horizontal direction, vertical direction, and the rectangle of 45 ° of directions of left rotation and right rotation.

The computing method of gray integration figure and gradient integrogram, Haar-like rectangular characteristic, edge orientation histogram feature, gradient orientation histogram feature be existing ripe computing method all, at this, do not repeat.

The 3rd step: dimensionality reduction: adopting Fisher linear discriminant is same Haar-like rectangle, the consistent one-dimensional characteristic of edge orientation histogram by the gradient orientation histogram Feature Conversion of multidimensional, then Haar-like rectangular characteristic, edge orientation histogram feature and gradient orientation histogram feature are formed to composite character storehouse.

The 4th step: build sorter: according to cascade AdaBoost algorithm, the training data in composite character storehouse is learnt and trained, and by these training data structure Weak Classifiers, again some Weak Classifiers are formed to strong classifier, a plurality of strong classifiers form cascade classifier.

Testing process on step 2, line:

The 1st step: extract picture or video frame images data.

The 2nd step: calculated product component: image is carried out to convergent-divergent and the gray processing of different scale, calculate respectively gray integration figure, gradient integrogram under each yardstick, as shown in Figure 2.

The 3rd step: ask for remarkable figure I _sal: picture is adjusted into compared with small scale, and utilizing the image partition method based on figure is some regions by picture segmentation; By calculating the color histogram in each region and other region, the diversity factor of histogram of gradients, show that picture significantly schemes I _salas shown in Figure 3;

For arbitrary region r _i, it is significantly worth S (r _i) be:

$S\left(r_i\right)=\underset{r_k\≠r_i}{\mathrm{\Σ}}\exp (-D_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_k\right)(D_c(r_k,r_i)+D_g(r_k,r_i));$

Wherein, r _kfor removing region r after cutting apart in image _ioutside arbitrary region, D _s(r _i, r _k) be r _iand r _kthe Euclidean distance of regional center, D _c(r _i, r _k), D _g(r _i, r _k) be respectively region r _i, r _kcolor histogram and the Euclidean distance of histogram of gradients, σ _sfor space length weight, ω (r _k) be r _kthe number of pixels in region.For the remarkable value normalization of All Ranges, just can obtain final remarkable figure I _sal.

The 4th step: testing process:

1) picture after each convergent-divergent is carried out to the scanning of each position, scanning window is 128 * 64;

2) according to integrogram, calculate fast each feature (comprising the Haar-like rectangular characteristic being obtained by gray integration figure, the edge orientation histogram feature obtaining according to gradient integrogram and gradient orientation histogram feature) corresponding under sorter;

3) calculate the remarkable coefficient of scanning window s wherein _winfor subwindow significance, i.e. the remarkable pixel average of corresponding subwindow in figure, S _averagefor the average significance of picture, i.e. remarkable all pixel average in figure, β is deviation ratio;

$S_{\mathrm{window}}=\frac{\underset{(x,y)\∈\mathrm{window}}{\mathrm{\Σ}}{I}_{\mathrm{sal}}(x,y)}{\mathrm{area}\left(\mathrm{window}\right)},$ Area (window) is window number of pixels;

$S_{\mathrm{average}}=\frac{\underset{(x,y)\∈\mathrm{image}}{\mathrm{\Σ}}{I}_{\mathrm{sal}}(x,y)}{\mathrm{area}\left(\mathrm{image}\right)},$ Area (image) is image pixel number;

4) discriminant of AdaBoost strong classifier is introduced after remarkable coefficient, and this subwindow is differentiated:

$F_s=\mathrm{sign}(\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)-\frac{1}{2}E\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t)$

Wherein, F _sfor s level strong classifier in whole cascade classifier, h _tfor f _sin t Weak Classifier (t the feature of selecting in corresponding training process), α _tweak Classifier h _tweights, T _sfor strong classifier F _sthe number of middle Weak Classifier.It is 1 that value in sign () function is more than or equal to 0, the s level strong classifier Output rusults, otherwise is 0; In whole cascade classifier, the Output rusults of all strong classifiers is 1, differentiate for pedestrian's subwindow, otherwise differentiate, be without pedestrian's subwindow.

5) obtain testing result.

The 5th step: the testing result of scaling pictures is mapped to original image in proportion, merge the detection window of lap, obtain final testing result, mark the position that all possible personage occurs, as shown in Figure 4.

Applicant moves on the computing machine of Pentium Dual-Core 2.60GHz processor, 2GB internal memory, use the present embodiment method, detection speed for 320 * 240 pictures is about 200ms, guarantee that accurate performance does not have in the situation of loss, in real time, accurately detecting of picture concerned or video can be completed, the fields such as intelligent transportation, video monitoring, compression of images, multimedia retrieval can be widely used in.

Claims (3)

1. the pedestrian detection method based on significance information, comprises detecting step on training step under line, line;

Training step under line: collect the positive sample that comprises pedestrian and the negative sample that does not comprise pedestrian; From positive sample and negative sample, extract feature as training data respectively, construct some Weak Classifiers; According to cascade AdaBoost algorithm, some Weak Classifiers are formed to strong classifier again, a plurality of strong classifiers form cascade classifier;

It is characterized in that detecting step on line: the remarkable figure that calculates image to be detected; From image, extract detection subwindow, and calculate and detect significance corresponding to subwindow according to remarkable figure; Calculate and detect individual features in subwindow, utilize cascade classifier to detect detecting individual features in subwindow, by being that cascade classifier distributes adjustment coefficient according to significance corresponding to this detection subwindow, obtain classification results simultaneously; Finally classification results corresponding to all detection subwindows merged, obtain pedestrian detection result; When significance is higher, the probability that detection subwindow is identified as pedestrian's window is larger;

Wherein, the concrete grammar that calculates the remarkable figure of image to be detected is:

Testing image is divided into some regions;

For arbitrary region r in image _i, it is significantly worth S (r _i) be:

$S\left(r_i\right)=\underset{r_k\≠r_i}{\mathrm{\Σ}}\exp ({-D}_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_k\right)(D_c(r_k,r_i)+D_g(r_k,r_i))$

Wherein, r _kfor image is cut apart rear except region r _ioutside arbitrary region, D _s(r _i, r _k) be r _iand r _kthe Euclidean distance of regional center, D _c(r _i, r _k), D _g(r _i, r _k) be respectively region r _i, r _kcolor histogram and the Euclidean distance of histogram of gradients, σ _sfor space length weight, ω (r _k) be r _kthe number of pixels in region; Significantly value is larger, and significance is higher;

After the remarkable value of All Ranges is calculated in image, to all remarkable value normalizeds, obtain final remarkable figure I _sal;

Detecting the significance that subwindow is corresponding reflects by detecting remarkable coefficient corresponding to subwindow:

Detect remarkable coefficient corresponding to subwindow s wherein _winfor subwindow significance, S _averagefor the average significance of picture, β is deviation ratio;

area (window) is subwindow number of pixels, I _sal(x, y) is the remarkable value after the normalization of pixel (x, y);

the number of pixels that area (image) is whole image; Subwindow significance is higher, detects the remarkable coefficient that subwindow is corresponding less.

2. the pedestrian detection method based on significance information as claimed in claim 1, is characterized in that, according to significance corresponding to this detection subwindow, is that cascade classifier distributes and adjusts coefficient, and the concrete mode that obtains classification results is:

$F_s=\mathrm{sign}(\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)-\frac{1}{2}E\underset{t=1}{\overset{T_s}{\mathrm{\Σ}}}{\mathrm{\α}}_t)$

Wherein, F _sfor s level strong classifier in whole cascade classifier is to detecting the classification results of subwindow, h _tfor F _sin t Weak Classifier, α _tweak Classifier h _tcorresponding weights, T _sfor strong classifier F _sthe number of middle Weak Classifier, E is for detecting remarkable coefficient corresponding to subwindow; Work as sign() to be more than or equal to 0, the s level strong classifier Output rusults be 1 for value in function, otherwise be 0; In whole cascade classifier, the Output rusults of all strong classifiers is 1, differentiate for pedestrian's subwindow, otherwise differentiate, be without pedestrian's subwindow.

3. the pedestrian detection method based on significance information as claimed in claim 1, it is characterized in that, described feature comprises Haar-like rectangular characteristic, edge orientation histogram feature and gradient orientation histogram feature, and described Haar-like rectangular characteristic, edge orientation histogram feature and gradient orientation histogram feature all comprise the rectangular characteristic of horizontal direction, vertical direction, 45 ° of directions of rotation.

Images