CN107862261A - Image people counting method based on multiple dimensioned convolutional neural networks - Google Patents

Abstract

The invention discloses an image crowd counting method based on a multi-scale convolutional neural network, step (1), generating continuous density map labels, and transforming marked images into continuous estimated density maps; step (2), using The multi-scale convolutional neural network obtains the accurate density map of the predicted crowd. After setting an initial parameter for the convolutional neural network, the loss L(θ) of the input image is calculated according to the actual density map, and then the entire network is updated in each optimization iteration. parameter until the loss value converges to a smaller value. Compared with the existing technology, the present invention solves the huge scale change of the crowd in a single image. On the basis of a single convolutional neural network, the features of different levels of networks are integrated before generating the predicted density map, and the corresponding images of different depths are extracted. The features of different scales greatly improve the accuracy of the predicted density map; solve the problems of scale change and occlusion in crowd images.

Description

Image crowd counting method based on multi-scale convolutional neural network

technical field

The invention relates to the technical field of crowd image analysis, in particular to a crowd counting algorithm based on a multi-scale convolutional neural network.

Background technique

Crowd counting is an intelligent surveillance application that calculates the number of people by predicting the density map of crowd images. With the exponential growth of the world population, rapid urbanization has promoted many large-scale activities, such as sports competitions, public parades, traffic congestion and other issues leading to large-scale crowd gatherings. Therefore, in order to better manage crowds and personal safety, crowd behavior analysis algorithms are of great significance.

With the continuous promotion of deep learning algorithms, the crowd counting algorithm based on convolutional neural network has greatly improved the detection accuracy compared with traditional algorithms. Algorithms based on convolutional neural networks are mainly divided into two types: one is a regression-based algorithm, and the other is a density map-based algorithm. The former uses the crowd image and the corresponding number of people as labels, trains the convolutional neural network to learn a nonlinear function mapping from the crowd image to the number of people, and the output of the network is the number of people. The latter uses the crowd image and the corresponding density map as labels to train the convolutional neural network to generate a density map corresponding to the input crowd image. Unlike the regression method, the algorithm network based on the density map uses the density map as the output. Calculate the crowd size based on the predicted density map. However, since crowd images are mostly taken by surveillance cameras and high altitudes, there are great changes in shooting angles, and there are great changes in the size and scale of people in the captured images. The multi-column convolutional neural network proposed by Zhang et al. has high network complexity and large network parameters. The three-column network needs to be pre-trained and then the output features of the multi-column network are fused. It cannot grasp the multi-scale information of a single image at the same time. .

Contents of the invention

The purpose of the present invention is to extract features of different depths using convolutional neural networks, and to integrate features of different scales, and propose a crowd density detection method based on multi-scale convolutional neural networks, which is calculated by predicting density maps from crowd images total number of people.

A kind of image crowd counting method based on multi-scale convolutional neural network of the present invention, the method comprises the following steps:

Step 1, generate continuous density map labels, specifically including the following processing:

The corresponding density map is generated by manually marking the human head coordinates, and the image with N human head marks is expressed as the following function:

In the formula, δ(xx _i ) is a delta function; _xi represents the position of a human head marking point;

Convert the labeled image into a continuous density map, the expression is as follows:

F(x)=H(x) ^*

Step 2. Use the multi-scale convolutional neural network to obtain the accurate density map of the predicted population, which specifically includes the following processing:

The multi-scale convolutional neural network obtains three convolutional layers through the connection of convolution-pooling-reconvolution-repooling, extracts the features of different receptive fields from the first three convolutional layers, and combines these features in cascade The method is fused, and then the corresponding density map is output through two convolutional layers;

Calculate the loss function L(θ) of the multi-scale convolutional neural network, the expression is as follows:

Wherein, N is the number of images input to the convolutional neural network, x _i is the i-th input image of the convolutional neural network, and M( _xi ) represents the standard density map matrix of the i-th input image;

After setting an initial parameter for the convolutional neural network, calculate the loss L(θ) of the input image according to the actual density map, and then update the parameters of the entire network in each optimization iteration until the loss value converges to a smaller value.

Compared with the prior art, the image crowd counting method based on the multi-scale convolutional neural network of the present invention has the following effects:

1. Able to use a single-column convolutional neural network to detect pedestrians of different scales in crowd images in combination with features of different depths in the case of low parameters;

2. It solves the huge scale change of the crowd in a single image. On the basis of a single convolutional neural network, the features of different levels of networks are integrated before generating the predicted density map, and the features of different depths corresponding to different scales are extracted, which greatly improves Improved the accuracy of predicted density maps;

3. Solve the problems of scale change and occlusion in crowd images.

Description of drawings

Fig. 1 is a schematic diagram of the overall flow of the multi-scale convolutional neural network-based image crowd counting method of the present invention;

Figure 2 is a structural diagram of a multi-scale convolutional neural network;

Figure 3 is the result of the experiment; Figure (a) is the crowd image, and Figure (b) is the corresponding density map.

Detailed ways

Embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

As shown in Figure 1, a crowd density detection method based on a multi-scale convolutional neural network of the present invention fuses the features of a single-column convolutional neural network at different depths, and the specific steps are as follows:

Step 1. Generate a continuous density map label, and convert the labeled image into a continuous estimated density map, which specifically includes the following processing:

The corresponding density map is generated by manually marking the human head coordinates, and the image with N human head marks is expressed as the following function:

In the formula, δ(xx _i ) is a delta function; _xi represents the position of a human head marking point;

The expression of the estimated density map F(x) is as follows:

F(x)=H(x) ^*

;

Step 2. Use the multi-scale convolutional neural network to obtain the accurate density map of the predicted population: the multi-scale convolutional neural network obtains three convolutional layers through the connection of convolution-pooling-reconvolution-repooling, from the first three The convolutional layer extracts features of different receptive fields. These features are composed of multi-level features extracted from three convolutional layers of different depths. As the network deepens, the higher the convolutional layer, the larger the receptive field. , the features extracted in the low-level convolutional layer can obtain more detailed information of small objects, and the high-level convolutional layer obtains advanced semantic features, and these features are fused in a cascaded and combined manner, That is, the feature map is superimposed, and then the corresponding density map is output through two convolutional layers. The loss function of this network is the Euclidean distance L(θ) between the estimated density map F( _xi ; θ) and the actual density map M( _xi ), the specific expression is as follows:

Among them, N is the number of images input to the convolutional neural network, _xi is the i-th input image of the convolutional neural network, and M( _xi ) represents the precise density map matrix of the i-th input image;

After setting an initial parameter for the convolutional neural network, calculate the loss L(θ) of the input image according to the actual precise density map, and then update the parameters of the entire network in each optimization iteration until the loss value converges to a smaller value .

Due to the shooting angle of the camera, the crowd images often have different degrees of perspective distortion. The overall performance is that the pedestrians who are closer to the camera occupy a larger area in the image, and the pedestrians who are far away from the camera occupy a smaller area in the image. In this step, a multi-scale convolutional neural network is used to monitor pedestrians of different scales in the crowd image. In a convolutional neural network, features at different depths in the network represent features at different levels. The convolutional neural network extracts the contour and shape features of the image at the low level, and the receptive field is relatively small. As the number of network layers deepens, the deep network extracts the high-level semantic features of the image, and the features of different levels in the network are analyzed. Superposition fusion combines the multi-scale features of the crowd image well, and finally produces a more accurate prediction of the crowd density map.

Specific embodiments are described as follows:

The problem to be solved in the present invention is "given a crowd image or a frame in a video, and then estimating the density and total number of people in each area of the image":

Express the known input image as a matrix of M×N: x∈R ^m×n , then the actual crowd density corresponding to the input image x is expressed as:

Among them, N represents the number of people in the image, x represents the position of each pixel in the image, x _i represents the position of the i-th head in the image, δ(xx _i ) represents the impact function, * represents the symbol of the convolution operation, G _δ (x) represents a Gaussian kernel with standard deviation δ.

The goal of this example is to learn a mapping function from an input image x to a crowd density map:

F:x→F(x)≈M(x)

Among them, F(x) is the estimated population density map.

In order to learn F, the following problems need to be optimized:

Among them, F(x; θ) is the estimated crowd density map, and θ is the parameter to be learned. In general, F is a complex nonlinear function.

As shown in FIG. 2 , it is a multi-scale convolutional neural network used by the present invention to learn a nonlinear function F from a crowd image to a density map. The multi-scale convolutional neural network is to fuse the features of different depth levels. The feature map of the first layer of the single-column convolutional neural network undergoes one convolution and two pooling, the second layer feature map undergoes one convolution and one pooling, and the features obtained by the first two layers and the features obtained by the third layer convolution The graphs are linked together in the "channel" dimension to form the total feature map Merged feature maps, which are then passed through two convolutional layers to obtain the final density map.

The loss function of the above multi-scale convolutional neural network is the Euclidean distance between the estimated density map and the actual density map:

During the training process, the gradient descent method is used to update the parameters L(θ) of the entire network in each optimization iteration until the loss value converges to a smaller value.

The present invention is compared with other methods on three public datasets, including shopping mall datasets MALL, UCSD and SHANGHAITECH datasets. The evaluation criteria for the experimental results are as follows:

Mean Absolute Error (MAE):

and mean squared error (MSE):

N is the number of pictures, z _i is the actual number of heads in the i-th image, The accuracy of the algorithm is measured by the number of heads output by the network provided by the present invention for the i-th image. On the MALL shopping mall data set, the technical comparison of the present invention and existing algorithm, as shown in table 1 (wherein MD-CNN is the algorithm of the present invention):

Table 1

On the UCSD data set, the present invention is compared with the prior art, as shown in Table 2:

Table 2

Method MAE MSE Kernelridge regression 2.16 7.45 Ridge regression 2.25 7.82 Gaussian process regression 2.24 7.97 Cumulative attribute regression 2.07 6.86 Zhangetal. 1.60 3.31 MCNN 1.07 1.35 MDCNN(ours) 1.16 1.75

The comparison with other existing algorithms on the SHANGHAITECH part_B dataset is shown in Table 3:

table 3

Method MAE MSE LBP+RR 59.1 87.1 Zhangetal. 32 49.8 MCNN 26.4 41.3 MDCNN(ours) 22.3 39.45

Claims (1)

1. a kind of image people counting method based on multiple dimensioned convolutional neural networks, it is characterised in that this method includes following Step：

Step (1), the continuous density map label of generation, the image marked is converted into continuous estimation density map, specific bag Include following processing：

By density map corresponding to the good number of people Coordinate generation of handmarking, the graphical representation with N number of people's labeling head is following letter Number：

<mrow> <mi>H</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <mi>&amp;delta;</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>-</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow>

In formula, δ (x-x_i) it is delta function；x_iRepresent the position where people's leader note point；

Estimate that density map F (x) expression formula is as follows：

F (x)=H (x)^*

；

Step (2), the accurate density map of prediction crowd is obtained using multiple dimensioned convolutional neural networks, specifically include following processing：

Multiple dimensioned convolutional neural networks obtain three convolutional layers by the connection in pond of the convolution of convolution-pond-again-again, from first three Individual convolutional layer extracts the wild feature of different feeling, and these features are merged in a manner of cascading merging, then by two Density map corresponding to convolutional layer output；

The loss function L (θ) of the multiple dimensioned convolutional neural networks is calculated, expression formula is as follows：

<mrow> <mi>L</mi> <mrow> <mo>(</mo> <mi>&amp;theta;</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mi>I</mi> <mrow> <mn>2</mn> <mi>N</mi> </mrow> </mfrac> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <mo>|</mo> <mo>|</mo> <mi>F</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>,</mo> <mi>&amp;theta;</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>M</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>|</mo> <msubsup> <mo>|</mo> <mn>2</mn> <mn>2</mn> </msubsup> </mrow>

Wherein, N be input convolutional neural networks amount of images, x_iFor the i-th width input picture of convolutional neural networks, M (x_i) table Show the accurate density map matrix of the i-th width input picture；

After one initial parameter is set for convolutional neural networks, the loss L of input picture is calculated according to the accurate density map of reality (θ), the parameter of whole network is then updated in Optimized Iterative each time, until penalty values converge to a less value.