CN109446920B - Detection method of urban rail transit passenger congestion based on convolutional neural network - Google Patents

Abstract

The invention discloses a method for detecting passenger congestion in urban rail transit based on a convolutional neural network. First, the video to be detected is preprocessed, segmented and the motion residual image is extracted, and the original image and the motion residual image are combined as a convolution The input of the neural network algorithm is to establish a feature extraction block including at least one convolutional layer and a maximum pooling layer, process and calculate the crowd state features contained in the original image and the motion residual image, and then combine the crowd state features with the motion features, Construct a feature fusion block that includes at least one convolutional layer, a maximum pooling layer, and a fully connected layer, perform fusion processing, and construct a classifier at the same time, use a prefabricated training set with a crowded degree label to train the convolutional neural network, so that The classifier correctly detects the degree of passenger congestion in the video to be tested, more comprehensively characterizes the passenger flow situation in the surveillance video, realizes the detection of the degree of congestion, and improves the accuracy of algorithm detection.

Description

Detection method of urban rail transit passenger congestion based on convolutional neural network

Field

The invention belongs to the technical field of rail transit transportation, and in particular relates to a method for detecting passenger congestion degree of urban rail transit based on a convolutional neural network.

Background technique

With the continuous acceleration of the urbanization process and the gradual improvement of the rail transit network pattern, urban rail transit has become the main undertaker of urban public transportation. The rapid growth of passenger flow puts forward higher requirements on the level of daily operation and management. On the one hand, in order to formulate a reasonable traffic plan and passenger flow organization plan, efficiently utilize operating resources and meet the rapidly changing travel needs of passengers on a large-scale online network, it is necessary to accurately grasp the passenger flow status and passenger flow data; on the other hand, due to rail transit The station is usually located in a closed underground or elevated track, and the area of the station hall is relatively limited. When the peak passenger flow arrives, the influx of a large number of passengers will easily cause congestion in the station hall and channel blockage. The large-density crowd not only makes it difficult to guide the passenger flow, but also easily causes large-scale group safety accidents, causing adverse social impacts. Therefore, a convenient and efficient method is needed to obtain the passenger flow distribution in real time, monitor the passenger flow status of the station, provide strong technical support for passenger flow organization, and ensure the safety of passengers and the normal operation of rail transit.

Existing urban rail transit stations are generally equipped with a complete video monitoring system. The content of the monitoring video clearly reflects the degree of passenger flow congestion within the monitoring range, and contains a large amount of effective passenger flow information. In the past, due to technical limitations, the acquisition speed and accuracy of information in video images were difficult to meet the needs of practical applications. With the continuous development of image processing technology, machine learning and computer performance, intelligent image recognition technology has emerged. By combining the image recognition technology with the video surveillance system installed in public places, the computer is used to process the passenger targets contained in the images captured by the surveillance cameras, automatically detect and judge the passenger flow status, and find abnormal targets and abnormal scenes in time. Send out an alarm to realize automatic and intelligent detection and monitoring of passenger flow congestion in urban rail transit.

Due to the obvious visual features of the crowd, most of the early crowd density estimation methods reflect the crowding degree by extracting the visual features of the crowd gathering. These methods include two categories: pixel-based and texture-based. The principle of the pixel-based detection method is simple, and it has achieved good recognition results in scenes with moderate crowd density. However, in crowded places such as stations and shopping malls, due to the serious mutual occlusion caused by excessive crowd density, the performance decline is more obvious; based on Texture features do not perform well in real-time performance, and the two methods are not satisfactory in actual operation and application. Therefore, a method with better performance is needed to realize the detection of passenger flow congestion in urban rail transit.

Contents of the invention

The present invention is aimed at the problems in the prior art, and provides a method for detecting passenger congestion in urban rail transit based on a convolutional neural network, which overcomes the problems of large measurement errors and poor real-time performance in the prior art. Based on the surveillance video collected by the surveillance system, using the powerful image recognition ability of the convolutional neural network, the mixed features of the crowd image and the motion residual image are extracted by constructing a multi-level convolutional neural network, so as to more comprehensively characterize the image in the surveillance video. Passenger flow status, to achieve the detection of congestion, improve the accuracy of algorithm detection.

In order to achieve the above object, the technical scheme that the present invention adopts is: the urban rail transit passenger crowding degree detection method based on convolutional neural network, comprises the steps:

S1, acquiring traffic monitoring video to be detected, preprocessing the video to be detected, segmenting and extracting motion residual images;

S2, combine the original image and the motion residual image as the input of the convolutional neural network algorithm, establish a feature extraction block including at least one convolutional layer and a maximum pooling layer, process the original image and the motion residual image, and calculate the original image respectively and the crowd state features contained in the motion residual image;

S3. Combining crowd state features with motion features, constructing a feature fusion block including at least one convolutional layer, a maximum pooling layer, and a fully connected layer, and performing fusion processing on the feature map obtained in step S2:

S4, constructing a classifier including a degree of congestion level;

S5, using the prefabricated training set with the degree of congestion label to train the convolutional neural network, so that the classifier can correctly detect the degree of passenger congestion in the video to be tested.

As an improvement of the present invention, the step S1 further includes,

S11, acquiring traffic monitoring video to be detected;

S12, setting the detection cycle T, dividing the video to be detected into video segments with a length T according to the detection cycle T, and the first frame image of the video segment is a reference image;

S13. Taking other images in the video segment and making differences with the reference image respectively to obtain a motion residual image.

As an improvement of the present invention, the number of each feature extraction block in the step S2 is 1, and the connection mode of the convolution layer and the pooling layer is alternate connection.

As another improvement of the present invention, the feature fusion block in the step S3 uses the feature map output by the feature extraction block as input, and the number of the feature fusion blocks is one.

As another improvement of the present invention, the number of fully connected layers in the step S3 is 3, and it is the last three layers, and the connection mode of the convolutional layer and the pooling layer is alternately connected, and they are all located in front of the fully connected layer .

As another improvement of the present invention, in the step S4, the grades of the classifiers are divided into ten layers, which are spacious: 0-2; comfortable: 3-5; crowded: 5-8; dangerous: 9-10 class.

As a further improvement of the present invention, during the training of the convolutional neural network in step S5, a stochastic gradient descent algorithm is used to correct parameters in the convolutional neural network, and the formula of the stochastic gradient descent method is as follows:

g(θ)=∑θx ⁱ

θ _m ＝θ _m-1 -η▽ _θ h(θ)

Among them, g(θ) represents the network hypothesis function, θ represents the parameter weight of the convolutional neural network, h(θ) represents the loss function, y ⁱ represents the sample value of the i-th sample, m represents the total number of algorithm iterations, σ Represents the penalty coefficient, ▽ _θ represents the gradient, and η represents the learning rate in gradient descent.

As a further improvement of the present invention, in the step S5, when the detected result is lower than the actual degree of congestion, the penalty coefficient σ=1+log(y ⁱ −g _θ ( ^xi )), otherwise σ=1.

Compared with the prior art, the present invention proposes a method for detecting passenger congestion in urban rail transit based on a convolutional neural network. The images and detection results collected by the monitoring system can reflect the actual passenger flow conditions within the monitoring area in real time and effectively; the division standard of the degree of congestion is combined with the actual scene to meet the operational needs of the place of use; real-time detection can minimize manual viewing of monitoring video workload and guide the passenger flow control work to improve the safety and service quality of urban rail transit operations.

Description of drawings

Fig. 1 is a schematic diagram of detection method steps of the present invention;

Fig. 2 is a schematic diagram of the connection mode of the convolutional layer and the maximum pooling layer in Embodiment 2 of the present invention;

Fig. 3 is a connection mode of the convolutional layer, the maximum pooling layer and the fully connected layer in the feature fusion block according to Embodiment 3 of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

Example 1

The detection method of urban rail transit passenger congestion based on convolutional neural network, as shown in Figure 1, includes the following steps:

S1, acquiring traffic monitoring video to be detected, preprocessing the video to be detected, segmenting and extracting motion residual images;

S11, acquiring traffic monitoring video to be detected;

S12, setting the detection period T, according to the detection period T, the video to be detected is divided into video segments whose length is T, and the first frame image of the video segment is a reference image, denoted as p ₁ ;

S13, take other images in the video segment, for example take a frame of images at 1/3T, 2/3T and T in the detection unit respectively, denoted as p ₂ , p ₃ , p ₄ , set p ₂ , p ₃ , p ₄ Make difference with the reference image p ₁ respectively to obtain the motion residual images of the crowd in the video segment to be detected, denoted as p ₁₂ , p ₁₃ , p ₁₄ .

S2, combining the reference image p ₁ and the motion residual images p ₁₂ , p ₁₃ , p ₁₄ into a group as the input of the passenger congestion degree convolutional neural network algorithm, for the input reference image p ₁ and the motion residual image p ₁₂ , p ₁₃ , p ₁₄ , build the first, second, third, and fourth feature extraction blocks corresponding to the four input images, calculate the crowd features contained in the reference image and the motion features contained in the residual image, the feature extraction A block contains at least one convolutional layer and a max pooling layer;

The first, second, third, and fourth feature extraction blocks are the input of the convolutional neural network, and the number of the first, second, third, and fourth processing blocks is 1, and the convolution layer and the maximum pooling layer The connection method is alternate connection.

S3, combining crowd state features and motion features, constructing a feature fusion block comprising at least one convolutional layer, a maximum pooling layer and a fully connected layer, and performing fusion processing on the feature map obtained in step S2, the fusion method is to The feature maps output by the first, second, third, and fourth processing blocks are added quantitatively, and then input into the fusion block for convolution operation.

The feature fusion block takes the feature map output by the first, second, third, and fourth feature extraction blocks as input, the number of feature fusion blocks is 1, the number of fully connected layers is 3, and it is the last three layers. The convolutional layer The connection method with the pooling layer is alternate connection, and they are all located in front of the fully connected layer. Connection layer, as shown in the following table:

S4, constructing a classifier including levels of crowding, the classifier levels are divided into ten layers, respectively spacious: 0-2 levels; comfortable: 3-5 levels; crowded: 5-8 levels; dangerous: 9-10 levels , so that the convolutional neural network has the ability to divide the degree of passenger congestion contained in the video to be detected.

S5, use the prefabricated training set with the degree of congestion label to train the convolutional neural network, and update the parameters in the network according to the improved gradient descent method, and finally the convolutional neural network that is trained by the video input to be detected makes the classifier Make a reasonable judgment on the degree of passenger congestion in the image.

Example 2

The difference between this embodiment and Embodiment 1 is that: in step S5 to train the convolutional neural network, the stochastic gradient descent algorithm is used to correct the parameters in the convolutional neural network, and the formula of the stochastic gradient descent method is as follows :

g(θ)=∑θx ⁱ

θ _m ＝θ _m-1 -η▽ _θ h(θ)

Among them, g(θ) represents the network hypothesis function, θ represents the parameter weight of the convolutional neural network, h(θ) represents the loss function, y ⁱ represents the sample value of the i-th sample, m represents the total number of algorithm iterations, σ Represents the penalty coefficient, ▽ _θ represents the gradient, and η represents the learning rate in gradient descent;

The convolutional neural network structure parameter update method involved in this method adopts an improved stochastic gradient descent method. Compared with the traditional method, the improved stochastic gradient descent method adds a penalty coefficient to the loss function in combination with actual operational requirements. In actual operating conditions, when the detection result of the congestion level is lower than the actual congestion level, it may lead to the delay of subsequent management measures, which in turn affects the normal operation of urban rail transit, and its adverse impact is far greater than that of the situation where the detection result is higher than the actual congestion level. Therefore, in order to reduce the number of detection results that are lower than the actual degree of congestion, a penalty coefficient is added to the loss function, so that when the detection results are lower than the actual degree of congestion, the adjustment range of the parameters increases. When the detected result is lower than the actual degree of congestion, σ=1+log(y ⁱ −g _θ ( ^xi )), otherwise σ=1.

Example 3

Step 1: Set the detection period T. In this embodiment, T=3 sec is taken as an example, and the video to be detected is divided into video segments with a length T according to the detection period T, which is hereinafter referred to as a detection unit. Take the first frame image of the detection unit as the reference image, denoted as p ₁ ; take the images at t=1s, t=2s and t=3s in the detection unit, denote as p ₂ , p ₃ , p ₄ ; denote p ₂ , p ₃ , p ₄ are respectively compared with the reference image p ₁ to obtain the motion residual image of the crowd in the detection unit, denoted as p ₁₂ , p ₁₃ , p ₁₄ ; the reference image p ₁ and the motion residual image p ₁₂ , p ₁₃ and p ₁₄ are combined into a group as the input of passenger congestion degree detection algorithm; according to the actual operation of urban rail transit, the congestion degree is divided into 10 levels: spacious {(0-2 level), comfortable (3-5 level ), crowding (level 5-8), danger (level 9-10)}, intercept images of surveillance video under different levels of crowding, add a label representing the level of crowding, as a training set for the neural network;

Step2: For the input reference image p ₁ and the motion residual image p ₁₂ , p ₁₃ , p ₁₄ , construct the first, second, third, and fourth processing blocks corresponding to the four input images, and calculate the The crowd features and the motion features contained in the residual image, the above processing blocks all include 4 convolutional layers and the maximum pooling layer, the connection form is shown in Figure 2; assume that the input image is a 224*224 color image that passes through the convolutional layer C1, use the convolution kernel of 11*11*3 to perform convolution operation, the convolution step size is 4, and generate a feature map of 48 layers of 55*55 pixels; after the maximum pooling layer MP1, the pooling operation scale is 3* 3. The step size is 2. After pooling, a feature map of 96 layers of 27*27 pixels is generated; a feature map of 256 layers of 13*13 is generated through the convolution layer C2 and the maximum pooling layer MP2; after the convolution layer C3 and the maximum The pooling layer MP3 generates a 384-layer 13*13 feature map; the convolutional layer C4 and the maximum pooling layer MP4 generate a 384-layer 13*13 feature map;

Step3: Build a feature fusion block to fuse the four sets of 13*13*384 feature maps containing crowd features and motion features output by the first, second, third, and fourth processing blocks. The feature fusion block includes a convolutional layer, a maximum pooling layer and 3 fully connected layers, and the connections are as shown in Figure 3; wherein, the convolutional kernel size of the convolutional layer C5 is 3*3, and the convolution step size is It is 1, and four groups of 13*13*256 feature maps are obtained after convolution operation; and then through the maximum pooling layer MP5 with a pooling scale of 3*3 and a step size of 2, four groups of 6*6*256 features are obtained Figure; 4096 outputs are obtained through three layers of fully connected layers with 4096 neurons.

Step4: Build a classifier, including 10 levels of congestion, namely {spacious (level 0-2), comfortable (level 3-5), crowded (level 5-8), dangerous (level 9-10)}, making the network It has the ability to divide the degree of passenger congestion included in the detection unit. The 4096 outputs of the fully connected layer are fully connected with 10 neurons of the classifier.

Step5: Use the preset training set to train the previously constructed convolutional neural network, and correct the parameters in the network according to the improved gradient descent method. The improved stochastic gradient descent method is based on the traditional method, and a penalty coefficient is added to the loss function according to the actual operation requirements. When the detection result is lower than the actual degree of congestion, σ=1+log(y ⁱ -g _θ (x ⁱ )), otherwise σ=1. After the training is completed, the unit to be detected is input into the convolutional neural network, so that the classifier can make a reasonable judgment on the video to be detected.

The convolutional layer activation function involved in this embodiment selects the ReLU function.

The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the industry should understand that the present invention is not limited by the above-mentioned examples. What are described in the above-mentioned examples and descriptions are only to illustrate the principles of the present invention. The present invention also has various changes without departing from the spirit and scope of the present invention. These changes and improvements all fall within the scope of the claimed invention. The protection scope of the present invention is defined by the appended claims and their equivalents.

Claims (7)

1. the urban rail transit passenger congestion degree detection method based on convolutional neural network, is characterized in that, comprises the steps: S1, acquiring traffic monitoring video to be detected, preprocessing the video to be detected, segmenting and extracting motion residual images; S2, combine the original image and the motion residual image as the input of the convolutional neural network algorithm, establish a feature extraction block including at least one convolutional layer and a maximum pooling layer, process the original image and the motion residual image, and calculate the original image respectively and the crowd state features contained in the motion residual image; S3. Combining crowd state features with motion features, constructing a feature fusion block including at least one convolutional layer, a maximum pooling layer, and a fully connected layer, and performing fusion processing on the feature map obtained in step S2: S4, constructing a classifier including a degree of congestion level; S5, use the prefabricated training set with the degree of congestion label to train the convolutional neural network, and use the stochastic gradient descent algorithm to correct the parameters in the convolutional neural network, so that the classifier can correctly determine the degree of passenger congestion in the video to be tested. Detection, the stochastic gradient descent method formula is as follows: g(θ)=∑θx ⁱ Among them, g(θ) represents the network hypothesis function; θ represents the parameter weight of the convolutional neural network; h(θ) represents the loss function; x ⁱ represents the network input value of the i-th sample; y ⁱ represents the value of the i-th sample Sample value; m represents the total number of algorithm iterations; σ represents the penalty coefficient; Represents the gradient; η represents the learning rate in gradient descent. 2. the urban rail transit passenger congestion detection method based on convolutional neural network as claimed in claim 1, is characterized in that described step S1 further comprises, S11, acquiring traffic monitoring video to be detected; S12, setting the detection cycle T, dividing the video to be detected into video segments with a length T according to the detection cycle T, and the first frame image of the video segment is a reference image; S13. Taking other images in the video segment and making differences with the reference image respectively to obtain a motion residual image. 3. as claimed in claim 1 or 2, is characterized in that the quantity of each feature extraction block is 1 in the described step S2, and described convolution layer and and The connection mode of the pooling layer is alternate connection. 4. the urban rail transit passenger congestion degree detection method based on convolutional neural network as claimed in claim 3, it is characterized in that in the described step S3, the feature figure that feature fusion block outputs with feature extraction block is input, and described feature fusion The number of blocks is 1. 5. the urban rail transit passenger congestion degree detection method based on convolutional neural network as claimed in claim 4, is characterized in that in described step S3, fully connected layer quantity is 3, and is last three layers, described convolutional layer The connection method with the pooling layer is alternate connection, and they are all located in front of the fully connected layer. 6. the urban rail transit passenger crowding degree detection method based on convolutional neural network as claimed in claim 3, is characterized in that classifier grade is divided into ten layers in the described step S4, is respectively spacious: 0-2 class; Comfortable : Level 3-5; Crowded: Level 5-8; Dangerous: Level 9-10. 7. the urban rail transit passenger congestion degree detection method based on convolutional neural network according to claim 6, is characterized in that in the described step S5, when the result of detection is lower than actual degree of congestion, penalty coefficient σ=1+ log(y ⁱ −g _θ ( ^xi )), otherwise σ=1.