CN112396587B - Method for detecting congestion degree in bus compartment based on collaborative training and density map - Google Patents
Method for detecting congestion degree in bus compartment based on collaborative training and density map Download PDFInfo
- Publication number
- CN112396587B CN112396587B CN202011315096.2A CN202011315096A CN112396587B CN 112396587 B CN112396587 B CN 112396587B CN 202011315096 A CN202011315096 A CN 202011315096A CN 112396587 B CN112396587 B CN 112396587B
- Authority
- CN
- China
- Prior art keywords
- training
- congestion
- crowded
- images
- bus compartment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 81
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 claims abstract description 38
- 238000012795 verification Methods 0.000 claims description 16
- 238000002372 labelling Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 3
- 230000008014 freezing Effects 0.000 claims description 3
- 238000007710 freezing Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000010257 thawing Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 6
- 238000013528 artificial neural network Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 241000282887 Suidae Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30268—Vehicle interior
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for detecting the degree of congestion in a bus compartment based on collaborative training and a density map, which comprises the following steps: acquiring a congestion image in a bus compartment to be detected; inputting the congestion images in the buses to be detected into a trained bus congestion detection network, wherein the bus congestion detection network respectively trains the congestion images in the buses with congestion classification labels and the congestion images in the buses without the congestion classification labels; and obtaining a detection result. In the process of training the neural network, different strategies are adopted to distinguish the data with the classification labels and the data without the classification labels, the density map of the data without the classification labels is marked, the training set with the labels and the density map are respectively utilized to train the neural network, the problem that part of manual work cannot accurately classify the crowded degree image data in the bus compartment is solved, and the classification accuracy is improved.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a method for detecting the degree of congestion in a bus compartment based on collaborative training and a density map.
Background
The intelligent video monitoring technology is that a computer is enabled to be like the brain of a person, a camera is enabled to be like the eyes of the person, and the computer and the camera can be matched to intelligently analyze images acquired from the camera so as to understand the content in a monitored scene. The crowding degree in the bus is a research hotspot for intelligent video monitoring, and the intelligent management energy for realizing urban public transportation is realized by extracting target information from vehicle-mounted video data based on a deep learning technology.
The detection of the crowding degree in the bus compartment belongs to the field of crowd density estimation, and the problem needs to be solved by extracting features from given images, predicting the crowd density in the images and judging the crowding degree of image scenes. Crowd density estimation methods are classified into conventional video and image density estimation algorithms and density estimation algorithms based on deep learning.
Traditional crowd density estimation algorithms fall into two categories, detection-based and regression-based methods. The detection-based method is to detect the population in a scene through a sliding window and count the population, and is mainly divided into detection based on the whole body of the person and detection based on a certain part of the body. The detection is carried out through the characteristics of wavelets, HOGs, edges and the like extracted from the whole body of a person based on the whole body detection, the classifier mainly comprises SVM, boosting, random forests and the like, the method is suitable for sparse people, as the crowd density rises, the shielding between people is more and more serious, and the method is not suitable for people. Therefore, in order to solve the problem of shielding between people, detection is performed on the basis of a certain part of the body, such as the head of a person, and compared with full-body detection, the effect of the method is slightly improved. No matter what kind of detection method is used, the problem of occlusion among people is difficult to solve, so that the regression-based method is gradually used for solving the problem of crowd density estimation, and the regression idea is mainly to learn a mapping relation of features to the number of people, and learn regression models, such as linear regression, ridge regression and Gaussian process regression, by extracting shallow features of scenes. Although the regression-based method solves the shielding problem to a certain extent, the regression is performed by using the characteristics of the whole image, the spatial information in the image is ignored, and then the crowd density estimation method based on the density map is provided, and the spatial information of the image is added in the density estimation and counting process by learning the mapping between the local characteristics of the image and the corresponding density map.
Compared with the traditional method, the deep learning method can more conveniently and efficiently extract high-level features, and the model obtained by training large-scale image data and fitting a large number of parameters in a network through constructing a multi-layer convolutional neural network model has very strong characterization capability. The shallow layer network of the convolutional neural network can extract shallow layer characteristics such as textures, edges and the like in the image; the deep network can extract high-level features with semantic information in the image, and the corresponding density map or crowd count of the deep network can be obtained from the original image by utilizing the learning capability of deep learning.
Therefore, how to accurately classify images difficult to classify the degree of congestion becomes a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the problems actually solved by the present invention include: accurately classifying images difficult to classify the crowdedness level.
In order to solve the technical problems, the invention adopts the following technical scheme:
a method for detecting the degree of congestion in a bus compartment based on collaborative training and a density map comprises the following steps:
acquiring a congestion image in a bus compartment to be detected;
inputting the congestion images in the buses to be detected into a trained bus congestion detection network, wherein the bus congestion detection network respectively trains the congestion images in the buses with congestion classification labels and the congestion images in the buses without the congestion classification labels;
and obtaining a detection result.
Preferably, the method for training the congestion degree detection network in the bus compartment comprises the following steps:
acquiring a crowded image set in a bus compartment, classifying and labeling the crowded image in the bus compartment meeting preset conditions based on the crowded degree to obtain a crowded degree classification tag data set, and classifying the crowded image in the bus compartment and the crowded degree classification tag data set meeting the preset conditions to obtain a training set, a verification set and a test set; carrying out head marking on crowded images in the buses which do not meet preset conditions to obtain a density icon marking data set;
constructing a congestion degree detection network in a bus compartment, wherein the congestion degree detection network in the bus compartment comprises a residual network branch and a cavity convolution branch;
pre-training the residual network branch by using a first training set, a first verification set and a first test set, fine-adjusting network weights, extracting the characteristics of the crowding degree classification label data set, and training a softmax classifier of the residual network branch to classify the crowding degree grade;
taking the image which does not meet the preset condition as the input of a congestion degree detection network in the bus compartment, taking a corresponding density map label as the output of a cavity convolution branch, and cooperatively training the cavity convolution branch based on a residual network branch;
after the cavity convolution branch training is completed, inputting the crowded images in the buses, which do not meet the preset conditions, into a crowded detection network in the buses to obtain corresponding predicted crowded classification label numbers, and adding the credible crowded classification label numbers in the predicted crowded classification label numbers and the corresponding crowded images in the buses into a training set;
training is continued until all crowded images in the buses which do not meet the preset conditions are added into the training set.
Preferably, the method for labeling the head of a person to obtain the density map label comprises the following steps:
labeling the heads of crowded pictures in the buses;
generating a single-channel picture with the same size as a congestion picture in a bus compartment, wherein in the single-channel picture, the pixel value of a point marked by a head is 1, and the values of the rest pixel points are all 0;
and processing the single-channel picture through Gaussian filtering to generate a corresponding density map label.
Preferably, the residual network branch comprises conv1, conv2, conv3, conv4, conv5, conv1 x 1, softmax classifiers connected in sequence; pre-training the residual network branches, fine-tuning the network weights, extracting features of the crowdedness classification tag dataset, and training the softmax classifier of the residual network branches to classify crowdedness levels comprises:
initializing weights of residual network branches;
freezing the weights of conv1, conv2, conv3 and conv4 of the residual network branch, not carrying out back propagation to update the weights, only updating the weights of the conv5 and softmax classifier, thawing the weight of the previous convolution layer when the accuracy rate of the verification set is not increased any more, repeating training until the accuracy rates corresponding to the first training set and the first verification set are not changed any more, and ending training; during training, softmax cross entropy is selected as a loss function, the tensor of the convolution layer Conv5_3 is subjected to softmax operation, the obtained vector y' is subjected to cross entropy calculation with the independent heat coding value y of the classification label of the corresponding sample, and a loss value H is obtained y'(y) The formula is as follows:
H y'(y) ＝∑ i y′ i log(y i )
wherein y 'is' i Representing the dimension, y 'of the vector' i ＝[y′ 1 ，y' 2 ，y' 3 ，…，y'k]，y′ 1 ，y' 2 ，y' 3 …, y' k represents the probability that the sample belongs to various kinds of congestion degree grades, and k is the number of congestion degree grade categories.
Preferably, the hole convolution branch comprises a hole convolution connected with conv5, and when the hole convolution branch is trained cooperatively based on a residual network branch, the loss function is as follows:
where Θ represents the weight of the hole convolution, N represents the number of samples during one training, Z (X) i The method comprises the steps of carrying out a first treatment on the surface of the Θ) represents the predicted density map label for the i-th sample,and (5) representing a density map label corresponding to the ith sample.
Compared with the prior art, the invention has the beneficial effects that:
(1) When the data in the bus compartment is marked, different strategies are adopted to distinguish the data with the classification label from the data without the classification label, the density map of the data without the classification label is marked, and the training set with the label and the density map are respectively utilized for training the neural network, so that the problem that the crowded degree image data in the bus compartment can not be accurately classified by part of manpower is solved, and the classification accuracy is improved.
(2) According to the method, the data with the classification labels and the data without the classification labels are trained respectively, dense graph labeling is not needed for the whole data set, and the labeling cost is greatly saved.
Drawings
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings, in which:
FIG. 1 is a flow chart of one embodiment of a method for detecting congestion in a bus compartment based on co-training and density maps in accordance with the present disclosure;
fig. 2 is a schematic structural diagram of an embodiment of a congestion degree detection network in a bus compartment according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a flowchart of an embodiment of a method for detecting congestion in a bus compartment based on collaborative training and a density map, where the embodiment includes:
acquiring a congestion image in a bus compartment to be detected;
inputting the congestion images in the buses to be detected into a trained bus congestion detection network, wherein the bus congestion detection network respectively trains the congestion images in the buses with congestion classification labels and the congestion images in the buses without the congestion classification labels;
and obtaining a detection result.
In specific implementation, the method for training the congestion degree detection network in the bus compartment comprises the following steps:
(1) Acquiring a crowded image set in a bus compartment, classifying and labeling the crowded image in the bus compartment which meets preset conditions (for images which can be accurately classified into various crowded levels, the images are considered to meet the preset conditions, only classification labels are needed for the images which cannot be accurately classified, the images are considered to be not met with the preset conditions, the images are marked by a head, the network is enabled to judge which category should belong to) based on the crowded level, classification label data sets are obtained by classifying the crowded images in the bus compartment which meets the preset conditions, the crowded images and the crowded level classification label data sets in the bus compartment are classified to obtain training sets, verification sets and test sets (the training sets are used for debugging a neural network (training stage), the verification sets are used for checking training effects (if the effect is not good, the training process is needed), the test sets are used for testing the actual learning ability of the network, for the verification sets are shown in the training process, for example, whether a model is over-trained by checking the change relation of the loss values of the training sets and the verification sets along with the epoch can be seen, if the training is stopped in time, then the model structure and super-parameters are adjusted according to the conditions, the time is greatly saved, the effect is shown in the final performance of the test is shown, the performance of the test is selected as the relevant characteristic, but the training algorithm is not selected, and the relevant characteristic is not selected; carrying out head marking on crowded images in the buses which do not meet preset conditions to obtain a density icon marking data set;
in the present invention, a plurality of levels may be classified according to the degree of congestion. For example, a classification into open, comfortable, relatively crowded and severely crowded, the criteria for each class are as follows:
open space: more than 50% of seats in the vehicle cabin have no passenger seats;
comfort: the empty seats in the carriage are lower than 50 percent, but the empty seats still exist;
comparing congestion: the seats in the carriage are full, passengers in the carriage walkway occupy less than or equal to 50 percent of the total quantity of the passengers accommodated in the walkway, and the passengers can conveniently move in the carriage;
severe congestion: the seats in the carriage are full, passengers are densely distributed on the walkway, and passengers are gathered at the front and rear doors of the vehicle.
The density map refers to the sparse matrix of head annotations in the image that the dataset needs to provide for a given image, which is transformed into a 2D density map by a gaussian filter. The sum of all cells in the density map is the actual number of people in the image. In actual classification, more image scenes are in the four states, the boundary is fuzzy, and manual classification is subjective, so that the image scenes which can be accurately classified are classified and marked, and the image scenes which cannot be accurately classified are marked by people's head, so that the density map label is generated.
The method for obtaining the density map label by labeling the head of a person comprises the following steps:
labeling the heads of crowded pictures in the buses;
generating a single-channel picture with the same size as a congestion picture in a bus compartment, wherein in the single-channel picture, the pixel value of a point marked by a head is 1, and the values of the rest pixel points are all 0;
and processing the single-channel picture through Gaussian filtering to generate a corresponding density map label.
The marking point in the image marked with the head is x i Then the point is denoted as delta (x-x i )，δ(x-x i ) As an impulse function, a label image with N heads is therefore expressed as:
i.e. a density map with N head images, marked with x head positions in the image i Where the pixel value of the point is 1 and the rest is 0.
The density map thus constructed is assumed to exist independently of the image plane, but the density map generated is very sparse, resulting in an overall output approaching 0 at the time of grid calculation loss, and is unfavorable for scenes where the statistical population density is large, so that it is necessary to convolve the above equation with a gaussian function, and the position marked as the head becomes the density function of the region. Thus, the sparse problem is solved to a certain extent, and the counting mode of the number of people is not changed.
Gaussian kernel selection is very important, in real scenes, especially when crowd density is high, each x i Since the picture is not independent, the pixel and the surrounding samples are inconsistent in scale in different scene areas due to perspective distortion, perspective transformation needs to be considered for accurately estimating the population density function, and assuming that the population around each head is uniformly distributed, the average distance of the image between the head and the nearest K neighbors is taken as a reasonable estimate of geometric distortion, namely, the parameter sigma is determined according to the distances of K heads in the image. For each head x i Give K neighbor head distancesCalculate its average distance +.>σ i (x) i Representing the variance of the gaussian kernel, x i Corresponding area on the picture where crowd density and +.>Proportional, so an adaptive gaussian kernel is chosen for convolution, the gaussian kernel is variable, and σ i (x) i And->Proportional:
wherein the method comprises the steps ofBeta=0.3. K takes 5, because of the image scene of the training set, each head region is adjacent to about 5 heads, F (x) is a density function representing the image of the tag marked with the head convolved with the Gaussian kernel, < ->Representing a gaussian kernel.
Since the collected data is evenly distributed over the various time periods, most of the image data is distributed in both open and comfortable states, with relatively little congestion and heavy congestion state data. In order to obtain more data in crowded and severely crowded states, data can be amplified by using data enhancement techniques to balance the amount of various types of data. The final image data set in the bus compartment mainly comprises four types, each type comprises 2 ten thousand pieces of data, and the data is divided into a training set, a verification set and a test set. Features of the image data in the vehicle cabin: the angles of the cameras are inconsistent, so that data enhancement is performed to a certain extent, and the robustness of the model can be enhanced.
(2) Constructing a congestion degree detection network in a bus compartment, wherein the congestion degree detection network in the bus compartment comprises a residual network branch and a cavity convolution branch;
the network for detecting the congestion degree in the bus compartment in the invention is shown in fig. 2.
The invention can build a network model based on a tensorflow deep learning open source framework, and the front end of the network adopts a resnet_v2 network structure as a feature extraction module of the whole model. The position of the activation function is adjusted by the resnet_v2 on the basis of the resnet, and a more stable result is obtained. The resnet has 5 sets of convolutions in total. The first group of convolution inputs has 224 x 224, the fifth group of convolution outputs has 7*7, which is reduced by 32 times, and BN (BacthNormalization) layers are added in the network. In order to balance accuracy and resource expenditure, the built residual error network has 50 layers in total, and a full connection layer is removed, so that the network is a full convolution network, and the size of an input picture can be any size.
The rear end adopts a cavity convolution module, and a density map of a corresponding image is generated while the receptive field is enlarged. A two-dimensional hole convolution is defined as follows:
x (M, N) is an image with length and width of M and N respectively, and the output y (M, N) of the cavity convolution is obtained through a convolution kernel w (i, j), wherein r represents the cavity rate. When r=1, it is a normal convolution. If the hole convolution rate is r, the convolution kernel of K is enlarged to K+ (K-1) (r-1). The cavity convolution utilizes sparse convolution to verify the alternate convolution and pooling operation, increases the receptive field on the premise of not increasing the calculation scale of network parameters, and is more suitable for crowd density estimation. And 6 layers of built cavity convolution networks are used, the cavity rates are the same, and a 1*1 convolution layer is adopted to output a result. All convolution layers are padded to maintain the original size.
To understand the effect of void fraction on the quality of the resulting density map, a comparison experiment was set up, with r= {1,2,4}, respectively, where a further set was r=2, 4 mixed. Experimental results show that r=2 works best.
(3) Pre-training the residual network branch by using a first training set, a first verification set and a first test set, fine-adjusting network weights, extracting the characteristics of the crowding degree classification label data set, and training a softmax classifier of the residual network branch to classify the crowding degree grade;
the residual network branch comprises conv1, conv2, conv3, conv4, conv5, conv1 x 1 and softmax classifiers which are connected in sequence; pre-training the residual network branches, fine-tuning the network weights, extracting features of the crowdedness classification tag dataset, and training the softmax classifier of the residual network branches to classify crowdedness levels comprises:
initializing weights of residual network branches;
the invention can employ a residual network pre-training weights on the ImageNet dataset as initial weights. The ImageNet data set is a large-scale labeled image data set organized according to the WordNet architecture, about 1500 tens of thousands and 2.2 tens of thousands are mainly used for tasks such as image classification, target identification and the like, and the transfer learning on the data set can not only accelerate the convergence rate of a model, but also improve the robustness of the model.
Freezing the weights of conv1, conv2, conv3 and conv4 of the residual network branch, not carrying out back propagation to update the weights, only updating the weights of the conv5 and softmax classifier, thawing the weight of the previous convolution layer when the accuracy rate of the verification set is not increased any more, repeating training until the accuracy rates corresponding to the first training set and the first verification set are not changed any more, and ending training; the learning rate can be set smaller due to the initial weight of pre-training during training. The batch was set at 32 according to the computational power of the GPU, eventually 10 epochs were trained and the model loss was no longer changing.
During training, softmax cross entropy is selected as a loss function, tensors of a convolution layer conv5_3 (the last convolution of Conv 5) are subjected to softmax operation, and the obtained vector y' and the independent thermal coding value y of the classification label of the corresponding sample are subjected to cross entropy calculation to obtain a loss value H y'(y) The formula is as follows:
H y'(y) ＝∑ i y′ i log(y i )
wherein y 'is' i Representing the dimension, y 'of the vector' i ＝[y′ 1 ，y' 2 ，y' 3 ，…，y' k ]，y 1 '，y' 2 ，y' 3 ，…，y' k And respectively representing the probability that the samples belong to various crowding degree grades, wherein k is the crowding degree grade class number.
(4) Taking the image which does not meet the preset condition as the input of a congestion degree detection network in the bus compartment, taking a corresponding density map label as the output of a cavity convolution branch, and cooperatively training the cavity convolution branch based on a residual network branch;
the cavity convolution branch comprises a cavity convolution connected with conv5, and when the cavity convolution branch is trained cooperatively based on a residual network branch, the loss function is as follows:
where Θ represents the weight of the hole convolution, N represents the number of samples during one training, Z (X) i The method comprises the steps of carrying out a first treatment on the surface of the Θ) represents the predicted density map label for the i-th sample,representing a density map label corresponding to the ith sample;
the hole convolution branches share the characteristics of residual branches, namely the weights of the residual branches are frozen at the moment and are not updated in the back propagation process, only the weights of the hole convolution branches are updated, and the trained hole convolution branches can predict the density map characteristics corresponding to the input image.
(5) After the cavity convolution branch training is completed, inputting the crowded images in the buses, which do not meet the preset conditions, into a crowded detection network in the buses to obtain corresponding predicted crowded classification label numbers, and adding the credible crowded classification label numbers in the predicted crowded classification label numbers and the corresponding crowded images in the buses into a training set;
(6) Training is continued (training of residual network branches and cavity convolution branches is performed again) until all crowded images in the bus which do not meet preset conditions are added into the training set.
Finally, it is noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. The method for detecting the degree of congestion in the bus compartment based on the collaborative training and the density map is characterized by comprising the following steps of:
acquiring a congestion image in a bus compartment to be detected;
inputting the congestion images in the buses to be detected into a trained bus congestion detection network, wherein the bus congestion detection network respectively trains the congestion images in the buses with congestion classification labels and the congestion images in the buses without the congestion classification labels;
obtaining a detection result;
the method for training the congestion degree detection network in the bus compartment comprises the following steps:
acquiring a crowded image set in a bus compartment, classifying and labeling the crowded image in the bus compartment meeting preset conditions based on the crowded degree to obtain a crowded degree classification tag data set, and classifying the crowded image in the bus compartment and the crowded degree classification tag data set meeting the preset conditions to obtain a training set, a verification set and a test set; carrying out head marking on crowded images in the buses which do not meet preset conditions to obtain a density icon marking data set; the method comprises the steps that for images which can be accurately divided into congestion levels, preset conditions are considered to be met; regarding the images which cannot be accurately divided, considering that the images do not meet preset conditions;
constructing a congestion degree detection network in a bus compartment, wherein the congestion degree detection network in the bus compartment comprises a residual network branch and a cavity convolution branch;
pre-training the residual network branch by using a first training set, a first verification set and a first test set, fine-adjusting network weights, extracting the characteristics of the crowding degree classification label data set, and training a softmax classifier of the residual network branch to classify the crowding degree grade;
taking the image which does not meet the preset condition as the input of a congestion degree detection network in the bus compartment, taking a corresponding density map label as the output of a cavity convolution branch, and cooperatively training the cavity convolution branch based on a residual network branch;
after the cavity convolution branch training is completed, inputting the crowded images in the buses, which do not meet the preset conditions, into a crowded detection network in the buses to obtain corresponding predicted crowded classification label numbers, and adding the credible crowded classification label numbers in the predicted crowded classification label numbers and the corresponding crowded images in the buses into a training set;
training is continued until all crowded images in the buses which do not meet the preset conditions are added into the training set.
2. The method for detecting the degree of congestion in the bus compartment based on cooperative training and density map as set forth in claim 1, wherein the method for labeling the head of a person to obtain the density map label comprises the steps of:
labeling the heads of crowded pictures in the buses;
generating a single-channel picture with the same size as a congestion picture in a bus compartment, wherein in the single-channel picture, the pixel value of a point marked by a head is 1, and the values of the rest pixel points are all 0;
and processing the single-channel picture through Gaussian filtering to generate a corresponding density map label.
3. The method for detecting the congestion degree in the bus compartment based on the cooperative training and the density map according to claim 1, wherein the residual network branch comprises conv1, conv2, conv3, conv4, conv5, conv1 x 1 and softmax classifiers which are sequentially connected; pre-training the residual network branches, fine-tuning the network weights, extracting features of the crowdedness classification tag dataset, and training the softmax classifier of the residual network branches to classify crowdedness levels comprises:
initializing weights of residual network branches;
freezing the weights of conv1, conv2, conv3 and conv4 of the residual network branch, not carrying out back propagation to update the weights, only updating the weights of the conv5 and softmax classifier, thawing the weight of the previous convolution layer when the accuracy rate of the verification set is not increased any more, repeating training until the accuracy rates corresponding to the first training set and the first verification set are not changed any more, and ending training; during training, softmax cross entropy is selected as a loss function, the tensor of the convolution layer Conv5_3 is subjected to softmax operation, the obtained vector y' is subjected to cross entropy calculation with the independent heat coding value y of the classification label of the corresponding sample, and a loss value H is obtained y'(y) The formula is as follows:
H y'(y) ＝∑ i y′ i log(y i )
wherein y 'is' i Representing the dimension, y 'of the vector' i ＝[y′ 1 ，y′ 2 ，y′ 3 ，…，y′ k ]，y′ 1 ，y′ 2 ，y′ 3 ，…，y′ k And respectively representing the probability that the samples belong to various crowding degree grades, wherein k is the crowding degree grade class number.
4. A co-training and density map-based method for detecting congestion in a bus compartment as set forth in claim 3, wherein said hole convolution branch comprises a hole convolution connected to conv5, and the loss function when the hole convolution branch is co-trained based on the residual network branch is as follows:
where Θ represents the weight of the hole convolution, N represents the number of samples during one training, Z (X) i The method comprises the steps of carrying out a first treatment on the surface of the Θ) represents the predicted density map label for the i-th sample,and (5) representing a density map label corresponding to the ith sample.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011315096.2A CN112396587B (en) | 2020-11-20 | 2020-11-20 | Method for detecting congestion degree in bus compartment based on collaborative training and density map |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011315096.2A CN112396587B (en) | 2020-11-20 | 2020-11-20 | Method for detecting congestion degree in bus compartment based on collaborative training and density map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112396587A CN112396587A (en) | 2021-02-23 |
| CN112396587B true CN112396587B (en) | 2024-01-30 |
Family
ID=74606007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011315096.2A Active CN112396587B (en) | 2020-11-20 | 2020-11-20 | Method for detecting congestion degree in bus compartment based on collaborative training and density map |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112396587B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052039B (en) * | 2021-03-16 | 2022-12-02 | 北京邮电大学 | Method, system and server for detecting pedestrian density of traffic network |
| CN113255480A (en) * | 2021-05-11 | 2021-08-13 | 中国联合网络通信集团有限公司 | Method, system, computer device and medium for identifying degree of congestion in bus |
| CN113869269B (en) * | 2021-10-13 | 2024-09-27 | 平安银行股份有限公司 | Method and device for detecting crowdedness degree of activity site, electronic equipment and storage medium |
| CN113989229A (en) * | 2021-10-28 | 2022-01-28 | 杭州图谱光电科技有限公司 | Cell counting method based on random forest and convolutional neural network |
| CN116363598A (en) * | 2023-05-29 | 2023-06-30 | 深圳市捷易科技有限公司 | Crowd crowding early warning method and device, electronic equipment and readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2704060A2 (en) * | 2012-09-03 | 2014-03-05 | Vision Semantics Limited | Crowd density estimation |
| CN108648495A (en) * | 2018-06-08 | 2018-10-12 | 华南理工大学 | A kind of method and system of the intelligence real-time display bus degree of crowding |
| CN109389044A (en) * | 2018-09-10 | 2019-02-26 | 中国人民解放军陆军工程大学 | Multi-scene crowd density estimation method based on convolutional network and multi-task learning |
| CN109446920A (en) * | 2018-09-30 | 2019-03-08 | 南京地铁集团有限公司 | Method for detecting passenger crowding degree of urban rail transit based on convolutional neural network |
| CN109858424A (en) * | 2019-01-25 | 2019-06-07 | 佳都新太科技股份有限公司 | Crowd density statistical method, device, electronic equipment and storage medium |
| CN111488794A (en) * | 2020-02-24 | 2020-08-04 | 华中科技大学 | Adaptive receptive wildman population density estimation method based on hole convolution |
| CN111563447A (en) * | 2020-04-30 | 2020-08-21 | 南京邮电大学 | Crowd density analysis and detection positioning method based on density map |
| CN111639668A (en) * | 2020-04-17 | 2020-09-08 | 北京品恩科技股份有限公司 | Crowd density detection method based on deep learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8488863B2 (en) * | 2008-11-06 | 2013-07-16 | Los Alamos National Security, Llc | Combinational pixel-by-pixel and object-level classifying, segmenting, and agglomerating in performing quantitative image analysis that distinguishes between healthy non-cancerous and cancerous cell nuclei and delineates nuclear, cytoplasm, and stromal material objects from stained biological tissue materials |
| US11195057B2 (en) * | 2014-03-18 | 2021-12-07 | Z Advanced Computing, Inc. | System and method for extremely efficient image and pattern recognition and artificial intelligence platform |
-
2020
- 2020-11-20 CN CN202011315096.2A patent/CN112396587B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2704060A2 (en) * | 2012-09-03 | 2014-03-05 | Vision Semantics Limited | Crowd density estimation |
| CN108648495A (en) * | 2018-06-08 | 2018-10-12 | 华南理工大学 | A kind of method and system of the intelligence real-time display bus degree of crowding |
| CN109389044A (en) * | 2018-09-10 | 2019-02-26 | 中国人民解放军陆军工程大学 | Multi-scene crowd density estimation method based on convolutional network and multi-task learning |
| CN109446920A (en) * | 2018-09-30 | 2019-03-08 | 南京地铁集团有限公司 | Method for detecting passenger crowding degree of urban rail transit based on convolutional neural network |
| CN109858424A (en) * | 2019-01-25 | 2019-06-07 | 佳都新太科技股份有限公司 | Crowd density statistical method, device, electronic equipment and storage medium |
| CN111488794A (en) * | 2020-02-24 | 2020-08-04 | 华中科技大学 | Adaptive receptive wildman population density estimation method based on hole convolution |
| CN111639668A (en) * | 2020-04-17 | 2020-09-08 | 北京品恩科技股份有限公司 | Crowd density detection method based on deep learning |
| CN111563447A (en) * | 2020-04-30 | 2020-08-21 | 南京邮电大学 | Crowd density analysis and detection positioning method based on density map |
Non-Patent Citations (6)
| Title |
|---|
| Cyber-physical modeling and control of crowd of pedestrians: a review and new framework;Kecai Cao等;《 IEEE/CAA Journal of Automatica Sinica》;全文 * |
| Real-Time Crowd Density Estimation Based on Convolutional Neural Networks;Baiping Li等;《2018 International Conference on Intelligent Transportation》;全文 * |
| 公交车厢内拥挤度和站台候车人数检测算法研究;张力丹;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;全文 * |
| 城市轨道交通车站客流群集规律及拥挤踩踏风险研究;李小红;《中国博士学位论文全文数据库 工程科技Ⅱ辑》;全文 * |
| 基于多尺度信息与注意力机制的人群密度估计算法研究;王徐庆;《中国优秀硕士学位论文全文数据库 信息科技辑》;全文 * |
| 基于注意力机制的狭小空间人群拥挤度分析;张菁;陈庆奎;;计算机工程(09);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112396587A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112396587B (en) | Method for detecting congestion degree in bus compartment based on collaborative training and density map | |
| CN109344736B (en) | Static image crowd counting method based on joint learning | |
| CN104992223B (en) | Intensive population estimation method based on deep learning | |
| CN108764072B (en) | Blood cell subtype image classification method based on multi-scale fusion | |
| Zheng et al. | Improving the generalization ability of deep neural networks for cross-domain visual recognition | |
| CN105095833A (en) | Network constructing method for human face identification, identification method and system | |
| CN111783589B (en) | Complex scene crowd counting method based on scene classification and multi-scale feature fusion | |
| CN113297936B (en) | Volleyball group behavior identification method based on local graph convolution network | |
| CN103390279A (en) | Target prospect collaborative segmentation method combining significant detection and discriminant study | |
| CN115131618B (en) | Semi-supervised image classification method based on causal reasoning | |
| CN111524140B (en) | Medical image semantic segmentation method based on CNN and random forest method | |
| CN114596726B (en) | Parking berth prediction method based on interpretable space-time attention mechanism | |
| CN114005085A (en) | Dense crowd distribution detection and counting method in video | |
| CN110599459A (en) | Underground pipe network risk assessment cloud system based on deep learning | |
| CN112562312B (en) | GraphSAGE traffic network data prediction method based on fusion features | |
| CN117315381B (en) | Hyperspectral image classification method based on second-order biased random walk | |
| CN113269224A (en) | Scene image classification method, system and storage medium | |
| CN114863348A (en) | Video target segmentation method based on self-supervision | |
| CN115393631A (en) | Hyperspectral image classification method based on Bayesian layer graph convolution neural network | |
| CN117132849A (en) | Cerebral apoplexy hemorrhage transformation prediction method based on CT flat-scan image and graph neural network | |
| CN111611919A (en) | Road scene layout analysis method based on structured learning | |
| CN112418227B (en) | Monitoring video truck segmentation method based on double self-attention mechanism | |
| CN108304546B (en) | Medical image retrieval method based on content similarity and Softmax classifier | |
| CN117745640A (en) | Domain distribution self-adaption-oriented reference-free image quality assessment method and system | |
| Kong et al. | Toward large-scale histopathological image analysis via deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |