CN109165582B - Urban street garbage detection and cleanliness assessment method - Google Patents

Abstract

The invention discloses an urban street garbage detection and cleanliness assessment method based on mobile edge calculation and deep learning. The method mainly collects street view images through a high-resolution camera and a handheld mobile device which are arranged on a garbage collection vehicle on the urban street; temporarily storing by using an edge server and preprocessing street view images; transmitting the data to a cloud center through an urban network, identifying street garbage categories and counting the garbage quantity by utilizing a deep learning fast-Rcnn algorithm, introducing the results into a street cleanliness evaluation framework based on a hierarchy, and finally visualizing the street cleanliness grade; the convenience is provided for the urban municipal administration manager to effectively arrange cleaning personnel.

Description

Urban street garbage detection and cleanliness assessment method

Technical Field

The invention relates to a method for detecting urban street garbage and evaluating cleanliness, in particular to a method for detecting urban street garbage and evaluating cleanliness based on mobile edge calculation and deep learning, and belongs to the technical field of information processing.

Background

As cities become more and more, urbanization progresses faster and faster, and keeping cities clean is an important goal of city development. Currently, street cleaning is becoming a major concern in city cleaning under smart cities. Traditionally, street cleaning requires manual intervention at each level. The city citizens manually check the positions where the garbage exists, and then submit reports to the city manager, and the city manager arranges the municipal staff nearby for cleaning. Some cities install cameras at the crossroads of the street to observe whether garbage exists in the area. These cannot timely grasp the garbage cleaning condition of the city streets. For this reason, researchers at home and abroad are studying an automated system. Begur H et al regularly captures streets using a cleaning vehicle with a camera, collects street information such as pictures, geographical location, date and time, then performs image detection in a cloud platform using machine learning techniques, and the resulting results are sent to a city management system for viewing by a city manager. Akshay Dabholkar et al used a deep learning approach to identify various types of dumped garbage, while suggesting the use of edge calculations to reduce unnecessary image transmission into the server. The image is sent to the server only when the image contains frequently dumped trash.

With the coming age of 5G, it is clear that large centralized computing with cloud computing as a core cannot efficiently process a large amount of data generated by edge devices. Therefore, researchers use the micro data center of edge computing as a bridge of mobile equipment and a cloud server, a small part of data is put into the edge of a network in advance for processing, the cloud center receives all computing services, and the mobile edge computing can be widely applied to telecommunication, industrial production, education, electronic commerce, mobile medical treatment, car networking and the like in the future.

Disclosure of Invention

The purpose of the invention is as follows: aiming at the problems in the prior art, the invention provides a city street garbage detection and cleanliness assessment method, which is characterized in that a high-resolution camera and a handheld mobile device which are arranged on a city street garbage collection vehicle are used for collecting street view images; temporarily storing by using an edge server and preprocessing street view images; transmitting the data to a cloud center through an urban network, identifying street garbage categories and counting the garbage quantity by using an Faster-Rcnn algorithm, introducing the results into a street cleanliness evaluation framework based on a hierarchy, and finally visualizing the street cleanliness grade; the convenience is provided for the urban municipal administration manager to effectively arrange cleaning personnel.

The technical scheme is as follows: a city street garbage detection and cleanliness assessment method based on mobile edge calculation and deep learning comprises the following steps:

step 1: carrying out street view image data collection by a garbage collection vehicle and mobile equipment held by urban residents;

step 2: the edge server preprocesses the street view image;

step 3, the cloud server performs garbage detection model training;

and 4, step 4: the cloud server performs garbage detection on the street view images and performs garbage counting;

and 5: calculating the street cleanliness based on the garbage recognition result;

step 6: and displaying each level of cleanliness maps of the city based on the level evaluation model.

The data collection in the step 1 mainly comprises three aspects, namely, (1) garbage image data used for training a garbage detection model needs to be shot from streets with various sizes, and a data set library for urban garbage detection is manufactured and contains main garbage types appearing on the streets. (2) The street image data used for garbage detection is shot at fixed points on each street line by a cleaning vehicle provided with a high-resolution camera according to the assignment of an administrator, the distance between adjacent shooting points is set by the administrator, each shooting point carries out omnidirectional shooting in four directions, namely front, back, left and right, and the general shooting range is 150 plus 300 square meters. For the mobile station, the following provisions are set: 1) a fixed image resolution; 2) a fixed vehicle speed; 3) a fixed distance of shot points; 4) each shot took 4 pictures. (3) Local management data for scheduling. The cleaning vehicle needs to report the position information of the cleaning vehicle to a city manager at regular time, and the manager responds timely and arranges cleaning personnel to clean the cleaning vehicle through local management.

The street view image data preprocessing process mainly carries out picture screening through an edge server, eliminates some unnecessary information to reduce the processing time delay of the whole system, and the step 2 comprises the following steps:

step 21: the edge server cuts the collected street view image data, and the picture size is 420 x 400 pixels;

step 22: and then, carrying out artificial road detection and screening, if the picture is detected to contain a road feasible region, namely effective data, transmitting the effective data to a cloud center for road garbage detection, and if the picture is detected to have no road feasible region, deleting the effective data.

In the step 3, in the process of training a garbage detection model by the cloud server, extracting a candidate frame and identifying a candidate frame target are realized through two modules, namely an RPN module and a Fast R-CNN module, a Fast-RCNN algorithm is a method for detecting a target in the computer vision field, and the step 3 comprises the following steps:

step 31: and (4) preparing for obtaining a characteristic diagram based on the design of the ZF-Net network. The CNN network selected is a ZF-Net network, 3-channel RGB garbage images with 224 x 224 of input layers are input, the first layer comprises 96 convolution kernels used for extracting garbage image space structures or characteristics, in order to avoid that the first layer convolution kernels are mixed with high-frequency and low-frequency information and lack intermediate-frequency information, the size of the convolution kernels of the first layer is set to be 7 x 7. Performing maximum pooling operation, setting convolution span as 2, comparing normalization operation to generate 96 feature templates with the size of 55 × 55, performing similar operation on layers 2, 3, 4 and 5, outputting 256 feature maps with the size of 6 × 6 on the 5 th layer, fully connecting the 6 th layer and the 7 th layer, inputting the sampling result of the 5 th layer into a classifier and a bounding box regression device, giving out the category of a candidate area by the classifier, and giving out an area suggestion frame of the candidate area by the bounding box regression device;

step 32: and (3) pre-training the RPN, carrying out supervised training (training by adopting an end-to-end back propagation algorithm and a random gradient descent method) on the RPN by the ImageNet network, initializing initial parameters of a training model by using the trained weight parameters, and initializing other newly added layers by using Gaussian distribution with the standard deviation of 0.01 and the mean value of 0. Then fine-tuning the tasks for regional suggestions end-to-end; since the resulting candidate regions are not the same as the previously labeled regions, a mapping relationship is found and fine tuned using linear regression modeling, i.e., bounding box regression fine tuning.

Step 33: performing Fast R-CNN network pre-training, performing end-to-end fine tuning training of the Fast R-CNN network by using the region suggestion frame obtained in the step 31 and performing network parameter initialization by using the weight pre-trained by the ImageNet model;

step 34: reinitializing the RPN network by using the Fast R-CNN network after the fine tuning in the step 33, fixing the shared convolution layer, namely setting the learning rate to be 0, not updating, and only finely tuning the layer unique to the RPN network;

step 35: the pinning step 34 shares the convolutional layer while only the fully-connected layer of Fast R-CNN is fine-tuned using the region proposal obtained in step 34.

In order to perform garbage detection by using the trained model, the step 4 further includes:

step 41: inputting a street test image;

step 42: reflecting image features such as texture, shape and color chroma of garbage to a feature map through calculation based on the ZF convolutional neural network;

step 43: each RPN candidate area network correspondingly calculates a candidate area to generate a candidate suggestion frame;

step 44: the candidate suggestion frame displays a garbage candidate region frame and the classification score of the region through a full connection layer, namely a classification layer and a regression layer;

step 45: counting the generated candidate region frames by using a counting function, namely, the number of detected garbage, wherein the number function is designed to be

Wherein C is a counting function for generating the candidate frame, namely the number of the detected garbage of a certain category, f is a result function detected by the garbage model, D is a test sample set, x is a test sample, y is a real garbage mark, and i is the number of input test samples;

aiming at the problem of calculating the cleanliness of the street based on the garbage recognition result, the step 5 is further as follows:

step 51: a minimum number n of street samples is first determined,

where N is the minimum number of samples, k is the sampling interval, and in order to ensure that a 95% confidence interval is achieved, k is 1.96, p represents the probability of occurrence of a sample, q represents the probability of non-occurrence of a sample, and is denoted as q 1-p, where p is 0.5, N is the total number of city streets, e is the estimation error, and e is 0.1.

Step 52: a street cleanliness index value SV is calculated,

wherein S is an observation area, and the method S is the range of the visual angle shot by the top camera of the garbage collection vehicle, and is set to be 150 square meters. λ and a are correction factors that affect the cleanliness of city streets because there are many factors that affect the cleanliness index, such as how good or bad the weather is, the type of street surface, etc. C is the street refuse weight a is usually between 1 and 2, the method a takes 1, lambda takes 1

Step 53: and classifying the cleanliness of the streets according to the street cleanliness index value.

In order to scientifically evaluate the cleanliness of each level of the city based on a level evaluation model, the step 6 is further as follows:

step 61: and establishing a hierarchical cleanliness model.

Step 62: based on the level evaluation model, firstly, performing street level evaluation, namely street cleanliness calculation, and the specific calculation process is shown in step 52;

and step 63: calculating the block level model cleanliness BV,

where BV (Block value) is the evaluation value of a block in the region, SV represents the evaluation value of each street, m₁Represents the total number of streets within a block;

step (ii) of64: calculating the cleanliness BV of the regional level model,

wherein AV (area value) represents an evaluation value of an area, BV represents an evaluation value of each block in the area, m₂Representing the total number of blocks in the region;

step 65: calculating the cleanliness CV of the urban hierarchical model,

where CV (City value) represents the evaluation value of a city, AV is the evaluation value of each area in the city, and m₃Represents the total number of regions in the city;

and step 66: and displaying the cleanliness maps of all levels of the city based on the cleanliness calculation results of all levels.

Drawings

FIG. 1 is a diagram of a process framework of the present invention;

FIG. 2 is a diagram of a hierarchical evaluation model according to the present invention;

FIG. 3 is a diagram of a data acquisition and processing process based on moving edge calculation;

FIG. 4 is a flow chart of the method of the present invention;

FIG. 5 shows the result of the detection accuracy of various types of garbage;

FIG. 6 is an image diagram of the garbage detection result;

FIG. 7 shows the classification and detection quantity of the refuse in the West street of Foucault;

FIG. 8 is a demonstration of the assessment of the cleanliness of the streets in Jiangning district;

fig. 9 shows the evaluation of hierarchical cleanliness of jiangning blocks.

Detailed Description

The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention, as various equivalent modifications of the invention will occur to those skilled in the art upon reading the present disclosure and fall within the scope of the appended claims.

As shown in fig. 1, a city street garbage detection and cleanliness assessment method based on mobile edge calculation and deep learning includes the following steps:

step 1: carrying out street view image data collection by a garbage collection vehicle and mobile equipment held by urban residents;

step 2: the edge server preprocesses the street view image;

step 3, the cloud server performs garbage detection model training;

and 4, step 4: the cloud server performs garbage detection on the street view images and performs garbage counting;

and 5: calculating the street cleanliness based on the garbage recognition result;

step 6: and displaying each level of cleanliness maps of the city based on the level evaluation model.

As shown in FIG. 2, a hierarchical assessment model diagram is used with the present invention that provides a multi-level assessment model across different levels. The lower layer provides information to the upper layer, which summarizes the processing results of the lower layer. This model can be divided into four layers, the first layer defining the whole city and setting the evaluation range, covering all streets of the city, being the base layer. The second layer of administrative regions is used for dividing the city into a plurality of regional layers, and each region is an administrative region. And the third layer is divided into a plurality of blocks according to the secondary administrative area in the area, and each block is combined into a unique identifier by the administrative area and the block name. The fourth layer is the topmost layer and represents each street on each block, and data collection points are arranged on the streets so as to be convenient for scientifically collecting data and making data sets;

as shown in fig. 3, a data acquisition and processing process diagram based on moving edge calculation proposed by the present invention includes three parts:

a mobile station: a specific garbage collection vehicle is arranged on each urban street, a camera with high resolution, high pixel and network transmission function is installed at the top of the garbage collection vehicle, the camera faces the ground and covers the range of 50 meters in front, the garbage collection vehicle shuttles back and forth on the urban streets every day, regular street garbage photographing is carried out according to a specific line, and the garbage photographing is transmitted to an edge server in real time. And meanwhile, the city citizen can also take the role of a garbage collection vehicle, collect street garbage data by using own mobile equipment and transmit the street garbage data to the edge server.

An edge server: the edge servers are arranged at the extreme edge of the network, are directly connected with nearby mobile equipment through a wireless data link, process part of service requests of the mobile equipment, and have the function of temporarily storing data from the mobile equipment.

Cloud: the layer is used for creating a training model, executing a street garbage detection task, presenting a city street cleanliness grade in real time and feeding relevant information back to a city manager.

As shown in fig. 4, a city street garbage detection and cleanliness assessment method based on moving edge calculation and deep learning includes the following steps:

step 101: a cleaning vehicle provided with a high-resolution camera collects street view image information and local management information;

the street view image information is transmitted into the edge server, the local management information is provided for an administrator, namely a cleaning vehicle, to report the position of the cleaning vehicle to a city administrator at regular time, and the administrator responds in time;

step 102: the edge server temporarily stores data from the cleaning vehicle;

step 103: for training and detecting the model, the edge server cuts the collected street view image data, and the picture size is 420 x 400 pixels;

step 104: detecting and screening artificial roads;

step 105: judging whether the image contains a road or not; if the picture is detected to contain the feasible road area, namely effective data, turning to step 107, and if the picture is detected to have no feasible road area, executing step 106;

step 106: deleting the image without the road;

step 107: performing garbage detection model training, and using a Faster-RCNN algorithm in the target detection field as a core algorithm of the garbage detection model training, wherein the algorithm training step is as follows: (1) the method comprises the steps of firstly pre-training an RPN network, carrying out supervised training on the RPN network by an ImageNet network, initializing the RPN network into initial parameters of a model by using the trained network parameters, and initializing other newly-added layers by using Gaussian distribution with standard deviation 0.01 and mean value 0. The tasks for regional proposal are then fine-tuned end-to-end. (2) Performing Fast R-CNN network pre-training, performing end-to-end fine tuning training of the Fast R-CNN network for detection by using the area suggestion box obtained in the step (1), and initializing network parameters by using an ImageNet model. (3) And (3) reinitializing the RPN network by using the Fast R-CNN network after the fine adjustment in the step (2), and fixing the shared convolution layer, namely setting the learning rate to be 0, not updating, and only fine adjusting the unique layer of the RPN network. (4) Fixing the shared convolution layer in the step (3), and only finely adjusting the full connection layer of Fast R-CNN by using the region suggestion obtained in the step (3);

step 108: detecting street view images collected by the cleaning vehicle, if detecting street garbage, executing step 110, otherwise executing step 109;

step 109: and if the picture is not garbage, discarding the picture.

Step 110: the classifier detects a garbage target to count, and specifically comprises the following steps: and (3) automatically counting the bounding boxes once every time a candidate area box is generated, namely sequentially increasing the counting function value by 1, and finally counting the detected category and the detected number of the candidate area boxes.

Wherein C is a counting function for generating the candidate box, that is, the number of the detected garbage of a certain category, f is a result function detected by the garbage model, D is a test sample set, x is a test sample, y is a real garbage mark, and i is the number of input test samples.

Step 111: after the street garbage is identified, calculating the cleanliness;

step 112: first, a minimum number n of street samples is determined, here by the formula

To represent; where N is the minimum number of samples, k is the sampling interval, k is 1.96, p represents the possibility of occurrence of sampling, q represents the possibility of non-occurrence of sampling, q is 1-p, in general, p is 0.5, N is the total number of city streets, e is the estimation error, and e is 0.1;

step 113: a street cleanliness index value SV is calculated,

wherein S is an observation area, and the method S is the range of the visual angle shot by the top camera of the garbage collection vehicle, and is set to be 150 square meters. λ and n are correction factors that affect the cleanliness of city streets because there are many factors that affect the cleanliness index, such as weather quality, street pavement type, etc., as shown in table 3. C is the street garbage weighted number, see Table 2. n represents the quantity change of the garbage under special conditions, and is usually between 1 and 2, and n is 1 in the method;

step 114: the streets are sorted for cleanliness based on the street cleanliness index values, see table 4.

In order to scientifically evaluate the cleanliness of each level of the city based on a level evaluation model, the method further comprises the following steps:

step 115: executing step 113 and step 114;

step 116: calculating the block level model cleanliness BV,

where BV (Block value) is the evaluation value of a block in the region, SV represents the evaluation value of each street, m₁Represents the total number of streets within a block;

step 117: calculating the cleanliness BV of the regional level model,

wherein AV (area value) represents an evaluation value of an area, BV represents an evaluation value of each block in the area, m₂Representing the total number of blocks in the region;

step 118: calculating the cleanliness CV of the urban hierarchical model,

where CV (City value) represents the evaluation value of a city, AV is the evaluation value of each area in the city, and m₃Represents the total number of regions in the city;

step 119: and presenting the cleanliness calculation results of all layers, and ending.

In order to verify the practical effect of the invention, the streets in Jiangning district of Nanjing city are selected as research objects, and for training data sets, people take artificial garbage photographs, and the common street garbage is divided into 9 types including waste paper, plastic bags, plastic bottles, fruit peels, cigarette ends, waste cloth, cigarette cases, leaves and pop cans, and 681 pieces of picture data are collected, wherein the size of the picture is 420 x 400 pixels. From these, 321 pieces were selected as training sets, 260 pieces were selected as test sets, and 100 pieces were selected as verification sets. The training results are shown in fig. 5:

for the acquisition of street data sets, the cleaning vehicle acts as a camera node every 50 meters. Shooting is carried out on one shooting node according to four directions, and the shooting visual angle is about 150 square meters. It is specified that each street is sampled by 1km, that is, each street vehicle takes 80 street view images. The Jiangning area has about 3875 streets, therefore, according to the minimum street sampling rule, the number of the streets researched by the invention is 100, about 8000 images are collected, and the collected street scenes are set at 11 to 16 points per day.

We feed the road images collected on each street into the fast-RCNN master classifier and detect the garbage images on the streets through the trained model, as shown in fig. 6. Each detected garbage is marked with a rectangular box and the garbage type and the similarity are displayed, each time a rectangular box is generated, the rectangular box is automatically counted, and finally the number of the rectangular boxes is the number of the garbage detected by the classifier. FIG. 7 shows the results of garbage detection for 1km on the Western street of Focheng in Jianning area of Nanjing.

Table 1 shows the weighted amount of trash in the west city of fond in the jiangning area, and classifies waste paper, plastic bags, plastic bottles, cigarette cases, and cans detected by the classifier as inorganic trash categories, and classifies peels, butts, and waste cloths as organic trash categories. Based on the garbage category and its weight in table 2 and the correction factor parameter in table 3, let C be 87, S be 150, λ be 1, and n be 1. Finally, through a cleanliness calculation formula, the index of the cleanliness of the florist city street is 58, and according to the table 4, the cleanliness of the florist city street belongs to the highest level, which means that the street surface is very clean.

FIG. 8 is a graph of the cleanliness of 100 streets in a visualized Jiangning region, wherein the line A represents SV less than 70, indicating that the street cleanliness is very high in grade; line B represents SV between 70 and 100, indicating that the street cleanliness rating is relatively high; line C represents SV between 100 and 150, indicating that the street cleanliness rating is medium; line D represents SV between 150 and 200, indicating that the street cleanliness rating is low; the line E represents SV greater than 200 and the street cleanliness rating is very low, so it can be seen that the lines D and E represent a high volume of street waste requiring municipal personnel to arrange personnel for cleaning.

Fig. 9 is a block level evaluation of jiangning district based on street level cleanliness evaluation, wherein 9 blocks are divided in the jiangning district, cleanliness evaluation of the 9 blocks is given according to a block level evaluation formula, A, B, C represents different cleannesses, and a block a and a block B represent lower cleanliness levels and the road surface is cleaner. The C block represents medium cleanliness, and there is rubbish on the local road surface, for example, the estimated value of the street cleanliness of the tombstone belonging to the block level is 119, which belongs to the C level. From this determination, the block requires the municipal staff to attend to the cleaning.

And finally, the city manager can evaluate the cleanliness of the city streets at different levels according to the needs of the manager, acquire the cleanliness levels of the streets in each area of the city in real time and reasonably arrange cleaning personnel.

TABLE 1 Total weighted number of spam

TABLE 2 garbage Categories and weights thereof

TABLE 3 value of correction factor λ

TABLE 4 street cleanliness index Classification

Claims (4)

1. A city street garbage detection and cleanliness assessment method based on mobile edge calculation and deep learning is characterized by comprising the following steps:

step 1: carrying out street view image data collection by a garbage collection vehicle and mobile equipment held by urban residents; the method comprises the following steps:

step 11: using a cleaning vehicle provided with a camera to take a picture at a fixed point on a street according to a set route, wherein the picture comprises the garbage condition of the street; meanwhile, the distance between adjacent shooting points is set by an administrator, each shooting point carries out omnibearing shooting according to the front direction, the rear direction, the left direction and the right direction, and the shooting range is set to be 150-300m²(ii) a For the cleaning vehicle, the following regulations are set: 1) a fixed image resolution; 2) a fixed vehicle speed; 3) a fixed distance of shot points; 4) 4 pictures are taken at each shooting point;

step 12: if the urban residents hold the photographing equipment to autonomously collect street image information, the photographing rule refers to the cleaning vehicle;

step 2: the edge server preprocesses the street view image;

step 3, the cloud server performs garbage detection model training;

and 4, step 4: the cloud server performs garbage detection on the street view images and performs garbage counting;

and 5: calculating the street cleanliness based on the garbage recognition result; the method comprises the following steps:

step 51: a minimum number n of street samples is first determined,

wherein N is the minimum number of samples, k is the sampling interval, p represents the probability of sampling occurrence, q represents the probability of sampling non-occurrence, and is recorded as q-1-p, N is the total number of city streets, and e is the estimation error;

step 52: a street cleanliness index value SV is calculated,

wherein S is the observation region; λ and a are correction factors that affect city street cleanliness, and C is the street garbage weighted number;

step 53: classifying the cleanliness of the streets according to the street cleanliness index value;

step 6: displaying each level cleanliness map of the city based on a level evaluation model; the method comprises the following steps:

step 61: establishing a hierarchical cleanliness model which is divided into four layers of city, area, block and street;

step 62: based on the level cleanliness model, firstly, performing street level evaluation, namely street cleanliness calculation values, and the specific calculation process is referred to step 52;

and step 63: calculating the block level model cleanliness BV,

where BV is the rating of a block in the region, SV represents the rating of each street, m₁Represents the total number of streets within a block;

step 64: the area level model cleanliness AV is calculated,

where AV represents the evaluation value of an area, BV represents the evaluation value of each block in the area, m₂Representing the total number of blocks in the region;

step 65: calculating the cleanliness CV of the urban hierarchical model,

where CV represents the estimated value of a city, AV is the estimated value of each area in the city, and m₃Represents the total number of regions in the city;

and step 66: and displaying the cleanliness maps of all levels of the city based on the cleanliness calculation results of all levels.

2. The city street garbage detection and cleanliness assessment method based on mobile edge computing and deep learning as claimed in claim 1, wherein the street view image data preprocessing process performs picture screening through an edge server to remove some unnecessary information to reduce the time delay of the whole system processing, said step 2 comprises the following steps:

step 21: the edge server cuts the collected street view image data, and the picture size is 420 x 400 pixels;

step 22: and then, carrying out artificial road detection and screening, if the picture is detected to contain a road feasible region, the picture is effective data, transmitting the effective data to a cloud center for road garbage detection, and if the picture is detected to have no road feasible region, deleting the image.

3. The city street garbage detection and cleanliness assessment method based on mobile edge computing and deep learning as claimed in claim 1, wherein in the step 3 cloud server garbage detection model training process, the extraction of candidate boxes and the identification of candidate box targets are realized through two modules, RPN and Fast R-CNN, the Fast-RCNN algorithm is a method for computer vision field target detection, and the step 3 comprises the following steps:

step 31: preparing for obtaining a characteristic diagram based on the design of the ZF-Net network;

step 32: inputting a street picture data set containing garbage, and transmitting the street picture data set into a ZF convolutional neural network for feature extraction;

step 33: carrying out RPN network pre-training, generating a region suggestion frame, and giving a region suggestion and a region score;

step 34: performing Fast R-CNN network end-to-end fine tuning training by using the area suggestion box obtained in the step 33;

step 35: reinitializing the RPN network by using the Fast R-CNN network after the fine tuning in the step 34, fixing the shared convolution layer, finely tuning the unique layer of the RPN network through the regression of a boundary frame, and simultaneously generating a suggested region;

step 36: the convolution layer is shared in the fixing step 35, while the fully-connected layers of Fast R-CNN are fine-tuned using the region recommendations obtained in step 35.

4. The city street garbage detection and cleanliness assessment method based on moving edge calculation and deep learning according to claim 1, wherein for garbage detection using a trained model, the step 4 further comprises:

step 41: inputting a street test image;

step 42: reflecting the image characteristics to a characteristic diagram through calculation based on the ZF convolutional neural network;

step 43: correspondingly calculating a candidate area by each RPN candidate convolutional neural network to generate a candidate suggestion frame; step 44: the candidate suggestion frame displays a garbage candidate region frame and the classification score of the region through a full connection layer, namely a classification layer and a regression layer;

step 45: counting the generated candidate region frames by using a counting function, namely, the number of detected garbage, wherein the number function is designed to be

Wherein C is a counting function for generating the candidate box, that is, the number of the detected garbage of a certain category, f is a result function detected by the garbage model, D is a test sample set, x is a test sample, y is a real garbage mark, and i is the number of input test samples.

Images