CN114283387B - Intelligent garbage point cleaning work order generation method and device and related medium - Google Patents
Intelligent garbage point cleaning work order generation method and device and related medium Download PDFInfo
- Publication number
- CN114283387B CN114283387B CN202210218590.XA CN202210218590A CN114283387B CN 114283387 B CN114283387 B CN 114283387B CN 202210218590 A CN202210218590 A CN 202210218590A CN 114283387 B CN114283387 B CN 114283387B
- Authority
- CN
- China
- Prior art keywords
- garbage
- overflow
- state
- work order
- rectangular frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000010813 municipal solid waste Substances 0.000 title claims abstract description 253
- 238000004140 cleaning Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001514 detection method Methods 0.000 claims abstract description 47
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 15
- 230000001537 neural effect Effects 0.000 claims abstract description 14
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 10
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 23
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 12
- 230000005284 excitation Effects 0.000 claims description 11
- 230000003044 adaptive effect Effects 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 10
- 230000010354 integration Effects 0.000 claims description 6
- 238000012549 training Methods 0.000 description 6
- 230000001960 triggered effect Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000007613 environmental effect Effects 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Landscapes
- Processing Of Solid Wastes (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and a related medium for intelligently generating a garbage point cleaning work order, wherein the method comprises the following steps: acquiring video data of a garbage point position through monitoring equipment, and performing frame extraction processing on the video data to obtain corresponding garbage point image data; performing target detection on the garbage point image data by adopting a target detection algorithm; calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result; and inputting the overflow rate of the garbage can and the garbage stacking amount into a self-adaptive neural fuzzy inference system, and outputting the probability of triggering the cleaning work order by the self-adaptive neural fuzzy inference system. The invention comprehensively calculates the garbage points by detecting the overflow rate and the garbage stacking amount of the garbage cans in the garbage points so as to generate the corresponding cleaning work order, thus improving the generation efficiency of the cleaning work order and further improving the cleaning efficiency of the garbage points.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a device for intelligently generating a garbage point cleaning work order and a related medium.
Background
With the improvement of living standard and civilization consciousness of people, the requirements on living environment are increasing. The ubiquitous trash can points need to be cleaned in time by environmental cleaning personnel under the conditions of overflowing and trash piling nearby the trash can. Due to different living habits and garbage bin capacities of people, the time-space change of the overflowing garbage bin is not strict, and the condition of timely treating the overflowing garbage bin cannot be met even if environmental protection personnel regularly clean the garbage bin.
With the development of technologies such as artificial intelligence and the like, aiming at the situation of overflowing of the garbage can, an intelligent garbage can or a garbage station detects the depth of garbage in the corresponding garbage can in a mode of installing infrared sensors and the like, transmits monitoring data in real time and reminds the overflowing of the garbage can; in addition, there is also a solution to identify whether the trash can is overfilled by a visual device such as a camera.
Although the method for installing the infrared sensors is high in reliability, each garbage can needs to be provided with the corresponding sensor, the cost is relatively high, energy consumption is needed, and garbage near the garbage can cannot be treated; and whether current detect the garbage bin through the vision camera is excessive, can't give concrete overflow degree, some unable conditions of handling a plurality of garbage bins to and do not detect the rubbish of stacking near the garbage bin.
Disclosure of Invention
The embodiment of the invention provides a method and a device for intelligently generating a garbage point cleaning work order, computer equipment and a storage medium, and aims to improve the generation efficiency of the cleaning work order so as to improve the cleaning efficiency of garbage points.
In a first aspect, an embodiment of the present invention provides an intelligent garbage spot cleaning work order generation method, including:
acquiring video data of a garbage point position through monitoring equipment, and performing frame extraction processing on the video data to obtain corresponding garbage point image data;
performing target detection on the garbage point image data by adopting a target detection algorithm;
calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result;
and inputting the overflow rate of the garbage can and the garbage stacking amount into a self-adaptive neural fuzzy inference system, and outputting the probability of triggering the cleaning work order by the self-adaptive neural fuzzy inference system.
In a second aspect, an embodiment of the present invention provides an apparatus for intelligently generating a garbage spot cleaning work order, including:
the data acquisition unit is used for acquiring video data of the position of the garbage point through monitoring equipment and performing frame extraction processing on the video data to obtain corresponding garbage point image data;
the target detection unit is used for carrying out target detection on the garbage point image data by adopting a target detection algorithm;
the first calculating unit is used for calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result;
and the probability output unit is used for inputting the overflow rate of the garbage can and the garbage stacking amount into the self-adaptive neuro-fuzzy inference system and outputting the probability of triggering the cleaning work order by the self-adaptive neuro-fuzzy inference system.
In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the intelligent garbage collection work order generation method according to the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the intelligent garbage spot cleaning work order generation method according to the first aspect.
The embodiment of the invention provides a method and a device for intelligently generating a garbage point cleaning work order, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring video data of a garbage point position through monitoring equipment, and performing frame extraction processing on the video data to obtain corresponding garbage point image data; performing target detection on the garbage point image data by adopting a target detection algorithm; calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result; and inputting the overflow rate of the garbage can and the garbage stacking amount into a self-adaptive neural fuzzy inference system, and outputting the probability of triggering the cleaning work order by the self-adaptive neural fuzzy inference system. According to the embodiment of the invention, the garbage points are comprehensively calculated by detecting the overflow rate and the garbage stacking amount of the garbage cans in the garbage points, so that the corresponding work orders are generated, and thus, the generation efficiency of the work orders can be improved, and the cleaning efficiency of the garbage points is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for intelligently generating a garbage spot cleaning work order according to an embodiment of the present invention;
fig. 2 is a sub-flow diagram of an intelligent garbage collection work order generation method according to an embodiment of the present invention;
fig. 3 is an exemplary diagram of target detection in an intelligent garbage spot cleaning work order generation method according to an embodiment of the present invention;
fig. 4 is an exemplary diagram of a work order generated by the intelligent garbage spot work order generation method according to the embodiment of the present invention;
fig. 5 is an exemplary graph of a fuzzy membership function in the intelligent garbage collection work order generation method according to the embodiment of the present invention;
fig. 6 is a schematic network structure diagram of an adaptive neural fuzzy inference system in an intelligent garbage collection work order generation method according to an embodiment of the present invention;
fig. 7 is a schematic network structure diagram of an intelligent garbage collection work order generation method according to an embodiment of the present invention;
fig. 8 is a schematic block diagram of an intelligent garbage spot cleaning work order generation apparatus according to an embodiment of the present invention;
fig. 9 is a sub-schematic block diagram of an intelligent garbage collection work order generation apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1, fig. 1 is a schematic flow chart of an intelligent garbage collection work order generation method according to an embodiment of the present invention, which specifically includes: s101 to S104.
S101, video data of a garbage point position is obtained through monitoring equipment, and frame extraction processing is carried out on the video data to obtain corresponding garbage point image data;
s102, performing target detection on the garbage point image data by adopting a target detection algorithm;
s103, calculating the overflow rate and the garbage stacking amount of the garbage can in the garbage point according to the target detection result;
and S104, inputting the overflow rate of the garbage can and the garbage stacking amount into a self-adaptive neuro-fuzzy inference system, and outputting the probability of triggering a cleaning work order by the self-adaptive neuro-fuzzy inference system.
In this embodiment, first, video data near a garbage point is acquired, and frame extraction processing is performed on the video data to obtain corresponding garbage point image data. And then, carrying out a target detection model on the obtained image data of the garbage points, calculating to obtain the overflow rate and the stacking quantity of the garbage (namely the garbage scattered or stacked outside the garbage) of the garbage can, and comprehensively calculating the overflow rate and the stacking quantity of the garbage can by using a self-adaptive neuro-fuzzy reasoning system so as to obtain the probability of whether a cleaning work order needs to be triggered.
This embodiment is through detecting the garbage bin overflow rate in the rubbish point and rubbish volume of piling up, comes to synthesize to the rubbish point and calculates, and then generates corresponding work order of keeping a public place clean, so, can improve the generation efficiency of work order of keeping a public place clean to the improvement is to the clean efficiency of rubbish point. The main technology used by the embodiment comprises computer vision, mode recognition, deep learning and the like, images can be acquired from visual equipment such as a camera near the garbage can to detect the state of the garbage can and garbage, and an environmental protection business work order is generated intelligently by combining the business experience of environmental protection personnel, so that the efficiency of timely cleaning the overflowing garbage can and stacked garbage in each scene can be effectively improved.
In the specific embodiment, the video data of the monitoring camera at the garbage can placing point is obtained, and different time intervals t can be taken according to different application scenes0Frame extraction is carried out on video data to obtain a corresponding image, and a scene in practical application is considered by t0Taking 10min as an example, and then setting corresponding monitoring areas S at different garbage points according to service requirements.
In one embodiment, the step S102 includes:
inputting the garbage point image data into a backbone network of a Yolov5s network model, and extracting feature maps with different sizes by the backbone network; the backbone network sequentially comprises a Focus module, a bottleneck CSP layer, a CBL module and an SPP layer;
utilizing the hack layer to carry out series connection and integration on the feature maps with different sizes to obtain semantic information with different sizes;
inputting the garbage point image data with semantic information of different sizes into a prediction layer, and outputting an overflow state position rectangular frame of a garbage can corresponding to the garbage point image data by the prediction layer
And/or stacked position rectangular frame
And an overfill condition
Overfill confidence
And confidence of stacking
。
In this embodiment, the labeled image data and the corresponding label are divided into a training set and a test set according to a certain proportion, a data set with garbage overflowing in the stacked garbage is constructed, and a target detection network is adopted for training. Taking the Yolov5s network model as an example, the above effects can be achieved by adopting other target detection networks. The model of Yolov5s is mainly composed of a backbone network, a neck layer and a prediction layer. The backbone network is composed of a Focus module, a bottleneck CSP layer, a CBL module and an SPP layer, wherein the CBL module is a module formed by a convolution layer, a BN layer and a Leaky ReLU layer in a cascade mode.
In a backbone network, a Focus module firstly slices an input picture, and then generates a feature map of 32 channels through 32 convolution kernels. The CBL module and the bottleneck CSP layer play a role in convolution and feature extraction. Through the SPP layer, the network can increase the receptive field and obtain features of different sizes. The neck layer is a pyramid structure which is based on the FPN framework from bottom to top, and connects and integrates feature maps with different sizes in series, so that semantic information with different sizes can be obtained, and the extraction capability of the network on the feature and position information of target objects with different sizes is improved. The prediction layer outputs category information, coordinate information, and confidence information of the target object.
In an embodiment, before the step S102, the method includes:
carrying out state labeling on the garbage point image data by using a rectangular frame; the state label comprises an overflow state and a stacking state, wherein the overflow state comprises an empty state, a half-full state and a full state;
in this embodiment, before performing target detection on the garbage dot image data, a rectangular frame is first used to perform state labeling on the garbage dot image data, specifically as shown in table 1, for example, a garbage bin therein is labeled as an empty state, a half-full state, a stacked state, or the like, where the stacked state refers to garbage scattered or stacked outside the garbage bin.
TABLE 1
Further, in an embodiment, the method for intelligently generating the garbage spot cleaning work order further includes:
the overflow state position rectangular frame and the stacking state position rectangular frame are collectively called as a prediction rectangular frame, and a Yolov5s network model is trained and optimized through the following formula:
in the formula,
to mark the position penalty between the rectangular box and the predicted rectangular box,
to label the class penalty between the rectangular box and the predicted rectangular box,
the confidence loss between the labeling rectangular frame and the prediction rectangular frame is taken as the confidence loss;
wherein,Lbboxthe method is calculated by adopting GIoU loss:
in the formula,
indicating the area intersection ratio of the predicted rectangular box and the real rectangular box, C indicating the minimum closed convex surface that can cover the real predicted box and the predicted real box, "\" indicating the area of C that is not covered to the real predicted box and the predicted real box,
the actual value is represented by the value of,
representing a predicted value;
wherein,
and
all adopt cross entropy loss function
And calculating to obtain:
in the formula, wnRepresents a weight, YnRepresenting the authentic label after unique hot encoding,
the value of the table is predicted,
representing a Sigmoid function.
In this embodiment, the loss function used for training and optimizing the Yolov5s network model includes three parts, which are object loss, class loss and bbox loss, respectively. Wherein, the object loss and class loss adopt BCEWithLoitsLoss, and the following formula is adopted:
in the formula,
is that
And
，
the weight is represented by a weight that is,
representing a genuine label after one-hot encoding,
the value of the table is predicted,
representing a Sigmoid function.
Further, the trained weight file is loaded, the single frame image to be detected (i.e. the garbage point image data) is input into the target detection network, as shown in fig. 3, and according to the network output, the overflow state position rectangular frame b of each garbage can be obtainedi(i represents the number of detected trash cans, i =0,1, …, m), and overfill status fi，
The values are 1,2 and 3, which respectively correspond to 3 states of empty, half-full and full, and the confidence coefficients thereof
(ii) a In addition, if scattered and piled garbage exists, the rectangular frame b in the piled state position can be obtained at the same timej(j represents the number of detected spills or dumps), j =0,1, …, k and its confidence level
。
Preferably, the detection result is filtered by using an artificial preset area S, if the preset area and the detection rectangular frame are
Or
If the two are crossed, the two are reserved, otherwise, the two are removed.
In one embodiment, the step S103 includes:
setting a fuzzy membership function according to the overflow state of each garbage can
；
Combined with overflow state
And fuzzy membership function
Calculating the overflow rate of the garbage can according to the following formula
：
In the formula,
and with
Respectively representing the overflow degree values of the upper and lower limits of the interval when the membership degree is 1.0,
in order to be the confidence level of the overflow,
indicating an overfill condition of
Fuzzy membership function of time, i =1,2, 3;
position rectangular frame based on overflow state
According to the following formula, the overflow rate of the garbage cans of the m garbage cans is calculated:
in this embodiment, in order to calculate the specific value of the overflow rate of the trash can, the output of step S102, that is, the detected overflow state (empty, half full, full) of the trash can, is combined with the experience of the environmental protection service staffData statistics of line, determining fuzzy membership function
An example of a membership function is given. As shown in FIG. 4, u1(x), u2(x) and u3(x) are membership functions of empty, half full and full 3 overflow states corresponding to the overflow degree of garbage respectively
Further based on corresponding overflow status
，
) And calculating the overflow rate of each garbage can according to the fuzzy membership function:
wherein,
and
respectively represent
The overflow degree of the upper and lower limits of the interval with membership of 1.0, e.g. u1(x)
And
1/3, 0. And for m garbage cans, the overflow rate of all the garbage cans can be calculated on average.
In an embodiment, the step S103 further includes:
the average size of the rectangular frame of the m overflow status positions is calculated according to the following formula
：
Wherein,
the size of a rectangular frame at the ith overflow state position;
position rectangular frame based on stacking state
Calculating the garbage stacking amount N of the k stacked garbage according to the following formula:
in the formula,
for the confidence in the stacking of the stacks,
in a stacked state
The size of (c).
In this embodiment, if the detected amount of scattered or stacked garbage output by the prediction layer is k, the amount of scattered or stacked garbage is determined
Comprises the following steps:
wherein,
j-th rectangle frame for detecting "others" garbage piling or scattering
The area of (a). Considering that the actual sizes of rectangular frames with the same size in images are inconsistent in different scenes due to inconsistent relative positions of a camera and a garbage bin in different scenes, in order to better balance the scattered or stacked garbage amount, the detected scattered or stacked garbage amount is normalized by taking the average size of the rectangular frames of the garbage bin detected by the current image as a reference.
In one embodiment, as shown in fig. 2, the step S104 includes:
s201, fuzzifying the overflow rate of the garbage can and the garbage stacking amount through a layer 1 in a self-adaptive neural fuzzy inference system, and outputting a corresponding fuzzy result;
s202, carrying out fuzzy rule operation on the fuzzy result by utilizing the layer 2, and outputting corresponding rule excitation strength;
s203, normalizing the regular excitation intensity by utilizing the layer 3;
s204, utilizing the layer 4 to output and calculate the normalized regular excitation intensity;
s205, performing summation calculation on all outputs through the 5 th layer to obtain a total output result, and taking the total output result as the probability of triggering the cleaning work order.
In this embodiment, when the overflow rate of the trash can and the amount of stacked trash are input to an adaptive neuro-fuzzy inference system (ANFIS), with reference to fig. 5, a processing procedure of the adaptive neuro-fuzzy inference system is as follows:
layer 1: and fuzzifying the input garbage bin overflow rate M and the relative quantity N of the scattered or stacked garbage, and outputting a corresponding fuzzy result. Each node of the layer is an adaptive node having a node function;
layer 2: and the fuzzy rule operation is realized. The nodes of the layer are fixed nodes, and each node is an algebraic product of all input signals and represents the excitation strength of a rule. The node function can also adopt a form of taking a small, bounded product or strong product;
layer 3: normalizing the regular excitation intensity output by the upper layer, wherein the nodes in the layer are also fixed nodes;
layer 4: the output of each rule is calculated. Each node of the layer is an adaptive node with a node function;
layer 5: the sum of all the transmitted signals is calculated as the total output. The single node of this layer is a fixed node of summation.
Specifically, in the training phase of the adaptive neuro-fuzzy inference system: acquiring a certain number of garbage point images, calculating the overflow rate M of the garbage can and the relative quantity N of scattered or stacked garbage of each image according to the steps 1-3, and marking the probability that each image may trigger a work order by combining with the experience of a service expert as an input label. Then, parameter learning is performed by using a least square method and a gradient descent method.
In the testing stage of the adaptive neural fuzzy inference system: inputting the overflow rate and the scattering or garbage stacking amount of the garbage can into ANFIS, calculating through a fuzzy layer and the like, and finally outputting the probability g for triggering work order actions.
In one embodiment, as shown in FIG. 6, when the work order is triggered, the detailed results as shown in FIG. 6, such as the preselected area S, the rectangular frame, etc., may be displayed on the work order
And
overflow rate of each garbage can
Etc., as well as site specific data, etc. In addition, it should be noted that, in most of the conventional work order generation, a single index or a plurality of indexes are adoptedIn the embodiment, the multiple indexes are adaptively learned as the trigger work order probability of the position, so that the intuitiveness of a single index is utilized, and the comprehensiveness of the multiple indexes is comprehensively considered.
In other embodiments, the work orders may be triggered in other manners, for example, when the overflow rate of the trash can and/or the amount of the trash stacked reaches a threshold value to be cleaned, the corresponding work orders may be triggered according to the business rules. In different application scenes, the tolerance degrees of the 2 indexes (namely the overflow rate of the garbage can and the garbage stacking amount) are different, and the indexes depend on certain business experience, so that the cleaning work order can be triggered in the following modes:
if the standard of the total overflow rate of the garbage can is M0=30%, the relative quantity of scattered or stacked garbage is N0=2.0, then the cleaning business rule can be defined as:
(1)if 
and
cleaning is needed, and the degree: emergency;
(2)if 
and
cleaning is needed, and the degree: generally;
(3)if 
and
no need of cleaning, degree: is not important;
compared with the business rules, the method for triggering the cleaning work order by the Adaptive Neural Fuzzy Inference System (ANFIS) is more accurate and quantitative, namely whether cleaning is needed or not is mapped into the range of [0,1], and business judgment is facilitated.
In another specific embodiment, with reference to fig. 7, the method for intelligently generating a garbage spot cleaning work order provided in the embodiment of the present invention specifically includes four parts, namely, data acquisition, target detection, index calculation, and work order triggering. The data acquisition part acquires video data, performs frame extraction processing on the video data to obtain a corresponding image, and sets a preselected area in the image. And in the target detection part, performing data annotation on the obtained image so as to train a target detection model, performing inference test by using the trained target detection model, and filtering results in a non-preselected region through a set preselected region. Then, in the index calculation part, firstly, the overflow membership function of the garbage points is determined, and then the overflow rate of the garbage can and the relative quantity of scattered or stacked garbage are calculated. Therefore, in the work order triggering part, parameter learning and reasoning tests can be carried out on the overflow rate of the garbage can and the relative quantity of scattered or stacked garbage by utilizing an ANFIS structure, so that the action probability for judging whether the work order is triggered is obtained.
Fig. 8 is a schematic block diagram of an apparatus 800 for intelligently generating a garbage spot cleaning work order according to an embodiment of the present invention, where the apparatus 800 includes:
a data obtaining unit 801, configured to obtain video data of a garbage point position through a monitoring device, and perform frame extraction processing on the video data to obtain corresponding garbage point image data;
a target detection unit 802, configured to perform target detection on the spam dot image data by using a target detection algorithm;
a first calculating unit 803, configured to calculate an overflow rate and a garbage stacking amount of a garbage can in a garbage point according to a target detection result;
and the probability output unit 804 is used for inputting the overflow rate of the garbage can and the garbage stacking amount into the self-adaptive neuro-fuzzy inference system, and outputting the probability of triggering the cleaning work order by the self-adaptive neuro-fuzzy inference system.
In one embodiment, the object detection unit 802 includes:
the characteristic diagram extraction unit is used for inputting the garbage point image data into a backbone network of a Yolov5s network model, and extracting characteristic diagrams with different sizes by the backbone network; the backbone network sequentially comprises a Focus module, a bottleneck CSP layer, a CBL module and an SPP layer;
the series and integration unit is used for carrying out series connection and integration on the feature graphs with different sizes by utilizing the hack layer to obtain semantic information with different sizes;
a prediction output unit for inputting the garbage point image data with semantic information of different sizes into a prediction layer and outputting the overflow state position rectangular frame of the garbage can corresponding to the garbage point image data by the prediction layer
And/or stacked position rectangular frame
And an overfill condition
And overflow confidence
And confidence of stacking
。
In an embodiment, the intelligent garbage spot cleaning work order generating device 800 further includes:
the state labeling unit is used for labeling the state of the garbage point image data by using a rectangular frame; the state label comprises an overflow state and a stacking state, wherein the overflow state comprises an empty state, a half-full state and a full state;
in an embodiment, the intelligent garbage spot cleaning work order generating device 800 further includes:
the training optimization unit is used for collectively referring the overflow state position rectangular frame and the stacking state position rectangular frame as a prediction rectangular frame, and training and optimizing a Yolov5s network model according to the following formula:
in the formula,
to mark the position penalty between the rectangular box and the predicted rectangular box,
to label the class penalty between the rectangular box and the predicted rectangular box,
the confidence loss between the labeling rectangular frame and the prediction rectangular frame is taken as the confidence loss;
wherein,Lbboxthe method is calculated by adopting GIoU loss:
in the formula,
indicating the area intersection ratio of the predicted rectangular box and the real rectangular box, C indicating the minimum closed convexity that can cover the real predicted box and the predicted real box, "\" indicating the area of C that is not covered to the real predicted box and the predicted real box,
the actual value is represented by the value of,
representing a predicted value;
wherein,
and
all adopt cross entropy loss function
And calculating to obtain:
in the formula, wnRepresents a weight, YnRepresenting a genuine label after one-hot encoding,
the value of the table is predicted,
representing a Sigmoid function.
In one embodiment, the first computing unit 803 comprises:
a function setting unit for setting fuzzy membership function according to the overflow state of each garbage can
；
An overflow rate calculation unit for combining the overflow status
And fuzzy membership function
Calculating the overflow rate of the garbage can according to the following formula
：
In the formula,
and
respectively representing the overflow degree values of the upper and lower limits of the interval when the membership degree is 1.0,
in order to be the confidence level of the overflow,
indicating an overfill condition of
Fuzzy membership function of time, i =1,2, 3;
a second computing unit for positioning the rectangular frame based on the overflow state
According to the following formula, the overflow rate of the garbage cans of the m garbage cans is calculated:
in an embodiment, the first computing unit 803 further comprises:
an average size calculation unit for calculating the average size of the m overflow state position rectangular frames according to the following formula
：
Wherein,
the size of a rectangular frame at the ith overflow state position;
a stacking amount calculating unit for positioning the rectangular frame based on the stacking state
According to the following formula, calculatekThe individual garbage stacking amount N for stacking garbage:
in the formula,
in order to provide confidence in the stacking,
in a stacked state
Of the cell.
In one embodiment, as shown in fig. 9, the probability output unit 804 includes:
the fuzzification processing unit 901 is used for fuzzifying the overflow rate and the garbage stacking amount of the garbage can through a layer 1 in the self-adaptive neural fuzzy inference system and outputting a corresponding fuzzy result;
a rule operation unit 902, configured to perform fuzzy rule operation on the fuzzy result by using the layer 2, and output a corresponding rule excitation strength;
a normalization processing unit 903, configured to perform normalization processing on the regular excitation strength by using the layer 3;
an output calculation unit 904, configured to perform output calculation on the normalized regular excitation intensity by using the layer 4;
and the summation calculation unit 905 is used for performing summation calculation on all the outputs through the 5 th layer to obtain a total output result, and taking the total output result as the probability of triggering the cleaning work order.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps provided by the above embodiments can be implemented. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiment of the present invention further provides a computer device, which may include a memory and a processor, where the memory stores a computer program, and the processor may implement the steps provided in the above embodiments when calling the computer program in the memory. Of course, the computer device may also include various network interfaces, power supplies, and the like.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (7)
1. An intelligent generating method of a garbage point cleaning work order is characterized by comprising the following steps:
acquiring video data of a garbage point position through monitoring equipment, and performing frame extraction processing on the video data to obtain corresponding garbage point image data;
performing target detection on the garbage point image data by adopting a target detection algorithm;
before the target detection is performed on the garbage point image data by adopting the target detection algorithm, the method comprises the following steps:
carrying out state labeling on the garbage point image data by using a rectangular frame to obtain a labeled rectangular frame; the state labels comprise an overflow state and a stacking state, and the overflow state comprises an empty state, a half-full state and a full state;
the method for performing target detection on the garbage point image data by adopting the target detection algorithm comprises the following steps:
inputting the garbage point image data into a backbone network of a Yolov5s network model, and extracting feature maps with different sizes by the backbone network; the Yolov5s network model comprises a backbone network, a neck layer and a prediction layer, wherein the backbone network sequentially comprises a Focus module, a bottleneck CSP layer, a CBL module and an SPP layer;
utilizing the hack layer to carry out series connection and integration on the feature maps with different sizes to obtain semantic information with different sizes;
inputting the garbage point image data with semantic information of different sizes into a prediction layer, and outputting an overflow state position rectangular frame of a garbage can corresponding to the garbage point image data by the prediction layer
And/or stacked position rectangular frame
And an overfill condition
Overfill confidence
And confidence of stacking
；
Calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result;
the step of calculating the overflow rate and the garbage stacking amount of the garbage can in the garbage point according to the target detection result comprises the following steps:
setting a fuzzy membership function according to the overflow state of each garbage can
；
Combined with overflow state
And fuzzy membership function
The garbage is calculated according to the following formulaOverflow rate of garbage can
：
In the formula,
and
respectively representing the overflow degree values of the upper and lower limits of the interval when the membership degree is 1.0,
in order to be the confidence level of the overflow,
indicating an overfill condition of
A function of fuzzy membership in time,
=1,2,3, i represents the overflow status position rectangular frame of the ith detected trash can;
position rectangular frame based on overflow state
According to the following formula, the overflow rate of the garbage cans of the m garbage cans is calculated:
and inputting the overflow rate of the garbage can and the garbage stacking amount into a self-adaptive neural fuzzy inference system, and outputting the probability of triggering the cleaning work order by the self-adaptive neural fuzzy inference system.
2. The intelligent garbage spot cleaning work order generation method according to claim 1, further comprising:
the overflow state position rectangular frame and the stacking state position rectangular frame are collectively called as a prediction rectangular frame, and a Yolov5s network model is trained and optimized through the following formula:
in the formula,
to mark the position penalty between the rectangular box and the predicted rectangular box,
to label the class penalty between the rectangular box and the predicted rectangular box,
the confidence loss between the labeling rectangular frame and the prediction rectangular frame is taken as the confidence loss;
wherein,
the method is calculated by adopting GIoU loss:
in the formula,
indicating the area intersection ratio of the predicted rectangular box and the real rectangular box, C indicating the minimum closed convexity that can cover the real predicted box and the predicted real box, "\" indicating the area of C that is not covered to the real predicted box and the predicted real box,
the actual value is represented by the value of,
representing a predicted value;
wherein,
and
all adopt cross entropy loss function
And calculating to obtain:
in the formula,
the weight is represented by a weight that is,
representing a genuine label after one-hot encoding,
the value of the table is predicted,
representing a Sigmoid function.
3. The method for intelligently generating a garbage spot cleaning work order according to claim 1, wherein the method for calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage spot according to the target detection result further comprises:
the average size of the rectangular frame of the m overflow status positions is calculated according to the following formula
：
Wherein,
the size of a rectangular frame at the ith overflow state position;
rectangular frame based on stacking state position
According to the following formula, calculatekThe individual garbage stacking amount N for stacking garbage:
in the formula,
for the confidence in the stacking of the stacks,
in a stacked state
The size of (c).
4. The intelligent garbage spot cleaning work order generation method according to claim 1, wherein the step of inputting the overflow rate of the garbage can and the garbage stacking amount into an adaptive neuro-fuzzy inference system, and outputting the probability of triggering the cleaning work order by the adaptive neuro-fuzzy inference system comprises the steps of:
fuzzification processing is carried out on the overflow rate of the garbage can and the garbage stacking amount through a layer 1 in a self-adaptive neural fuzzy inference system, and a corresponding fuzzy result is output;
carrying out fuzzy rule operation on the fuzzy result by utilizing the layer 2, and outputting corresponding rule excitation intensity;
normalizing the regular excitation intensity by utilizing a layer 3;
utilizing the layer 4 to output and calculate the normalized regular excitation intensity;
and performing summation calculation on all outputs through the 5 th layer to obtain a total output result, and taking the total output result as the probability of triggering the cleaning work order.
5. The utility model provides a rubbish point cleaning work order intelligence generates device which characterized in that includes:
the data acquisition unit is used for acquiring video data of the position of the garbage point through monitoring equipment and performing frame extraction processing on the video data to obtain corresponding garbage point image data;
the target detection unit is used for carrying out target detection on the garbage point image data by adopting a target detection algorithm;
the intelligent garbage point cleaning work order generating device further comprises:
the state labeling unit is used for labeling the state of the garbage point image data by using a rectangular frame; the state label comprises an overflow state and a stacking state, wherein the overflow state comprises an empty state, a half-full state and a full state;
the target detection unit includes:
the characteristic diagram extraction unit is used for inputting the garbage point image data into a backbone network of a Yolov5s network model, and extracting characteristic diagrams with different sizes by the backbone network; the Yolov5s network model comprises a backbone network, a neck layer and a prediction layer, wherein the backbone network sequentially comprises a Focus module, a bottleneck CSP layer, a CBL module and an SPP layer;
the series and integration unit is used for carrying out series connection and integration on the feature graphs with different sizes by utilizing the hack layer to obtain semantic information with different sizes;
a prediction output unit for inputting the garbage point image data with semantic information of different sizes into a prediction layer and outputting the overflow state position rectangular frame of the garbage can corresponding to the garbage point image data by the prediction layer
And/or stacked position rectangular frame
And an overfill condition
Overfill confidence
And confidence of stacking
；
The first calculating unit is used for calculating the overflow rate of the garbage can and the garbage stacking amount in the garbage point according to the target detection result;
the first calculation unit includes:
a function setting unit for setting fuzzy membership function according to the overflow state of each garbage can
；
An overflow rate calculation unit for combining the overflow status
And fuzzy membership function
Calculating the overflow rate of the garbage can according to the following formula
：
In the formula,
and
respectively representing the overflow degree values of the upper and lower limits of the interval when the membership degree is 1.0,
in order to be the confidence level of the overflow,
indicating an overfill condition of
A function of fuzzy membership in time,
=1,2,3, i denotes the ith detected trashA barrel overflow state position rectangular frame;
a second calculation unit for positioning the rectangular frame based on the overflow state
According to the following formula, the overflow rate of the garbage cans of m garbage cans is calculated:
and the probability output unit is used for inputting the overflow rate of the garbage can and the garbage stacking amount into the self-adaptive neuro-fuzzy inference system and outputting the probability of triggering the cleaning work order by the self-adaptive neuro-fuzzy inference system.
6. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the intelligent garbage collection sheet generation method of any one of claims 1 to 4 when executing the computer program.
7. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the intelligent garbage spot cleaning work order generating method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210218590.XA CN114283387B (en) | 2022-03-08 | 2022-03-08 | Intelligent garbage point cleaning work order generation method and device and related medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210218590.XA CN114283387B (en) | 2022-03-08 | 2022-03-08 | Intelligent garbage point cleaning work order generation method and device and related medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114283387A CN114283387A (en) | 2022-04-05 |
| CN114283387B true CN114283387B (en) | 2022-06-24 |
Family
ID=80882321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210218590.XA Active CN114283387B (en) | 2022-03-08 | 2022-03-08 | Intelligent garbage point cleaning work order generation method and device and related medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114283387B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115043111A (en) * | 2022-05-23 | 2022-09-13 | 海南省量心环保科技有限公司 | Intelligent garbage can detection system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883921A (en) * | 2021-03-22 | 2021-06-01 | 北京易华录信息技术股份有限公司 | Garbage can overflow detection model training method and garbage can overflow detection method |
| CN113255588A (en) * | 2021-06-24 | 2021-08-13 | 杭州鸿泉物联网技术股份有限公司 | Garbage cleaning method and device for garbage sweeper, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112777169B (en) * | 2020-12-24 | 2022-03-08 | 中标慧安信息技术股份有限公司 | Internet of things monitoring method and system applied to garbage classification putting points |
| CN113468976B (en) * | 2021-06-10 | 2024-08-06 | 浙江大华技术股份有限公司 | Garbage detection method, garbage detection system, and computer-readable storage medium |
| CN113450401A (en) * | 2021-07-19 | 2021-09-28 | 北京航空航天大学杭州创新研究院 | Trash can fullness degree determining method, device and equipment and trash can |
| CN114119959A (en) * | 2021-11-09 | 2022-03-01 | 盛视科技股份有限公司 | Vision-based garbage can overflow detection method and device |
-
2022
- 2022-03-08 CN CN202210218590.XA patent/CN114283387B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883921A (en) * | 2021-03-22 | 2021-06-01 | 北京易华录信息技术股份有限公司 | Garbage can overflow detection model training method and garbage can overflow detection method |
| CN113255588A (en) * | 2021-06-24 | 2021-08-13 | 杭州鸿泉物联网技术股份有限公司 | Garbage cleaning method and device for garbage sweeper, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114283387A (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Nowakowski et al. | Application of deep learning object classifier to improve e-waste collection planning | |
| Yang et al. | WasNet: A neural network-based garbage collection management system | |
| Arebey et al. | Solid waste bin level detection using gray level co-occurrence matrix feature extraction approach | |
| CN108460362A (en) | A kind of system and method for detection human body | |
| CN110087193A (en) | Information uploading method, device, electronic equipment and the readable storage medium storing program for executing of dustbin | |
| CN111611970B (en) | Urban management monitoring video-based random garbage throwing behavior detection method | |
| CN113657143B (en) | Garbage classification method based on classification and detection combined judgment | |
| KR102324684B1 (en) | Marine debris monitoring system based on unmanned observation and marine debris monitoring method using thereof | |
| CN114283387B (en) | Intelligent garbage point cleaning work order generation method and device and related medium | |
| CN112257799A (en) | Method, system and device for detecting household garbage target | |
| CN116682098A (en) | Automatic urban household garbage identification and classification system and method | |
| CN113052005B (en) | Garbage sorting method and garbage sorting device for household service | |
| CN110738131A (en) | Garbage classification management method and device based on deep learning neural network | |
| CN112707058B (en) | Detection method, system, device and medium for standard actions of kitchen waste | |
| CN106875061A (en) | Method and relevant apparatus that a kind of destination path determines | |
| CN115545441A (en) | Road garbage detection method, system, terminal and storage medium | |
| CN115035474A (en) | Scene attention-based garbage detection method and device and related medium | |
| CN112183460A (en) | Method and device for intelligently identifying environmental sanitation | |
| Hin et al. | An Intelligent Smart Bin for Waste Management | |
| CN115035442A (en) | Garbage classification collection and transportation supervision method based on improved YOLOv3 network | |
| CN118097249A (en) | Object detection model-based target detection method and device and electronic equipment | |
| CN115984361B (en) | Garbage can overflow detection method and system | |
| Zailan et al. | YOLO-based Network Fusion for Riverine Floating Debris Monitoring System | |
| KR102219025B1 (en) | Smart cctv system for analysis of waste | |
| Arebey et al. | Bin level detection using gray level co-occurrence matrix in solid waste collection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |