CN113642509A - Garbage bin overflow state detection method and device, storage medium and electronic equipment - Google Patents
Garbage bin overflow state detection method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN113642509A CN113642509A CN202110996174.8A CN202110996174A CN113642509A CN 113642509 A CN113642509 A CN 113642509A CN 202110996174 A CN202110996174 A CN 202110996174A CN 113642509 A CN113642509 A CN 113642509A
- Authority
- CN
- China
- Prior art keywords
- garbage
- state
- frame
- trash
- overflow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for detecting the overflow state of a garbage can, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring an image to be detected; carrying out multi-target detection on an image to be detected to obtain at least one target detection frame; if the at least one target detection frame comprises a garbage bin detection frame, performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, acquiring the current state of the garbage can according to the historical state of the garbage can. Therefore, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage can is determined according to the historical state of the garbage can, so that false alarm caused by shielding of the garbage can be reduced.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a garbage can overflow state detection method and device, a storage medium and electronic equipment.
Background
At present, the management to the garbage bin is still mostly in the stage by the regularly clearance of cleaning staff, in order to reduce the condition that the garbage bin can't in time obtain the clearance, can shorten the clearance interval of garbage bin long usually, but this can increase cleaning staff's work burden. In order to solve the problem, a computer vision algorithm is provided in the related technology to alarm the overflow state of the garbage can, specifically, a video image to be detected is input into a detection model to perform target detection to obtain a detection result, the detection result is compared with a detection threshold value, when the detection result is greater than the detection threshold value, classification processing is performed according to a classifier to obtain a classification result, and if the classification result is garbage overflow, prompt information is sent. Although this mode can realize the remote monitoring to the garbage bin, but after having objects such as pedestrian, motor vehicle to shelter from, although can detect the garbage bin, when carrying out classification processing through the classifier, the probability greatly increased of misclassification, wrong report to police, increased clean staff's work burden on the contrary.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, a first object of the present invention is to provide a method for detecting an overflow state of a trash can, in which at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is determined according to the historical state of the trash can, so that false alarms caused by shielding of the trash can be reduced, and further, the workload of cleaning workers is reduced.
The invention also provides a device for detecting the overflow state of the garbage bin.
A third object of the invention is to propose a computer-readable storage medium.
A fourth object of the invention is to propose an electronic device.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for detecting an overflow state of a trash can, including: acquiring an image to be detected; carrying out multi-target detection on an image to be detected to obtain at least one target detection frame; if the at least one target detection frame comprises a garbage bin detection frame, performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, acquiring the current state of the garbage can according to the historical state of the garbage can.
According to the garbage bin overflow state detection method, an image to be detected is obtained, multi-target detection is conducted on the image to be detected, at least one target detection frame is obtained, if the at least one target detection frame comprises a garbage bin detection frame, target tracking is conducted on each target detection frame in the at least one target detection frame, at least one target tracking frame is obtained, and if the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage bin is obtained according to the historical state of the garbage bin. Therefore, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage can is determined according to the historical state of the garbage can, so that false alarm caused by shielding of the garbage can be reduced, and further the workload of cleaning workers is reduced.
According to one embodiment of the invention, acquiring the current state of the trash can according to the historical state of the trash can comprises the following steps: acquiring the historical state of the garbage can; if the garbage bin has a history state, updating the current state of the garbage bin to be the history state of the previous frame, and not performing garbage bin overflow alarm reminding; and if the garbage can does not have the historical state, updating the current state of the garbage can to be a non-overflow state, and not carrying out the alarm reminding of the overflow of the garbage can.
According to an embodiment of the invention, the garbage bin overflow state detection method further comprises: and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold value, classifying the garbage can detection frame to obtain the current state of the garbage can.
According to an embodiment of the present invention, classifying the trash can detection frame to obtain the current state of the trash can includes: performing edge expansion treatment on the garbage can detection frame; acquiring a garbage can image in a garbage can detection frame after edge expansion processing; and carrying out state classification on the garbage can image to obtain the current state of the garbage can.
According to an embodiment of the present invention, after classifying the trash can detection box to obtain the current state of the trash can, the method further includes: if the current state of the garbage can is an overflow state, acquiring the historical state of the previous frame of the garbage can; if the previous frame of historical state of the garbage can is a non-overflow state or the garbage can does not have the historical state, performing garbage can overflow alarm reminding; and if the previous frame of historical state of the garbage can is an overflow state, not performing garbage can overflow alarm reminding.
According to one embodiment of the invention, the multi-target detection is carried out on the image to be detected through the target detection model to obtain at least one target detection frame, wherein the TensorRT is adopted to carry out forward reasoning on the target detection model.
According to one embodiment of the invention, the garbage bin detection frame is classified through a garbage bin state classification model to obtain the current state of the garbage bin, wherein the garbage bin state classification model is subjected to forward reasoning by using TensorRT.
According to one embodiment of the invention, the at least one object detection box comprises at least one of a trash can detection box, a pedestrian detection box, a motor vehicle detection box and a non-motor vehicle detection box.
In order to achieve the above object, a second aspect of the present invention provides a device for detecting an overflow condition of a trash can, including: the image acquisition module is used for acquiring an image to be detected; the target detection module is used for carrying out multi-target detection on the image to be detected to obtain at least one target detection frame; the target tracking module is used for carrying out target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame if the at least one target detection frame comprises the garbage bin detection frame; and the state acquisition module is used for acquiring the current state of the garbage can according to the historical state of the garbage can if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value.
According to the garbage bin overflow state detection device provided by the embodiment of the invention, an image to be detected is obtained through the image obtaining module, multi-target detection is carried out on the image to be detected through the target detection module, at least one target detection frame is obtained, if the at least one target detection frame comprises the garbage bin detection frame, target tracking is carried out on each target detection frame in the at least one target detection frame through the target tracking module, at least one target tracking frame is obtained, and if the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage bin is obtained through the state obtaining module according to the historical state of the garbage bin. Therefore, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage can is determined according to the historical state of the garbage can, so that false alarm caused by shielding of the garbage can be reduced, and further the workload of cleaning workers is reduced.
To achieve the above object, a third embodiment of the present invention provides a computer-readable storage medium having a trash can overflow status detecting program stored thereon, where the trash can overflow status detecting program, when executed by a processor, implements the trash can overflow status detecting method according to the first embodiment.
According to the computer-readable storage medium of the embodiment of the invention, by the garbage bin overflow state detection method, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, the current state of the garbage bin is determined according to the historical state of the garbage bin, so that false alarms caused by shielding of the garbage bin can be reduced, and the workload of cleaning workers is further reduced.
To achieve the above object, a fourth aspect of the present invention provides an electronic device, including: the garbage can overflow state detection method comprises the steps of storing a program, storing the program in a memory, storing the program in the memory, and running the program on a processor, wherein when the program is executed by the processor, the garbage can overflow state detection method is realized.
According to the electronic equipment provided by the embodiment of the invention, by the garbage bin overflow state detection method, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage bin is determined according to the historical state of the garbage bin, so that false alarm caused by shielding of the garbage bin can be reduced, and the workload of cleaning workers is further reduced.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a diagram of an application environment of a method for detecting an overflow condition of a trash can according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of detecting an overflow condition of a trash can according to one embodiment of the present invention;
FIG. 3 is a flow chart of a method of detecting an overflow condition of a trash can according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of a garbage bin overflow state detecting device according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The method for detecting the overflow state of the trash can be applied to the application environment shown in fig. 1. The camera is communicated with the processing equipment through a network, acquires an image of the garbage can to obtain an image to be detected and sends the image to the processing equipment, the processing equipment acquires the image to be detected and performs multi-target detection on the image to be detected to obtain at least one target detection frame, and when the at least one target detection frame comprises the garbage can detection frame, performs target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, acquires the current state of the garbage can according to the historical state of the garbage can. It should be noted that the camera can be a high-definition/full-high-definition starlight camera arranged around the trash can, and the camera can be shared with cameras of other services, such as an intelligent security/park camera, so as to save the installation and maintenance cost and realize the sharing of images; the processing device may be a server or an edge computing box, etc.
In some embodiments, as shown in fig. 2, a method for detecting an overflow status of a trash can is provided, which is exemplified by the method applied to the processing device in fig. 1, and may include the following steps:
and S101, acquiring an image to be detected.
Specifically, when the overflow state of the trash can needs to be detected, information such as identification information and threshold information of a camera corresponding to the trash can to be detected may be determined, a configuration file, such as a YAML file, may be generated based on the information, and then the configuration file may be imported into the processing device. And after the processing equipment obtains the configuration file, acquiring an image to be detected from the video stream of the corresponding camera according to the identification information of the camera in the configuration file, and detecting the overflow state of the garbage can.
And S102, carrying out multi-target detection on the image to be detected to obtain at least one target detection frame.
Specifically, after obtaining an image to be detected, the processing device performs multi-target detection on the image to be detected to obtain at least one target detection frame, wherein the multi-target may include at least one of a trash can, a pedestrian, a motor vehicle and a non-motor vehicle, and the corresponding at least one target detection frame may include at least one of a trash can detection frame, a pedestrian detection frame, a motor vehicle detection frame and a non-motor vehicle detection frame. For example, the image to be detected may be subjected to trash can detection, pedestrian detection, motor vehicle detection, and non-motor vehicle detection, and since only part of the trash can, the pedestrian, the motor vehicle, and the non-motor vehicle may exist in the same image to be detected, the obtained target detection frame may only include part of the trash can detection frame, the pedestrian detection frame, the motor vehicle detection frame, and the non-motor vehicle detection frame, and is specifically determined by the detection result.
Optionally, the multi-target detection may be performed on the image to be detected through a target detection model to obtain at least one target detection frame, where the TensorRT is used to perform forward reasoning on the target detection model.
Specifically, an initial target detection model may be established first, and the trained target detection model may be obtained by training the initial target detection model using an image with a trash can, a pedestrian, a motor vehicle, and a non-motor vehicle. Wherein, a State-of-the-art open source detection model YOLOV5 can be used as an initial target detection model for customized training to ensure the detection accuracy and throughput performance of the garbage can, the pedestrian, the motor vehicle and the non-motor vehicle; meanwhile, the TensorRT can be adopted to carry out forward reasoning on the operation of reasoning throughput of the optimization model such as operator fusion, kernel function optimization, weight quantization and the like on the target detection model so as to ensure the real-time performance of multi-target detection and the optimal performance under the limited budget. In practical application, the image to be detected is input into the trained target detection model, a plurality of targets existing in the image to be detected can be obtained through the trained target detection model, and a corresponding target detection frame is formed. Wherein, the number and the kind of target detection frame are according to actual waiting to detect the image and confirm, that is to say, the number and the kind of target detection frame can include a plurality ofly, and every kind of target detection frame also can include a plurality of target detection frames, for example when two pedestrians appear in waiting to detect the image, can generate two pedestrian detection frames respectively according to two pedestrians.
Step S103, if the at least one target detection frame comprises a garbage bin detection frame, performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame.
Specifically, when the processing device detects that a trash can exists in an image to be detected, that is, at least one target detection frame includes a trash can detection frame, target Tracking can be performed on each target detection frame by using a single on line And real Tracking (SORT) algorithm to obtain a target Tracking frame corresponding to each target detection frame, for example, the trash can detection frame corresponds to the trash can Tracking frame, the pedestrian detection frame corresponds to the pedestrian Tracking frame, the motor vehicle detection frame corresponds to the motor vehicle Tracking frame, And the non-motor vehicle detection frame corresponds to the non-motor vehicle Tracking frame.
It should be noted that the multi-target tracking algorithm is an algorithm for performing online real-time tracking on a detected target by using IoU (Intersection over Union), kalman filter, hungarian algorithm, and the like, and each obtained target detection frame is tracked by the multi-target tracking algorithm, so that at least one of a trash can tracking frame, a pedestrian tracking frame, a motor vehicle tracking frame, and a non-motor vehicle tracking frame can be formed, and each target tracking frame may have more than one target tracking frame, for example, two pedestrian tracking frames and three non-motor vehicle tracking frames can be formed according to an image to be detected.
And step S104, if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, acquiring the current state of the garbage can according to the historical state of the garbage can.
Specifically, after the processing equipment obtains at least one target tracking frame, whether the area of the garbage can tracking frame is overlapped with that of other target tracking frames or not is judged, and if the area of the garbage can tracking frame is overlapped with that of other target tracking frames, the current state of the garbage can is obtained according to the historical state of the garbage can. The overlapping degree is the percentage of the overlapping area of the garbage can tracking frame and all other target tracking frames in the area of the garbage can tracking frame, and the preset threshold value can be set according to the actual situation, for example, can be set to 10%.
Further, in some embodiments, obtaining the current state of the trash can according to the historical state of the trash can includes: acquiring the historical state of the garbage can; if the garbage bin has a history state, updating the current state of the garbage bin to be the history state of the previous frame, and not performing garbage bin overflow alarm reminding; and if the garbage can does not have the historical state, updating the current state of the garbage can to be a non-overflow state, and not carrying out the alarm reminding of the overflow of the garbage can.
Specifically, when the overlapping degree of the garbage can tracking frame and other target tracking frames is larger than a preset threshold value, the processing equipment acquires the historical state of the garbage can, and if the garbage can has the historical state, the current garbage can state is updated to the historical state of the previous frame, and the garbage can overflow alarm reminding is not carried out; if the garbage can has no history state, namely the previous frame of the garbage can has no history data, the current state of the garbage can is a non-overflow state by default, and the garbage can overflow alarm reminding is not carried out. That is to say, when the overlapping degree of the garbage can tracking frame and other target tracking frames is greater than the preset threshold value, only the current state of the garbage can is updated, and the garbage can overflow alarm is not performed, so that the false alarm condition caused by excessive shielding of the garbage can by pedestrians, motor vehicles and non-motor vehicles is prevented.
In the embodiment, the multi-target detection can be performed on the image to be detected, when the garbage can is detected to be detected, the multi-target tracking is performed, and when the area overlapping exists between the garbage can tracking frame and other target tracking frames in the multi-target tracking and the overlapping rate is higher than the preset threshold value, the current state of the garbage can is determined according to the historical data of the garbage can, the garbage can to be shielded is screened out, the current state of the garbage can is determined according to the historical data of the garbage can, and the garbage can is not classified, so that the misclassification and the false alarm probability caused by shielding are effectively reduced, and the accuracy of detecting the overflow state of the garbage can is improved; meanwhile, the garbage can is tracked in real time, the garbage can be guaranteed to be the same garbage can all the time, the reliability of detecting the overflow state of the garbage can be improved, and the pedestrian, the motor vehicle and the non-motor vehicle can be tracked in real time, so that reusable detection results can be provided for other services of a smart park or a smart city, such as forbidden area intrusion, motor vehicle violation, non-motor vehicle violation and the like, and the expandability of a service scene of detecting the overflow state of the garbage can is improved.
In some embodiments, the above method for detecting an overflow condition of a trash can further includes: and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold value, classifying the garbage can detection frame to obtain the current state of the garbage can.
Specifically, the processing device can obtain an image to be detected first, perform multi-target detection on the image to be detected to obtain at least one target detection frame, perform target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame when the at least one target detection frame comprises the trash can detection frame, and then judge whether area overlapping exists between the trash can tracking frame in the at least one target tracking frame and other target tracking frames and the overlapping degree is larger than a preset threshold value, if not, perform classification processing on the trash can detection frames to obtain the current state of the trash can. It should be noted that, if only the trash can tracking frame is in the at least one target tracking frame, the trash can detection frame is also classified, so as to obtain the current state of the trash can. From this, when the garbage bin is not sheltered from or the area that is sheltered from is very little, adopt the classification mode to acquire the current state of garbage bin, guarantee to follow the actual state of garbage bin in real time.
Further, in some embodiments, classifying the trash can detection box to obtain a current state of the trash can includes: performing edge expansion treatment on the garbage can detection frame; acquiring a garbage can image in a garbage can detection frame after edge expansion processing; and carrying out state classification on the garbage can image to obtain the current state of the garbage can.
Specifically, when the processing equipment carries out classification processing on the garbage bin detection frame, the side length of the garbage bin detection frame can be enlarged by a certain multiple, such as one time, and a garbage bin image is extracted from the enlarged garbage bin detection frame, so that the integrity of the garbage bin image can be ensured as much as possible, and then the garbage bin image is subjected to state classification to obtain the current state of the garbage bin, so that the accuracy of classification processing can be improved, the accuracy of garbage bin detection is further improved, and the probability of misjudgment and false alarm is reduced.
Optionally, the garbage can detection frame can be classified through the garbage can state classification model to obtain the current state of the garbage can, and the garbage can state classification model is subjected to forward reasoning through TensorRT.
Specifically, an initial garbage bin state classification model can be established first, and the initial garbage bin state classification model is trained by using an image with an overflow state and a non-overflow state of the garbage bin, so as to obtain a trained garbage bin state classification model. The ResNet18 with balanced performance can be used as an initial garbage can state classification model for training to ensure the accuracy of overflow and non-overflow state judgment and the throughput performance; meanwhile, the TensorRT can be adopted to carry out forward reasoning on the operation of reasoning throughput of the optimization models such as operator fusion, kernel function optimization, weight quantization and the like on the garbage can state classification model so as to ensure the real-time performance of garbage can state classification and the optimal performance under the limited budget. In practical use, the garbage can image is input into the trained garbage can state classification model, and the state of the garbage can is obtained through the trained garbage can state classification model.
Further, in some embodiments, after classifying the trash can detection box to obtain the current state of the trash can, the method further includes: if the current state of the garbage can is an overflow state, acquiring the historical state of the previous frame of the garbage can; if the previous frame of historical state of the garbage can is a non-overflow state or the garbage can does not have the historical state, performing garbage can overflow alarm reminding; and if the previous frame of historical state of the garbage can is an overflow state, not performing garbage can overflow alarm reminding.
Specifically, after the processing equipment classifies the garbage bin detection frame to obtain the current state of the garbage bin, the current state of the garbage can is also judged, if the current state of the garbage can is the overflow state, the historical state of the previous frame of the garbage can is firstly obtained, if the history status of the previous frame of the trash can is a non-overflow status or the trash can does not have a history status, the garbage can is changed from a non-overflow state to an overflow state, at the moment, the garbage can overflow alarm reminding is carried out, if the previous frame of the historical state of the garbage can is an overflow state, the garbage can overflow alarm reminding is already carried out, at the moment, the garbage can overflow alarm reminding is not carried out any more, thereby can avoid the garbage bin to overflow the state and continuously report to the police, reduce and can not play supplementary control personnel's effect because of continuously reporting to the police, can increase the condition emergence of control personnel burden on the contrary.
As a specific example, referring to fig. 3, the garbage bin overflow state detecting method may include the steps of:
step S201, an image to be detected is acquired.
Specifically, the image to be detected can be obtained from the video stream of the camera corresponding to the trash can to be detected.
And S202, carrying out multi-target detection on the image to be detected to obtain at least one target detection frame.
Specifically, after the image to be detected is obtained, the target detection model obtained through pre-training can be used for performing target detection on the image to be detected, such as garbage cans, pedestrians, motor vehicles, non-motor vehicles and the like, so as to obtain at least one target detection frame, such as one or more of the garbage can detection frame, the pedestrian detection frame, the motor vehicle detection frame and the non-motor vehicle detection frame.
Step S203, determine whether there is a trash can detection frame. If so, step S204 is performed, otherwise step S217 is performed.
And step S204, carrying out target tracking on each target detection frame to obtain a corresponding target tracking frame.
Specifically, each target detection frame can be subjected to target tracking by using a multi-target tracking algorithm, so that one-to-one corresponding target tracking frames, such as a trash can tracking frame, a pedestrian tracking frame and the like, are formed.
Step S205, determine whether the other target tracking frames overlap with the trash can tracking frame and the overlapping degree is greater than a preset threshold. If so, step S206 is performed, otherwise step S210 is performed.
Step S206, judging whether the garbage can has a history state. If so, step S207 is performed, otherwise step S208 is performed.
Step S207, updating the current state of the trash can to the previous frame history state. Namely, when the garbage can has a history state, the current garbage can state is updated to the history state of the previous frame.
And step S208, updating the current state of the garbage bin to be a non-overflow state. Namely, the garbage can has no history state, the current state of the garbage can is updated to be a non-overflow state.
And step S209, the garbage bin overflow alarm reminding is not carried out. That is to say, when the overlapping degree of the garbage can tracking frame and other target tracking frames is greater than the preset threshold value, only the current state of the garbage can is updated, and the garbage can overflow alarm is not performed, so that the false alarm condition caused by excessive shielding of the garbage can by pedestrians, motor vehicles and non-motor vehicles is prevented.
Step S210, performing edge expansion processing on the garbage can detection frame, and acquiring a garbage can image in the garbage can detection frame after the edge expansion processing.
Specifically, when the area overlapping between the other target tracking frames and the trash can tracking frame is not generated or the area overlapping degree is smaller than or equal to a preset threshold value, the edges of the trash can detection frame are expanded, and a trash can image is extracted from the expanded trash can detection frame.
Step S211, performing state classification on the garbage can images to obtain the current state of the garbage can.
Specifically, the garbage can image can be input into a garbage can state classification model obtained through pre-training to obtain a current state of the garbage can, such as an overflow state or a non-overflow state.
In step S212, it is determined whether the trash can is in an overflow state. If so, step S213 is executed, otherwise, step S208 is returned to.
In step S213, the current state of the trash can is updated to be the overflow state.
In step S214, it is determined whether there is a previous frame history status and an overflow status. If so, return to step S209, otherwise execute step S215.
And step S215, carrying out garbage bin overflow alarm reminding.
Specifically, if the garbage can is in an overflow state currently, and the previous frame history state of the garbage can is in a non-overflow state or the garbage can has no history state, the garbage can is changed from the initial non-overflow state to the overflow state, so as to perform alarm reminding of the overflow of the garbage can,
in step S216, the garbage can status of the current frame is updated to a history status. The current trash can state is saved and used as the historical state of the previous frame for the next use.
Step S217, waits for the next frame of decoded data.
Therefore, when the overlapping degree of the garbage can tracking frame and other target tracking frames is larger than a preset threshold value, only the current state of the garbage can is updated, and the garbage can overflow alarm is not performed, so that the false alarm condition caused by excessive shielding of the garbage can by pedestrians, motor vehicles and non-motor vehicles is prevented; meanwhile, when the overlapping degree of the garbage can tracking frame and other target tracking frames is smaller than or equal to a preset threshold value, the garbage can overflow alarm reminding can be carried out only when the garbage can is changed from a non-overflow state to an overflow state, and if the previous frame of historical state of the garbage can is the overflow state, the garbage can overflow alarm reminding is not carried out, so that the problem of continuous alarm of the garbage can in the overflow state is solved.
In summary, according to the method for detecting the overflow state of the trash can in the embodiment of the present invention, an image to be detected is obtained, multi-target detection is performed on the image to be detected, so as to obtain at least one target detection frame, if the at least one target detection frame includes the trash can detection frame, target tracking is performed on each target detection frame in the at least one target detection frame, so as to obtain at least one target tracking frame, and if the overlapping degree between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is obtained according to the historical state of the trash can. Therefore, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage can is determined according to the historical state of the garbage can, so that false alarm caused by shielding of the garbage can be reduced, and further the workload of cleaning workers is reduced.
Fig. 4 is a schematic structural diagram of a garbage bin overflow state detecting device according to an embodiment of the present invention. As shown in fig. 4, the apparatus 100 for detecting an overflow state of a trash can includes: an image acquisition module 110, a target detection module 120, a target tracking module 130, and a status acquisition module 140.
The image obtaining module 110 is configured to obtain an image to be detected; the target detection module 120 is configured to perform multi-target detection on an image to be detected to obtain at least one target detection frame; the target tracking module 130 is configured to perform target tracking on each target detection frame of the at least one target detection frame to obtain at least one target tracking frame if the at least one target detection frame includes the trash can detection frame; the state obtaining module 140 is configured to obtain a current state of the trash can according to a historical state of the trash can if an overlapping degree of the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold.
In some embodiments, the state acquisition module 140 is specifically configured to: acquiring the historical state of the garbage can; if the garbage bin has a history state, updating the current state of the garbage bin to be the history state of the previous frame, and not performing garbage bin overflow alarm reminding; and if the garbage can does not have the historical state, updating the current state of the garbage can to be a non-overflow state, and not carrying out the alarm reminding of the overflow of the garbage can.
In some embodiments, the status acquisition module 140 is further configured to: and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold value, classifying the garbage can detection frame to obtain the current state of the garbage can.
In some embodiments, the status acquisition module 140 is further configured to: performing edge expansion treatment on the garbage can detection frame; acquiring a garbage can image in a garbage can detection frame after edge expansion processing; and carrying out state classification on the garbage can image to obtain the current state of the garbage can.
In some embodiments, the status acquisition module 140 is further configured to: after the garbage can detection frame is classified to obtain the current state of the garbage can, if the current state of the garbage can is an overflow state, the historical state of the previous frame of the garbage can is obtained; if the previous frame of historical state of the garbage can is a non-overflow state or the garbage can does not have the historical state, performing garbage can overflow alarm reminding; and if the previous frame of historical state of the garbage can is an overflow state, not performing garbage can overflow alarm reminding.
In some embodiments, the target detection module 120 is specifically configured to: and performing multi-target detection on the image to be detected through the target detection model to obtain at least one target detection frame, wherein the TensorRT is adopted to perform forward reasoning on the target detection model.
In some embodiments, the state acquisition module 140 is specifically configured to: and classifying the garbage bin detection frame through the garbage bin state classification model to obtain the current state of the garbage bin, wherein the garbage bin state classification model is subjected to forward reasoning by adopting TensorRT.
In some embodiments, the at least one object detection box comprises at least one of a trash can detection box, a pedestrian detection box, a motor vehicle detection box, and a non-motor vehicle detection box.
It should be noted that, for the description of the device for detecting the overflow state of the trash can in the present application, please refer to the description of the method for detecting the overflow state of the trash can in the present application, and detailed description thereof is omitted here.
According to the garbage bin overflow state detection device provided by the embodiment of the invention, an image to be detected is obtained through the image obtaining module, multi-target detection is carried out on the image to be detected through the target detection module, at least one target detection frame is obtained, if the at least one target detection frame comprises the garbage bin detection frame, target tracking is carried out on each target detection frame in the at least one target detection frame through the target tracking module, at least one target tracking frame is obtained, and if the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage bin is obtained through the state obtaining module according to the historical state of the garbage bin. Therefore, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage can is determined according to the historical state of the garbage can, so that false alarm caused by shielding of the garbage can be reduced, and further the workload of cleaning workers is reduced.
Embodiments of the present invention also provide a computer-readable storage medium having stored thereon a trash can overflow state detecting program, which when executed by a processor, implements the trash can overflow state detecting method as described above.
According to the computer-readable storage medium of the embodiment of the invention, by the garbage bin overflow state detection method, at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, the current state of the garbage bin is determined according to the historical state of the garbage bin, so that false alarms caused by shielding of the garbage bin can be reduced, and the workload of cleaning workers is further reduced.
An embodiment of the present invention further provides an electronic device, including: the garbage can overflow state detection method comprises a memory, a processor and a garbage can overflow state detection program which is stored in the memory and can run on the processor, wherein when the processor executes the program, the garbage can overflow state detection method is realized.
According to the electronic equipment provided by the embodiment of the invention, by the garbage bin overflow state detection method, the at least one target tracking frame is obtained through multi-target tracking, and when the overlapping degree of the garbage bin tracking frame in the at least one target tracking frame and other target tracking frames is larger than a preset threshold value, the current state of the garbage bin is determined according to the historical state of the garbage bin, so that false alarm caused by shielding of the garbage bin can be reduced, and the workload of cleaning workers is further reduced.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (11)
1. A method for detecting an overflow condition of a trash can, the method comprising:
acquiring an image to be detected;
performing multi-target detection on the image to be detected to obtain at least one target detection frame;
if the at least one target detection frame comprises a garbage bin detection frame, performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame;
and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value, acquiring the current state of the garbage can according to the historical state of the garbage can.
2. The method for detecting the overflow status of the trash can according to claim 1, wherein the obtaining the current status of the trash can according to the historical status of the trash can comprises:
acquiring the historical state of the garbage can;
if the garbage can has a history state, updating the current state of the garbage can to be the history state of the previous frame, and not performing garbage can overflow alarm reminding;
and if the garbage can does not have the historical state, updating the current state of the garbage can to be a non-overflow state, and not carrying out the alarm reminding of the overflow of the garbage can.
3. The method of detecting a trash can overflow condition of claim 1, further comprising:
and if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and the other target tracking frames is less than or equal to the preset threshold, classifying the garbage can detection frame to obtain the current state of the garbage can.
4. The method for detecting the overflow status of the trash can according to claim 3, wherein the classifying the trash can detection frame to obtain the current status of the trash can comprises:
performing edge expansion treatment on the garbage can detection frame;
acquiring a garbage can image in a garbage can detection frame after edge expansion processing;
and carrying out state classification on the garbage can image to obtain the current state of the garbage can.
5. The method for detecting the overflow status of the trash can according to claim 3, wherein after the sorting of the trash can detection frame to obtain the current status of the trash can, the method further comprises:
if the current state of the garbage can is an overflow state, acquiring the historical state of the previous frame of the garbage can;
if the previous frame of historical state of the garbage can is a non-overflow state or the garbage can does not have the historical state, alarming and reminding the overflow of the garbage can;
and if the previous frame of historical state of the garbage can is an overflow state, not performing garbage can overflow alarm reminding.
6. The method for detecting the overflow state of the trash can as claimed in claim 1, wherein the image to be detected is subject to multi-target detection through a target detection model to obtain at least one target detection box, wherein the target detection model is subject to forward reasoning by using TensorRT.
7. The method for detecting the overflow state of the trash can according to claim 3, wherein the current state of the trash can is obtained by classifying the trash can detection frame through a trash can state classification model, and the trash can state classification model is subjected to forward reasoning by using TensorRT.
8. The method of any of claims 1-7, wherein the at least one object detection box comprises at least one of a trash bin detection box, a pedestrian detection box, a motor vehicle detection box, and a non-motor vehicle detection box.
9. A trash can overflow condition detection device, the device comprising:
the image acquisition module is used for acquiring an image to be detected;
the target detection module is used for carrying out multi-target detection on the image to be detected to obtain at least one target detection frame;
the target tracking module is used for carrying out target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame if the at least one target detection frame comprises the garbage bin detection frame;
and the state acquisition module is used for acquiring the current state of the garbage can according to the historical state of the garbage can if the overlapping degree of the garbage can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold value.
10. A computer-readable storage medium, having stored thereon a trash can overflow state detecting program that, when executed by a processor, implements the trash can overflow state detecting method according to any one of claims 1-8.
11. An electronic device, comprising: a memory, a processor and a trash can overflow status detection program stored on the memory and operable on the processor, the processor implementing the trash can overflow status detection method according to any one of claims 1-8 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110996174.8A CN113642509A (en) | 2021-08-27 | 2021-08-27 | Garbage bin overflow state detection method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110996174.8A CN113642509A (en) | 2021-08-27 | 2021-08-27 | Garbage bin overflow state detection method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113642509A true CN113642509A (en) | 2021-11-12 |
Family
ID=78424282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110996174.8A Pending CN113642509A (en) | 2021-08-27 | 2021-08-27 | Garbage bin overflow state detection method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113642509A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114155467A (en) * | 2021-12-02 | 2022-03-08 | 上海皓维电子股份有限公司 | Garbage can overflow detection method and device and electronic equipment |
-
2021
- 2021-08-27 CN CN202110996174.8A patent/CN113642509A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114155467A (en) * | 2021-12-02 | 2022-03-08 | 上海皓维电子股份有限公司 | Garbage can overflow detection method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Aboah | A vision-based system for traffic anomaly detection using deep learning and decision trees | |
| WO2022105243A1 (en) | Event detection method, apparatus, electronic device, and storage medium | |
| US9460361B2 (en) | Foreground analysis based on tracking information | |
| US11145046B2 (en) | Detection of near-field occlusions in images | |
| CN111629181B (en) | Fire-fighting life passage monitoring system and method | |
| CN109766867B (en) | Vehicle running state determination method and device, computer equipment and storage medium | |
| CN109360362A (en) | A kind of railway video monitoring recognition methods, system and computer-readable medium | |
| CN112883921A (en) | Garbage can overflow detection model training method and garbage can overflow detection method | |
| Doshi et al. | An efficient approach for anomaly detection in traffic videos | |
| CN112560546A (en) | Throwing behavior detection method and device and storage medium | |
| CN112487877A (en) | Monitoring method, system, device and medium for standard operation of kitchen waste | |
| CN113642509A (en) | Garbage bin overflow state detection method and device, storage medium and electronic equipment | |
| JP7125843B2 (en) | Fault detection system | |
| US11532158B2 (en) | Methods and systems for customized image and video analysis | |
| US20200394802A1 (en) | Real-time object detection method for multiple camera images using frame segmentation and intelligent detection pool | |
| CN115294169A (en) | Vehicle tracking method and device, electronic equipment and storage medium | |
| CN114049771A (en) | Bimodal-based traffic anomaly detection method and system and storage medium | |
| TWI706381B (en) | Method and system for detecting image object | |
| CN114511978A (en) | Intrusion early warning method and device, vehicle and computer readable storage medium | |
| CN110581979B (en) | Image acquisition system, method and device | |
| JP6978986B2 (en) | Warning system, warning control device and warning method | |
| JP6678706B2 (en) | Type determination program, type determination device and type determination method | |
| CN112686136A (en) | Object detection method, device and system | |
| US20240046647A1 (en) | Method and device for detecting obstacles, and computer storage medium | |
| CN111008544B (en) | Traffic monitoring and unmanned auxiliary system and target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |