CN112257489A - Automatic detection system and method for garbage bag throwing - Google Patents
Automatic detection system and method for garbage bag throwing Download PDFInfo
- Publication number
- CN112257489A CN112257489A CN202010664561.7A CN202010664561A CN112257489A CN 112257489 A CN112257489 A CN 112257489A CN 202010664561 A CN202010664561 A CN 202010664561A CN 112257489 A CN112257489 A CN 112257489A
- Authority
- CN
- China
- Prior art keywords
- target
- tracking
- video
- garbage
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Abstract
The invention discloses an automatic detection system and method for throwing garbage bags, which are based on computer vision target detection and target tracking and have the function of automatically identifying whether wet garbage thrown by a user is correct or not. When the violation of putting the wet garbage by the user is detected, the system automatically records the violation, and then transmits the video to the server through the network to prepare for subsequent verification. The automatic detection system and the method for garbage bag throwing in the invention acquire external video data by means of the camera sensor of the automatic detection system, and then automatically judge whether the behavior of throwing garbage by a user is correct or not based on the machine learning vision system, and have the following advantages: (1) the method is full-automatic, does not need human intervention, automatically judges, automatically records the screen, and automatically uploads; (2) the detection accuracy is high and is more than 90 percent; (3) convenient deployment and easy large-scale expansion.
Description
Technical Field
The invention belongs to the technical field of intelligent garbage classification equipment, and particularly relates to an automatic garbage bag throwing detection system and method.
Background
Along with the importance and investment of the state on garbage classification, more and more intelligent garbage classification machine systems appear in daily life of people to facilitate the garbage classification of people. Due to the particularity of wet garbage (kitchen garbage), when people throw the wet garbage, people need to pour the wet garbage into the wet garbage can from the garbage bag, and put the garbage bag into the dry garbage can.
In order to prevent people from putting wet garbage and an external garbage bag into a wet garbage can, a conventional method is to manually monitor the behavior of a user, but the method has high labor cost and low cost performance. There is no mature automatic detection system in the existing market.
The problem of putting and detecting of the garbage bags during garbage classification is solved effectively. The invention adopts an automatic detection system and method for garbage bag throwing, which is based on computer vision target detection and target tracking and has the function of automatically identifying whether wet garbage throwing by a user is correct or not. When the violation of putting the wet garbage by the user is detected, the system automatically records the violation, and then transmits the video to the server through the network to prepare for subsequent verification.
Disclosure of Invention
The invention aims to provide a system and a method for automatically judging whether the behavior of throwing garbage by a user is correct, the system and the method are fully automatic, do not need human intervention, automatically judge, automatically record a screen, automatically upload, have high detection accuracy and precision of more than 90 percent, are convenient to deploy and are easy to expand in scale.
The technical scheme of the invention is as follows:
an automatic checkout system is put in to disposal bag is provided with:
the camera input module comprises a main camera and a video acquisition camera, wherein the main camera is vertically opposite to the garbage throwing port and used for action detection and judgment, and the video acquisition camera is used for acquiring the integral throwing action video of a user;
the target detection module comprises a detection module and an azimuth coordinate estimation module, wherein the detection module is used for receiving the action video acquired by the main camera, identifying a target object in the video, then transmitting the acquired information to the azimuth coordinate estimation module, and calculating the position information of the target object;
the target tracking judgment module is used for judging the probability of the same target between two adjacent frames through a tracking algorithm after receiving the target position information of each frame of the video, selecting the same target for tracking to form a tracking history track of the same target, and judging whether the garbage bags are thrown correctly or not through the track;
the information collecting and sending module comprises a signal transmission module and a video transmission module, after a signal is received, the signal transmission module transmits the semaphore to a remote server end through an internet protocol, the signal is sent to the video acquisition camera, the video acquisition camera starts to record a video, and the video is uploaded to a remote server through the video transmission module to be subsequently processed.
Further, the target objects in the video are recognized as hands and garbage bags, image recognition is carried out on the image information by collecting the image information of the thrown garbage bags, and the position information is coordinate values.
Further, the tracking algorithm can pre-judge the target position of the next frame by using a filtering algorithm based on the currently detected target, the filtering parameter is corrected according to the pre-judged position and the actual position of the target of the next frame, the historical coordinate information and the parameter of each frame of the target object can be recorded to ensure the accuracy of target tracking, a counter is mainly used for target tracking judgment within specific time, when the time is longer than the threshold time, the algorithm automatically considers that the original object tracking is invalid, the newly detected target object is newly tracked, and if the algorithm judges that the target is put wrongly, an instruction is immediately sent to the next module.
The invention also comprises an automatic detection method for garbage bag throwing, which comprises the following steps:
step 1, carrying out detection algorithm training, marking an identified target object as a garbage bag, manually acquiring related pictures for calibration and training, training by using an SSD (solid State disk) detection network as a backbone network, and outputting a model parameter file to an embedded mainboard for model deployment after training is finished;
step 2, deploying an actual algorithm, artificially defining the coordinate positions of the dry trash can and the wet trash can according to the image visual angle input by the camera, starting the target detection module to work, and marking by using a colored rectangular frame when a target object trash bag or hand is detected near the wet trash can;
and 3, when the target objects are detected in the images of the continuous frames, awakening the target tracking judgment module, and judging whether the target objects in the continuous frames are the same object by using a tracking algorithm to obtain a correct delivery result.
Further, the tracking algorithm can pre-judge the target position of the next frame by using a filtering algorithm based on the currently detected target, the filtering parameter is corrected according to the pre-judged position and the actual position of the target of the next frame, and the historical coordinate information and the parameter of each frame of the target object can be recorded to ensure the accuracy of target tracking.
In conclusion, the beneficial effects of the invention are as follows: the automatic detection system and the method for garbage bag throwing in the invention acquire external video data by means of the camera sensor of the automatic detection system, and then automatically judge whether the behavior of throwing garbage by a user is correct or not based on the machine learning vision system, and have the following advantages: (1) the automatic control system is fully automatic and does not need human intervention. Automatic judgment, automatic frequency recording and automatic uploading; (2) the detection accuracy is high and is more than 90 percent; (3) convenient deployment and easy large-scale expansion.
The present invention will be described in more detail below with reference to examples.
Drawings
Fig. 1 is a block diagram of an automatic detection system for garbage bag feeding according to the present invention.
Fig. 2 is a flow chart of an automatic detection method for garbage bag throwing according to the present invention.
Detailed Description
The invention is further illustrated by the following examples in conjunction with the accompanying drawings.
The detection system is a system based on computer vision target detection and target tracking, and has the function of automatically identifying whether wet garbage thrown by a user is correct or not. When the violation of putting the wet garbage by the user is detected, the system automatically records the violation, and then transmits the video to the server through the network to prepare for subsequent verification.
As shown in fig. 1, it is a block diagram of the system of the present invention, and the automatic detecting system for garbage bag throwing of the present invention is provided with: the camera input module comprises a main camera and a video acquisition camera, wherein the main camera is vertically opposite to the garbage throwing port and used for detecting and judging actions, and the video acquisition camera is used for acquiring the integral throwing action video of a user;
the target detection module comprises a detection module and an orientation coordinate estimation module, wherein the detection module is used for receiving the action video acquired by the main camera, identifying a target object (a hand or a garbage bag) in the video, then transmitting the acquired information to the orientation coordinate estimation module, and calculating the position information (coordinate value) of the target object;
the target tracking judgment module is used for judging the probability of the same target between two adjacent frames through a tracking algorithm after receiving the target position information of each frame of the video, selecting the same target for tracking to form a tracking history track of the same target, and judging whether the garbage bags are thrown correctly or not through the track; the tracking algorithm can pre-judge the target position of the next frame by using a filtering algorithm based on the currently detected target, the filtering parameter is corrected according to the pre-judged position and the actual position of the target of the next frame, and the historical coordinate information and the parameter of each frame of the target object can be recorded to ensure the accuracy of target tracking. The counter is mainly used for target tracking judgment within specific time, and when the time is longer than the threshold time, the algorithm automatically considers that the original object tracking is invalid, and the newly detected target object is tracked. If the algorithm judges that the delivery is wrong, an instruction is sent to the next module immediately.
The information collecting and sending module comprises a signal transmission module and a video transmission module, after a signal is received, the signal transmission module transmits the semaphore to a remote server end through an internet protocol, the signal is sent to the video acquisition camera, the video acquisition camera starts to record a video, and the video is uploaded to a remote server through the video transmission module to be subsequently processed.
Fig. 2 is an automatic detection method for garbage bag throwing, the specific flow of the method of the invention comprises:
the detection algorithm training part is firstly carried out, and since the algorithm for identifying the target object is based on machine learning, a data set needs to be prepared in advance to train our model. The target object identified at this time is 1, a garbage bag 2 and a hand. 2000 relevant pictures are collected at the early stage for calibration and training, and an SSD detection network is used as a backbone network for training. After training is completed, outputting the model parameter file to the embedded mainboard for model deployment;
and secondly, carrying out an actual algorithm deployment part, and manually defining the coordinate positions of the dry trash can and the wet trash can according to the image visual angle input by the camera (the area where the wet trash can is located is an area A). The target detection module starts to work, when a target object (a garbage bag or a hand) is detected near the wet garbage can, a colored rectangular frame is used for marking (the garbage bag can be represented by a green frame, and the hand can be represented by a yellow frame and a red frame);
and thirdly, awakening the target tracking judgment module when the target object is detected in the images of the continuous frames. Since multiple persons may deliver the garbage at the same time, the situation of multiple hands and garbage bags may occur, and even the situation that the targets of the hands and the garbage bags disappear, so that the tracking algorithm judges whether the targets in the continuous frames are the same object, and a correct delivery result is obtained. The tracking algorithm can pre-judge the target position of the next frame by using a filtering algorithm based on the currently detected target, the filtering parameter is corrected according to the pre-judged position and the actual position of the target of the next frame, and the historical coordinate information and the parameter of each frame of the target object can be recorded to ensure the accuracy of target tracking. In the initial stage of the target object, the parameters of the filtering algorithm are unstable, and the tracking accuracy is ensured by matching the current frame and the next frame of target object by using an additional Euclidean distance and a data association method of feature extraction. If a garbage bag is detected in the wet garbage belt area, the garbage bag is tracked to form track information based on coordinates. And then judging whether the launching is correct or not through the last disappeared point in the track information. The last disposal bag position information disappears in the dry trash can area, so the algorithm judges correct delivery.
The following is an example of erroneous drop, and the algorithm determines an erroneous drop because the last missing position of the garbage bag trajectory is found to be within the wet garbage drop area.
The algorithm not only detects and tracks the track information of the garbage bags, but also simultaneously detects and tracks the movement track information of the hands. The main reason is that the situation that the identification and tracking of the garbage bags are lost in an actual test occurs, so that the auxiliary judgment is carried out by the track information of the hand. When the garbage bag tracking is lost, if the vanishing point of the tracking track information of the hand appears in the dry garbage throwing area, the algorithm considers that the garbage bag is thrown correctly. Otherwise, the delivery is not in compliance and is wrong.
The algorithm is deployed to a mobile terminal, a trained deep neural network model has certain requirements on computing capacity, and a processor needs to complete a task of target tracking at the same time, so that the processor is selectively deployed at the mobile terminal and can efficiently process the deep learning model inference calculation, such as an embedded neural Network Processor (NPU), and the NPU adopts a data-driven parallel computing architecture to complete the deep learning model inference calculation in real time at the same time of low power consumption. In addition, various deep learning model compression and target tracking optimization methods at the mobile end are required. And finally, the mobile terminal equipment has the functions of receiving the image information of the camera input module and completing information transceiving through the network connection equipment.
The automatic detection system and method for garbage bag throwing in the invention are not limited to the structure of the above embodiment, and various modifications can be made. In conclusion, all modifications that do not depart from the scope of the invention are intended to be within the scope of the invention.
Claims (5)
1. The utility model provides an automatic checkout system is put in to disposal bag which characterized in that is provided with:
the camera input module comprises a main camera and a video acquisition camera, wherein the main camera is vertically opposite to the garbage throwing port and used for action detection and judgment, and the video acquisition camera is used for acquiring the integral throwing action video of a user;
the target detection module comprises a detection module and an azimuth coordinate estimation module, wherein the detection module is used for receiving the action video acquired by the main camera, identifying a target object in the video, then transmitting the acquired information to the azimuth coordinate estimation module, and calculating the position information of the target object;
the target tracking judgment module is used for judging the probability of the same target between two adjacent frames through a tracking algorithm after receiving the target position information of each frame of the video, selecting the same target for tracking to form a tracking history track of the same target, and judging whether the garbage bags are thrown correctly or not through the track;
the information collecting and sending module comprises a signal transmission module and a video transmission module, after a signal is received, the signal transmission module transmits the semaphore to a remote server end through an internet protocol, the signal is sent to the video acquisition camera, the video acquisition camera starts to record a video, and the video is uploaded to a remote server through the video transmission module to be subsequently processed.
2. An automatic garbage bag feeding detection system according to claim 1, characterized in that: the method comprises the steps of identifying the target objects in the video as hands and garbage bags, carrying out image identification on image information by collecting image information of the thrown garbage bags, wherein the position information is coordinate values.
3. An automatic garbage bag feeding detection system according to claim 1, characterized in that: the tracking algorithm is used for prejudging the target position of the next frame by using a filtering algorithm based on the currently detected target, filtering parameters are corrected through the prejudged position and the actual position of the target of the next frame, historical coordinate information and parameters of each frame of the target object can be recorded to ensure the accuracy of target tracking, a counter is mainly used for target tracking judgment within specific time, when the time is longer than threshold time, the algorithm automatically considers that the original object tracking is invalid, the newly detected target object is tracked, and if the algorithm judges that the target object is put wrongly, an instruction is immediately sent to the next module.
4. An automatic detection method for garbage bag throwing is characterized by comprising the following steps:
step 1, carrying out detection algorithm training, marking an identified target object as a garbage bag, manually acquiring related pictures for calibration and training, training by using an SSD (solid State disk) detection network as a backbone network, and outputting a model parameter file to an embedded mainboard for model deployment after training is finished;
step 2, deploying an actual algorithm, artificially defining the coordinate positions of the dry trash can and the wet trash can according to the image visual angle input by the camera, starting the target detection module to work, and marking by using a colored rectangular frame when a target object trash bag or hand is detected near the wet trash can;
and 3, when the target objects are detected in the images of the continuous frames, awakening the target tracking judgment module, and judging whether the target objects in the continuous frames are the same object by using a tracking algorithm to obtain a correct delivery result.
5. The automatic detection method for garbage bag throwing according to claim 4, wherein the tracking algorithm prejudges the target position of the next frame by using a filtering algorithm based on the currently detected target, and corrects the filtering parameters according to the prejudged position and the actual position of the target of the next frame, so that the historical coordinate information and parameters of each frame of the target object can be recorded to ensure the accuracy of target tracking.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010664561.7A CN112257489A (en) | 2020-07-10 | 2020-07-10 | Automatic detection system and method for garbage bag throwing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010664561.7A CN112257489A (en) | 2020-07-10 | 2020-07-10 | Automatic detection system and method for garbage bag throwing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112257489A true CN112257489A (en) | 2021-01-22 |
Family
ID=74223898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010664561.7A Pending CN112257489A (en) | 2020-07-10 | 2020-07-10 | Automatic detection system and method for garbage bag throwing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257489A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114120449A (en) * | 2021-11-29 | 2022-03-01 | 平安国际智慧城市科技股份有限公司 | Image information-based junk placement behavior determination method and related product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170069222A1 (en) * | 2014-04-21 | 2017-03-09 | Winnow Solutions Limited | A system and method for monitoring food waste |
| CN107918765A (en) * | 2017-11-17 | 2018-04-17 | 中国矿业大学 | A kind of Moving target detection and tracing system and its method |
| CN109101944A (en) * | 2018-08-27 | 2018-12-28 | 四创科技有限公司 | A kind of real-time video monitoring algorithm identifying rubbish of jettisoninging into river |
| CN110795999A (en) * | 2019-09-21 | 2020-02-14 | 万翼科技有限公司 | Garbage delivery behavior analysis method and related product |
-
2020
- 2020-07-10 CN CN202010664561.7A patent/CN112257489A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170069222A1 (en) * | 2014-04-21 | 2017-03-09 | Winnow Solutions Limited | A system and method for monitoring food waste |
| CN107918765A (en) * | 2017-11-17 | 2018-04-17 | 中国矿业大学 | A kind of Moving target detection and tracing system and its method |
| CN109101944A (en) * | 2018-08-27 | 2018-12-28 | 四创科技有限公司 | A kind of real-time video monitoring algorithm identifying rubbish of jettisoninging into river |
| CN110795999A (en) * | 2019-09-21 | 2020-02-14 | 万翼科技有限公司 | Garbage delivery behavior analysis method and related product |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114120449A (en) * | 2021-11-29 | 2022-03-01 | 平安国际智慧城市科技股份有限公司 | Image information-based junk placement behavior determination method and related product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108062349B (en) | Video monitoring method and system based on video structured data and deep learning | |
| CN110852219B (en) | Multi-pedestrian cross-camera online tracking system | |
| CN109948582B (en) | Intelligent vehicle reverse running detection method based on tracking trajectory analysis | |
| CN102324024B (en) | Airport passenger recognition and positioning method and system based on target tracking technique | |
| WO2018214909A1 (en) | Target tracking method, target tracking device, and computer storage medium | |
| WO2019127273A1 (en) | Multi-person face detection method, apparatus, server, system, and storage medium | |
| CN105957354A (en) | Road side parking management system based on terrestrial magnetism and camera array linkage and method thereof | |
| CN104050679B (en) | Illegal parking automatic evidence obtaining method | |
| WO2016004673A1 (en) | Intelligent target recognition device, system and method based on cloud service | |
| WO2017092224A1 (en) | Rfid-based gesture recognition method and system | |
| CN113284168A (en) | Target tracking method and device, electronic equipment and storage medium | |
| US11023717B2 (en) | Method, apparatus, device and system for processing commodity identification and storage medium | |
| CN111325048B (en) | Personnel gathering detection method and device | |
| JP2016001447A (en) | Image recognition system, image recognition device, image recognition method and computer program | |
| CN105519102A (en) | Video monitoring method, video monitoring system and computer program product | |
| CN105185118A (en) | Vehicle peccancy detection device and method based on video detection | |
| CN106331636A (en) | Intelligent video monitoring system and method of oil pipelines based on behavioral event triggering | |
| CN106911914A (en) | Infrared thermal imaging animal activity tracing system | |
| CN102917207A (en) | Motion sequence based abnormal motion vision monitoring system | |
| CN108196680A (en) | A kind of robot vision follower method extracted based on characteristics of human body with retrieving | |
| CN111339811A (en) | Image processing method, device, equipment and storage medium | |
| US20230206093A1 (en) | Music recommendation method and apparatus | |
| CN105227918B (en) | A kind of intelligent control method | |
| CN111126529B (en) | Barrel-turning data acquisition method and device, electronic equipment and garbage truck | |
| CN112949439A (en) | Method and system for monitoring invasion of personnel in key area of oil tank truck |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210122 |