CN115424170A - Garbage throwing detection system and method - Google Patents
Garbage throwing detection system and method Download PDFInfo
- Publication number
- CN115424170A CN115424170A CN202211013480.6A CN202211013480A CN115424170A CN 115424170 A CN115424170 A CN 115424170A CN 202211013480 A CN202211013480 A CN 202211013480A CN 115424170 A CN115424170 A CN 115424170A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- image
- foreground block
- frame
- foreground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a garbage throwing detection system and a method, wherein the method comprises the steps of acquiring a video stream in a monitoring area in real time, and extracting a pedestrian track and a foreground block from the video stream; determining a first image frame corresponding to the stable foreground block based on the foreground block; judging whether the foreground block in the first image frame is overlapped with the area where the pedestrian is located; if the images do not coincide with each other, judging whether the distance between the area of the pedestrian and the foreground block in at least one frame of image in a preset time period is smaller than a preset first threshold value; if yes, determining that the pedestrian has a garbage throwing behavior; the method overcomes the defects of the prior art, detects pedestrians and garbage in the monitoring area respectively, and associates the pedestrian tracks in the picture when the garbage is detected, so that personnel throwing the garbage are determined, and subsequent persuasion and punishment are facilitated.
Description
Technical Field
The invention belongs to the technical field of computer vision, relates to a detection system, in particular to a garbage throwing detection system, and also relates to a detection method, in particular to a garbage throwing detection system.
Background
With the rapid development of the urban civilization construction, higher requirements are put forward for the urban environment, the garbage throwing is an important factor influencing the urban environment, but the effective management of the garbage throwing is a headache problem, if the garbage throwing behavior can be monitored, the follow-up persuasion and penalty according to evidence have very important significance for improving the urban environment, and with the improvement of computer graphic processing means, the garbage throwing behavior identified by a machine is the trend of the future.
At present, there are various schemes for identifying a garbage throwing behavior based on a machine, for example, a chinese patent with application number 202111663196.9 entitled "a scenic spot throwing object video identification system and detection method" provides a throwing object identification scheme, in which a distribution result of throwing object pixel values is clustered through a Kemeans algorithm, a plurality of pixel value clustering results are obtained through screening and filtering, and a suspected throwing object in the pixel value clustering results is identified to determine the throwing object.
This scheme is though whether to have the throwing thing in can discerning the monitoring area, does not abandon the action of throwing the thing to the pedestrian and traces, also can not carry out other operations, for example advises and punishs etc. to the action of throwing the thing to the realization is stopped to the action of throwing rubbish.
Another feasible method is to recognize the suspicious posture of the human body to realize the recognition of the garbage throwing behavior, firstly, the main body of the garbage throwing, namely the pedestrian, is positioned, and then the garbage throwing posture and the garbage in the hand are verified.
The method needs to position the human body firstly, then position key points of the human body, and then perform posture analysis and auxiliary verification of objects in hands, but the method has large limitation, high requirement on judgment calculation of behaviors, and easy confusion and misjudgment with normal actions of urban residents.
Disclosure of Invention
The invention aims to provide a garbage throwing detection system and method, which overcome the limitation of the prior art on garbage throwing behavior detection, respectively track and detect pedestrians and garbage in a monitoring area, and correlate the pedestrian tracks in a picture when the garbage is detected, so that personnel for garbage throwing are determined, and subsequent persuasion and punishment are facilitated.
According to a first aspect of the present invention there is provided a first waste disposal detection system according to the first aspect of the present invention comprising:
the video image acquisition unit is used for acquiring real-time image data of a monitored area;
a network transmission unit for network-transmitting the image data;
the foreground block detection unit is used for carrying out foreground analysis on the image data and determining a stable foreground block area in the image data;
the pedestrian detection unit is used for analyzing and processing the image data and determining the position of a pedestrian;
the target tracking unit is used for analyzing the image with the pedestrian, determining the position track of the pedestrian and acquiring a track chain of the pedestrian;
and the central processing unit is used for performing contact analysis on the track chain of the pedestrian and the position of the foreground block, determining whether the pedestrian belongs to the garbage throwing of the pedestrian or not and acquiring the whole process belonging to the garbage throwing.
According to the first junk throwing detection system of the first aspect of the present invention, there is provided the second junk throwing detection system of the first aspect of the present invention, wherein the video image acquisition unit comprises a plurality of network cameras and a data stream processing module, the plurality of network cameras are installed at different positions in the cell, and the data stream processing module is configured to acquire video data streams of the network cameras and decode the video data streams into image data by using an H264 decoding technology.
According to the first or second refuse throwing detection system of the first aspect of the present invention, there is provided a third refuse throwing detection system according to the first aspect of the present invention, the foreground block detection unit includes a background detection module and a foreground block analysis module;
the background detection module is used for modeling and analyzing the background of the image data, performing frame extraction processing according to the image data, then performing cutting operation, performing graying preprocessing, performing Gaussian mixture modeling operation on the preprocessed image to obtain a foreground video image, and then performing secondary modeling to obtain a background video image;
the foreground block analysis module is used for determining a foreground block, the foreground block analysis module firstly carries out corrosion-first and then expansion treatment on a foreground video image according to a morphological processing method, then carries out expansion-first and then corrosion treatment, fuses foreground pixels in a certain range, carries out communication region detection on all background targets to obtain a communication block, screens the communication block and selects a stable foreground block as a target foreground block.
According to the first to third garbage throwing detection systems of the first aspect of the present invention, there is provided a fourth garbage throwing detection system of the first aspect of the present invention, wherein the graying pretreatment specifically comprises: calculating a graying weighting coefficient of the image based on a gull algorithm; and carrying out graying processing on the image by using the image graying weighting coefficient to obtain a grayscale image.
According to any one of the first to fourth refuse throwing detection systems of the first aspect of the present invention, there is provided a fifth refuse throwing detection system according to the first aspect of the present invention, wherein the contour features of the stable foreground block include: the outline area of the foreground object is more than 50, the area of the region frame is more than 100, and the duty ratio is more than 0.3.
According to the refuse throwing detection system of any one of the first to fifth aspects of the present invention, there is provided the sixth refuse throwing detection system of the first aspect of the present invention, wherein the specific steps of analyzing and processing the image data in the pedestrian detection unit include:
(1) Adjusting image data Resize to 416 × 416 — 1280 × 720;
(2) Separating and normalizing the image data channels, separating the BGR channels, and normalizing the pixel values to 0-1;
(3) Sending the processed image into a pre-trained deep convolutional neural network, extracting the output in the deep convolutional neural network as image characteristics, and generating a characteristic map;
(4) And identifying the characteristic graph and determining a pedestrian position frame.
According to a sixth debris throwing detection system of the first aspect of the present invention, there is provided the seventh debris throwing detection system of the first aspect of the present invention, before the processed image is fed into the pre-trained deep convolutional neural network, comprising:
performing floating point training on the preset convolutional neural network model with the BN layer after the convolutional layer by using preset training data to obtain a first convolutional neural network model;
in the first deep convolutional neural network model, fusing the weight parameters of the convolutional layer and the parameters of the BN layer and updating the convolutional weight to obtain a second convolutional neural network model;
removing the BN layer in the second convolutional neural network model to obtain a third convolutional neural network model;
and performing fixed-point training on the third convolutional neural network model to obtain a fixed-point deep convolutional neural network model.
According to the first to seventh garbage throwing detection system of the first aspect of the present invention, there is provided an eighth garbage throwing detection system of the first aspect of the present invention, wherein the analysis is performed on the image with the pedestrian, the position track of the pedestrian is determined, and the track chain of the pedestrian is obtained, the specific steps include:
(1) Acquiring a central point of a moving object and a cutout of the moving object in moving object detection;
(2) Initializing the environment of a Kalman filtering algorithm in a first frame, and defaulting all moving targets as targets to be tracked;
(3) Then, matching each moving object of each frame with the existing track through Hungarian algorithm, and if the matching is successful, adding the moving objects into the corresponding track; if the matching fails, predicting the position of the track in the current frame through a Kalman filtering algorithm;
(4) If a certain track is not successfully matched with the moving object for a plurality of times, namely the number of track points obtained by continuous prediction through a Kalman filtering algorithm is greater than a specific value A, the track is considered to be finished;
(5) If the number of points of a certain track is larger than the threshold value B, the track is considered to be effective, otherwise, the track is considered to be ineffective.
According to the debris throwing detection system of any one of the first to eighth aspects of the present invention, there is provided the ninth debris throwing detection system according to the first aspect of the present invention, further comprising a video storage unit for buffering video stream data of a motion trajectory judged to belong to debris throwing by the central processing unit, the video storage unit compressing the image data belonging to debris throwing into video stream data by an H264 compression method.
According to a second aspect of the present invention, there is provided a first refuse throwing detection method according to the second aspect of the present invention, comprising: acquiring a video stream in a monitoring area in real time, and extracting pedestrian tracks and foreground block information from the video stream, wherein the foreground block is an area corresponding to garbage in an image frame of the video stream; determining a first image frame corresponding to a stable foreground block based on the foreground block information; judging whether the foreground block in the first image frame is overlapped with the area where the pedestrian is located; if the images do not coincide with each other, judging whether the distance between the area of the pedestrian and the foreground block in at least one frame of image in a preset time period is smaller than a preset first threshold value; and if so, determining that the pedestrian has the garbage throwing behavior.
According to a first spam throwing detection method of a second aspect of the present invention, there is provided a second spam throwing detection method of the second aspect of the present invention, wherein extracting pedestrian trajectories and foreground block information based on the video stream, comprises: identifying pedestrians in each frame image of the video stream, extracting the area where the pedestrians are located in each frame image, and tracking the pedestrians based on the sequence of the image frames in the video stream and the area where the pedestrians are located to obtain the pedestrian track; processing each frame image in the video frame, identifying the region where the junk is located in the processed image, and determining the foreground block information in each frame image based on the region where the junk is located.
According to a second refuse throwing detection method of the second aspect of the present invention, there is provided a third refuse throwing detection method according to the second aspect of the present invention, further comprising: after all pedestrian tracks in the video stream are obtained, judging whether a first pedestrian track exists, wherein the time length between a second image frame corresponding to the last occurrence of a pedestrian in the first pedestrian track and the first image frame is not less than a preset second threshold value; and if so, rejecting the first pedestrian track.
According to the first method for detecting garbage throwing in the second aspect of the present invention, there is provided a fourth method for detecting garbage throwing in the second aspect of the present invention, which determines a first image frame corresponding to a stable foreground block based on the foreground block information, comprising: respectively calculating the variation of the foreground block in each frame of image and the variation of the foreground block in the adjacent front and rear frames of images based on the foreground block information; and determining the image frame corresponding to the first time that the change is smaller than a preset third threshold value, and taking the image frame as the first image frame.
According to the second aspect of the present invention, there is provided a fifth method for detecting garbage throwing, which determines whether a foreground block coincides with a pedestrian in a first image frame, the method including: identifying a region in which one or more pedestrians are located in the first image frame and a foreground block in the first image frame; and respectively calculating the distance between the area where each pedestrian is located and the foreground block, and if the distance between the area where one pedestrian is located and the foreground block is smaller than a preset fourth threshold value, determining that the foreground block in the first image frame is overlapped with the area where the pedestrian is located.
According to the second aspect of the present invention, there is provided a sixth refuse throwing detection method according to the second aspect of the present invention, further comprising: and if the garbage throwing behavior is detected, generating prompt information according to the detection result, and pushing the prompt information, wherein the prompt information prompts a user that the garbage throwing behavior exists.
According to a sixth method for detecting garbage throwing in the second aspect of the present invention, there is provided the seventh method for detecting garbage throwing in the second aspect of the present invention, wherein the prompt message is a voice prompt message, a video prompt message or a text prompt message.
According to a seventh garbage throwing detection method of the second aspect of the present invention, there is provided an eighth garbage throwing detection method of the second aspect of the present invention, wherein if the prompt information is a video prompt information, generating a prompt information according to a detection result includes: extracting the pedestrian area from each image frame of the video stream based on the pedestrian track, combining the pedestrian area and the foreground block in the same image frame to obtain combined image frames, obtaining a rubbish throwing action video stream based on the time sequence of each combined image frame, and taking the rubbish throwing action video stream as the prompt information.
According to an eighth garbage throwing detection method of the second aspect of the present invention, there is provided the ninth garbage throwing detection method of the second aspect of the present invention, before generating the prompt information according to the detection result, the method further includes: judging whether the video stream corresponding to the pedestrian track is pushed or not; if it has already been pushed, the process results.
According to a third aspect of the present invention there is provided a waste disposal detection apparatus according to the third aspect of the present invention, comprising: a processor and a memory; wherein the memory is used for storing instructions executed by the at least one processor; a processor for executing instructions stored in the memory to perform the method of the second aspect.
Compared with the prior art, the invention has the following beneficial effects:
the garbage throwing detection system respectively detects pedestrians and garbage in the monitoring area, and correlates the pedestrian tracks in the picture when the garbage is detected, so that personnel throwing the garbage are determined, and subsequent persuasion and punishment are facilitated.
Drawings
Fig. 1 is a schematic block diagram of a garbage throwing detection system according to an embodiment of the present invention;
FIG. 2 is a block diagram illustrating a schematic flow of a garbage throwing detection method according to an embodiment of the present invention;
fig. 3A is a schematic diagram of image data acquired by a video image acquisition unit according to an embodiment of the present invention;
fig. 3B is a schematic diagram of a foreground block extracted by the foreground block detecting unit according to the embodiment of the present invention
FIG. 4 is a diagram illustrating a result of detecting a pedestrian in a video stream according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart illustrating a process for tracking a pedestrian trajectory according to an embodiment of the present invention;
fig. 6 is a schematic process diagram of a garbage throwing detection method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 illustrates a block diagram of a debris throw detection system according to an embodiment of the present application.
A refuse throwing detection system as shown in fig. 1, the system comprising: video image acquisition unit. The video image capturing unit may be composed of a single video image capturing device with a video capturing function, such as a camera, a video camera, etc., or may be composed of a plurality of video image capturing devices with a video capturing function, for example, the video image capturing unit includes a plurality of webcams, wherein the plurality of webcams are distributively installed at different positions in the cell for capturing video data (or video streams) at different positions in the cell.
For example, in order to reduce the bandwidth resource occupied by the transmitted video data and reduce the size of the video data, the video image capturing unit may perform compression processing on the captured video data, and the compression mode includes, but is not limited to, H264, YUV, and the like.
In addition, if the data acquired by the video image acquisition unit is compressed video data, in order to facilitate the subsequent analysis and detection of the garbage throwing behavior on the video data, the video image acquisition unit further comprises a data stream processing module, wherein the data stream processing module is used for decoding the compressed video data to obtain original video data, the original video data is composed of one image which is continuous in time, and for example, the data stream processing module decodes the video data acquired by the image acquisition device into image data (or video frames) by adopting an H264 decoding technology.
Returning to fig. 1, as another example, in order to analyze and detect whether a garbage throwing behavior exists in the video data in the monitoring area collected by the image collecting unit, the garbage throwing detection system shown in fig. 1 further includes a garbage throwing detection device, and the garbage throwing detection device is configured to perform functions of analyzing and processing the image data obtained, so as to detect the existing garbage throwing behavior. The garbage throwing detection device is a computer, a server or a terminal device with image processing and analyzing functions.
Also as an example, in order to realize the transmission of image data between the video image capturing unit (image capturing device) and the refuse throwing detection device, the system shown in fig. 1 should further include a network transmission unit. The network transmission unit transmits image data between the image capturing unit (image capturing device) and the refuse drop detection device by, for example, a real time Transport Protocol (RTP). It should be understood that the network transmission unit may also use other protocols to transmit the image data, as long as the image data can be transmitted, and is not limited herein.
Further, after receiving the real-time image data in the monitoring area collected by the video image collecting unit, the garbage throwing detection equipment can process and detect the received image data to determine whether garbage throwing behaviors exist in the monitoring area.
In order to detect the garbage throwing behavior of the pedestrian, not only the garbage needs to be detected, but also the garbage and the pedestrian need to be associated to determine the garbage throwing behavior of the pedestrian.
With reference to fig. 1, as an example, in order to associate the garbage with the pedestrian, after acquiring the image data acquired by the video image acquisition unit, the garbage throwing detection device needs to divide the video data into at least two identical paths of video data, where one path of video data is used for tracking the pedestrian involved therein; and the other path of video data is used for tracking the garbage.
Further to achieve this function, the refuse throwing detection apparatus shown in fig. 1 includes a row foreground block detection unit, a pedestrian detection unit, a target tracking unit, and a central processing unit; the foreground block detection unit is configured to perform foreground block analysis on each frame of image in the received real-time video data (or video stream) and track an image frame corresponding to the determined stable foreground block region (see below for details).
The pedestrian detection unit is used for detecting pedestrians appearing in the image data and determining the positions of the pedestrians; the target tracking unit is used for tracking each pedestrian based on the video stream, determining the position track of the pedestrian and acquiring a track chain of the pedestrian; and the central processing unit is used for performing contact analysis on the track chain of the pedestrian and the position of the foreground block, determining whether the pedestrian belongs to the garbage throwing of the pedestrian, and acquiring the whole process belonging to the garbage throwing.
For the convenience of understanding, the following will briefly describe the process of detecting the garbage throwing action by the garbage throwing detection device by taking fig. 2 as an example.
As shown in fig. 2, at step 201, the debris throwing detection apparatus acquires a video stream in a monitored area in real time from a video image acquisition unit, and extracts a pedestrian track and a foreground block from the video stream.
As an example, in order to extract a foreground block from a video stream, a foreground block detection unit in the spam detection apparatus extracts the foreground block (the region where the spam exists) by using the spam in the image data as the foreground and the remaining part as the background.
As an example, in fig. 1, the foreground block detecting unit includes a background detecting module and a foreground block analyzing module.
The background detection module is used for modeling and analyzing the background of the image data; for example, the background detection module performs frame extraction processing according to image data, then performs cutting operation, performs graying preprocessing, performs gaussian mixture modeling operation on a preprocessed image (gray image) to obtain a foreground video image, and then performs secondary modeling to obtain a background video image; for example, the graying weighting coefficients of the image may be calculated based on the gull algorithm; and carrying out graying processing on the image by utilizing the image graying weighting coefficient to obtain a grayscale image. Further, after obtaining a foreground video image and a background video image, a foreground block analysis module firstly carries out corrosion-first and then expansion treatment on the foreground video image according to a morphological processing method, then carries out expansion-first and then corrosion treatment, fuses foreground pixels in a certain range, carries out communication region detection on all background targets to obtain a communication block, and screens the communication block to extract an effective foreground block; the effective foreground block refers to a foreground block meeting specified requirements, for example, the outline area of the effective foreground block is greater than 50, the area of a region frame is greater than 100, and the duty ratio is greater than 0.3.
It should be understood that the foreground blocks mentioned by the scheme provided by the embodiment of the present invention are all valid foreground blocks (hereinafter collectively referred to as foreground blocks). For another example, as shown in fig. 3A and fig. 3B, wherein fig. 3A is a schematic diagram of image data acquired by a video image acquisition unit according to an embodiment of the present invention; fig. 3B is a schematic diagram of a foreground block extracted by the foreground block detecting unit according to the embodiment of the present invention. Referring to fig. 3B, the portion framed by the rectangular frame is an area corresponding to a foreground block obtained by performing foreground block analysis on the image data shown in fig. 3A by the foreground block detection unit, and the other portion in the drawing is a background.
In addition, since the whole process of garbage throwing may involve multiple frames of images, when foreground block (garbage) extraction is performed, foreground block extraction needs to be performed on all image frames containing garbage, for example, regions corresponding to foreground blocks are marked in each image frame in the form of marking frames.
After the foreground block detection unit extracts the foreground blocks in each image frame, the image frames containing the foreground blocks can be sequenced based on the time information indicated in each image frame to obtain an image sequence, and then the foreground blocks are tracked, namely the garbage is tracked.
It has been described above that in order to detect the throwing behavior of refuse, it is necessary to track the refuse on the one hand and the pedestrian throwing the refuse on the other hand.
Continuing to return to fig. 1, in order to realize the pedestrian tracking, the garbage throwing detection device further comprises a pedestrian detection unit and a target tracking unit.
By way of example, the pedestrian detection unit identifies a pedestrian in each frame image of the video stream; then, the region where the pedestrian is located in each frame image is extracted, for example, the region corresponding to the pedestrian is also marked in the image frame in the form of a marking frame. For example, the human detection unit first adjusts the image frame size, such as adjusting the image frame size to be between 416 × 416 and 1280 × 720; then processing the image frame, such as separating a BGR channel in the image frame, and normalizing the pixel value to 0-1; then inputting the processed image frame into a pre-trained deep convolution neural network, and generating a feature map based on image features output by the deep convolution neural network; and identifying the feature map, determining the position of the pedestrian in the image frame, and marking a marking frame in the image frame based on the position.
For example, as shown in fig. 4, the pedestrian in the image frame is framed in the form of a labeled frame.
Fig. 5 is a schematic flow chart illustrating a process of tracking a pedestrian trajectory according to an embodiment of the present invention.
As another example, as shown in fig. 5, the moving object is a pedestrian.
The target tracking unit first performs matting on a moving target in each image frame in the video stream, for example, based on the foregoing, a pedestrian puts a mark frame on the image frame, and based on the mark frame, a region image corresponding to the pedestrian is scratched out. Because a plurality of moving objects may be involved in the image frames of the video stream, before the moving objects are tracked, all the moving objects in the first frame image in the video stream need to be used as the objects to be tracked, then initialization is carried out based on a Kalman filtering algorithm, then each moving object in each frame image is matched with the existing track through the Hungary algorithm based on moving object matting, and if matching is successful, the moving objects are added into the corresponding tracks; if the matching fails, predicting the position of the track in the current frame through a Kalman filtering algorithm. If a certain track is not successfully matched with the moving object for a plurality of times, namely the number of track points obtained by continuous prediction through a Kalman filtering algorithm is larger than a specific value A, the track is considered invalid; and if the number of points of a certain track is greater than the threshold value B, the track is considered to be effective, the invalid track is discarded to obtain an effective track, and the effective track is taken as a pedestrian track.
Further, after extracting the foreground block and the pedestrian trajectory from the video stream, the garbage throwing detection device continues returning to fig. 2, and the garbage throwing detection device executes step 202 to determine the image frame corresponding to the stable foreground block based on the foreground block information.
The process of throwing the garbage comprises two states, wherein the first state is that the pedestrian throws the garbage in the hand to the ground, and the second state is that the garbage is thrown to the ground.
The process of throwing the garbage from the hand of the pedestrian to the ground is a dynamic process, and the foreground blocks extracted from the corresponding multi-frame images are changed, for example, the positions of the foreground blocks in the two adjacent frames of images are different.
When the garbage is thrown to the ground, the position of the garbage does not change, and the foreground block in the image frame related to the state is a stable foreground block (for example, the position of the foreground block does not change).
When the embodiment of the invention detects whether the garbage throwing action exists, the foreground block needs to be tracked to determine the state of the garbage, and the garbage throwing action is determined by identifying that the garbage is thrown to the ground. I.e. by tracking the foreground blocks to find stable foreground blocks.
For example, after the foreground block detection unit extracts the foreground block in each image frame, the foreground block detection unit may sort the image frames including the foreground block based on the time information indicated in each image frame to obtain an image sequence, sequentially compare the foreground blocks marked in two adjacent image frames in the image sequence, and determine whether the area size corresponding to the foreground block in the two adjacent image frames and the position change of the foreground block in the image frame are within a preset threshold; if not, the foreground block is not stable; if so, the foreground block is a stable foreground block.
Furthermore, the garbage throwing detection equipment needs to associate the pedestrians with the foreground blocks after extracting the pedestrian tracks and stabilizing the image frames corresponding to the foreground blocks, and the pedestrians throwing the garbage objects are determined.
The image frames corresponding to the foreground block stabilization in the video stream corresponding to the monitored area may be more than one, for example, the image frame corresponding to the foreground block stabilization is an image frame in which the size of the area corresponding to the foreground block in two adjacent image frames and the position change of the area in the image frame are within a preset threshold value when the image frame first appears, and may also be any image frame corresponding to the foreground block stabilization. For example, the foreground block detection unit extracts foreground blocks from a fifth frame image, a sixth frame image, a seventh frame image, an eighth frame image, a ninth frame image and a tenth frame image of the video data, compares the foreground blocks in two adjacent frame images, respectively, if the area size and the position change of the foreground block in the seventh frame image and the sixth frame image are not within a preset threshold, and the area size and the position change of the foreground block in the seventh frame image and the eighth frame image are within a preset threshold, the foreground blocks in the eighth frame image, the ninth frame image and the tenth frame image are all stable, and the area size and the position change of the foreground block in the image frame image are within a preset threshold, that is, the foreground blocks in the seventh frame image, the eighth frame image, the ninth frame image and the tenth frame image are all stable, the seventh frame image can be selected as the image frame corresponding to the foreground block when the foreground block is stable, and the eighth frame image, the ninth frame image and the tenth frame image can also be selected as the frame image corresponding to the foreground block.
Next, in step 203, the garbage throwing detection apparatus determines whether the region of the foreground block in the image frame corresponding to the stable foreground block coincides with the region of the pedestrian in the image frame. For example, the garbage throwing detection device identifies an area where one or more pedestrians in an image frame corresponding to a stable foreground block are located and the foreground block in the image frame; and respectively calculating the distance between the area where each pedestrian is located and the foreground block by the image frame, and if the distance between the area where one pedestrian is located and the foreground block is smaller than a preset threshold value, determining that the foreground block in the image frame corresponding to the stable foreground block is overlapped with the area where the pedestrian is located. In addition, the overlapping does not mean that the position of the area where the foreground block is located is the same as the position of the area where the pedestrian is located in the strict sense, but means that the position of the area where the foreground block is located is close to the position of the area where the pedestrian is located (if the position of the area is smaller than a preset threshold).
Further, if the region where the foreground block in the image frame corresponding to the stable foreground block is located does not coincide with the region where the pedestrian is located in the image frame, in step 204, the refuse throwing detection apparatus continues to determine whether there is at least one frame of image in a preset time period, where the distance between the region where the pedestrian is located and the foreground block is smaller than a preset threshold. For example, referring to fig. 1, the refuse drop detection apparatus further comprises a central processing unit. The central processing unit associates each pedestrian track with a stable foreground block, and judges whether the pedestrian track has behavior close to the foreground block within 5 seconds (if the distance between the pedestrian tracks and the stable foreground block is less than a specific value), and if the pedestrian track has the behavior close to the foreground block, the pedestrian track is regarded as suspected garbage throwing behavior; and judging the position of the suspected garbage throwing behavior to see whether the suspected garbage throwing behavior is in the detection area, and if so, determining the suspected garbage throwing behavior is a real garbage throwing behavior.
Further, in order to reduce calculation amount and improve calculation efficiency, the pedestrian tracks which are detected by mistake are removed before whether the garbage throwing action exists is detected. As another example, after the image frame corresponding to the stable foreground block is determined, whether a first pedestrian track exists is also determined, wherein a time length between the image frame corresponding to the last occurrence of the pedestrian in the first pedestrian track and the image frame corresponding to the stable foreground block is not less than a preset threshold; and if so, rejecting the first pedestrian track.
Next, in step 205, if the distance between the region where the pedestrian in the at least one frame of image is located and the foreground block is smaller than the preset threshold in the preset time period, it is determined that the pedestrian has a garbage throwing behavior.
Further, if the garbage throwing detection equipment detects that a garbage throwing action exists, prompt information is generated according to a detection result, and the prompt information is pushed, wherein the prompt information prompts a user that the garbage throwing action exists. The prompt message is, for example, a voice prompt message, a video prompt message or a text prompt message. For example, if the prompt information is video prompt information, the pedestrian region is extracted from each image frame of the video stream based on the pedestrian track, the pedestrian region and the foreground block in the same image frame are combined to obtain combined image frames, a video stream of the refuse throwing action is obtained based on the time sequence of each combined image frame, and the video stream of the refuse throwing action is used as the prompt information.
As another example, before generating the prompt information according to the detection result, the method further includes: judging whether the video stream corresponding to the pedestrian track is pushed or not; if it has already been pushed, the process results.
For another example, if the region of the foreground block in the image frame corresponding to the stable foreground block coincides with the region of the pedestrian in the image frame, the process jumps to step 205 to determine that the pedestrian has a garbage throwing behavior. For another example, if in step 204, the refuse throwing detection apparatus continues to determine that there is no at least one frame of image within a preset time period, and the distance between the area where the pedestrian is located and the foreground block is smaller than a preset threshold, then step 203 is skipped, the refuse throwing detection apparatus determines whether the area where the foreground block is located in the image frame corresponding to the stable foreground block coincides with the area where another pedestrian is located in the image frame, and continues to follow steps 204 and 205 until all pedestrians in the video stream are associated. In addition, the embodiment of the application also provides a completion process of the garbage throwing detection, as shown in fig. 6.
By way of further example, referring to fig. 1, the litter drop detecting device further comprises a video storage unit. The video storage unit is used for caching the video stream data of the motion track judged by the central processing unit to belong to the garbage throwing, and the video storage unit compresses the image data belonging to the garbage throwing into the video stream data by an H264 compression method.
According to the embodiment provided by the invention, pedestrians and foreground blocks (regions where garbage is located) are tracked respectively for video streams in a monitoring region, the foreground blocks are associated with the pedestrians, and the pedestrian and garbage throwing behaviors of garbage throwing are determined by judging whether the positions of the foreground blocks of one frame of image and the pedestrians are close (smaller than a preset threshold value) in the image frames corresponding to the track sequence of the pedestrians and the foreground blocks of the image, so that the pedestrians are persuaded according to the garbage throwing behaviors, and the garbage throwing phenomenon is reduced.
It is noted that for the sake of brevity, this application describes some methods and embodiments thereof as a series of acts and combinations thereof, but those skilled in the art will appreciate that the aspects of the application are not limited by the order of the acts described. Accordingly, one of ordinary skill in the art will appreciate, in light of the disclosure or teachings herein, that certain steps may be performed in other sequences or concurrently. Further, those skilled in the art will appreciate that the embodiments described herein are capable of alternative embodiments, i.e., acts or modules referred to herein are not necessarily required for the implementation of the solution or solutions described herein. In addition, the description of some embodiments of the present application is also focused on different schemes. In view of the above, those skilled in the art will understand that portions that are not described in detail in one embodiment of the present application may also be referred to in the related description of other embodiments.
In particular implementation, based on the disclosure and teachings of the present application, one skilled in the art will appreciate that the several embodiments disclosed in the present application may be implemented in other ways not disclosed herein. For example, as for the units in the foregoing embodiments of the electronic device or apparatus, the units are split based on the logic function, and there may be another splitting manner in the actual implementation. Also for example, multiple units or components may be combined or integrated with another system or some features or functions in a unit or component may be selectively disabled. The connections discussed above in connection with the figures may be direct or indirect couplings between the units or components in terms of connectivity between the different units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, where the communication interface may support electrical, optical, acoustic, magnetic, or other forms of signal transmission.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A garbage throwing detection method is characterized by comprising the following steps:
acquiring a video stream in a monitoring area in real time, and extracting pedestrian tracks and foreground block information from the video stream, wherein the foreground block is an area corresponding to garbage in an image frame of the video stream;
determining a first image frame corresponding to a stable foreground block based on the foreground block information;
judging whether the foreground block in the first image frame is overlapped with the area where the pedestrian is located;
if the images do not coincide with each other, judging whether the distance between the area of the pedestrian and the foreground block in at least one frame of image in a preset time period is smaller than a preset first threshold value;
and if so, determining that the pedestrian has the garbage throwing behavior.
2. The method of claim 1, wherein extracting pedestrian trajectories and foreground block information based on the video stream comprises:
identifying pedestrians in each frame image of the video stream, extracting the area where the pedestrians are located in each frame image, and tracking the pedestrians based on the sequence of the image frames in the video stream and the area where the pedestrians are located to obtain the pedestrian track;
processing each frame image in the video frame, identifying the region where the junk is located in the processed image, and determining the foreground block information in each frame image based on the region where the junk is located.
3. The method for detecting refuse throwing according to claim 2, further comprising:
after all pedestrian tracks in the video stream are obtained, judging whether a first pedestrian track exists, wherein the time length between a second image frame corresponding to the last occurrence of a pedestrian in the first pedestrian track and the first image frame is not less than a preset second threshold value;
and if so, rejecting the first pedestrian track.
4. The method as claimed in claim 3, wherein determining the first image frame corresponding to the stable foreground block based on the foreground block information comprises:
respectively calculating the variation of the foreground block in each frame of image and the variation of the foreground block in the adjacent front and rear frames of images based on the foreground block information;
and determining the image frame corresponding to the first time that the change is smaller than a preset third threshold value, and taking the image frame as the first image frame.
5. The method as claimed in claim 4, wherein the step of determining whether the foreground block coincides with the pedestrian in the first image frame comprises:
identifying a region in which one or more pedestrians are located in the first image frame and a foreground block in the first image frame;
and respectively calculating the distance between the area where each pedestrian is located and the foreground block, and if the distance between the area where one pedestrian is located and the foreground block is smaller than a preset fourth threshold value, determining that the foreground block in the first image frame is overlapped with the area where the pedestrian is located.
6. The method for detecting refuse throwing according to claim 5, further comprising:
and if the garbage throwing behavior is detected, generating prompt information according to the detection result, and pushing the prompt information, wherein the prompt information prompts a user that the garbage throwing behavior exists.
7. The method of claim 6, wherein the prompt message is a voice prompt message, a video prompt message or a text prompt message.
8. The method for detecting the garbage throwing according to claim 7, wherein if the prompt message is a video prompt message, generating the prompt message according to the detection result comprises:
and extracting the pedestrian region from each image frame of the video stream based on the pedestrian track, combining the pedestrian region and the foreground block in the same image frame to obtain a combined image frame, obtaining a garbage throwing behavior video stream based on the time sequence of each combined image frame, and taking the garbage throwing behavior video stream as the prompt information.
9. The method for detecting garbage throwing according to claim 8, wherein before generating the prompt message according to the detection result, the method further comprises:
judging whether the video stream corresponding to the pedestrian track is pushed or not;
if it has already been pushed, the process results.
10. A refuse chute detection system according to claim 9, comprising:
the video image acquisition unit is used for acquiring real-time image data of a monitored area;
a network transmission unit for network-transmitting the image data;
the foreground block detection unit is used for carrying out foreground analysis on the image data and determining a stable foreground block area in the image data;
the pedestrian detection unit is used for analyzing and processing the image data and determining the position of a pedestrian;
the target tracking unit is used for analyzing the image with the pedestrian, determining the position track of the pedestrian and acquiring a track chain of the pedestrian;
and the central processing unit is used for performing contact analysis on the track chain of the pedestrian and the position of the foreground block, determining whether the pedestrian belongs to the garbage throwing of the pedestrian or not and acquiring the whole process belonging to the garbage throwing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211013480.6A CN115424170A (en) | 2022-08-23 | 2022-08-23 | Garbage throwing detection system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211013480.6A CN115424170A (en) | 2022-08-23 | 2022-08-23 | Garbage throwing detection system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115424170A true CN115424170A (en) | 2022-12-02 |
Family
ID=84199175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211013480.6A Pending CN115424170A (en) | 2022-08-23 | 2022-08-23 | Garbage throwing detection system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115424170A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152751A (en) * | 2023-10-30 | 2023-12-01 | 西南石油大学 | Image segmentation method and system |
-
2022
- 2022-08-23 CN CN202211013480.6A patent/CN115424170A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152751A (en) * | 2023-10-30 | 2023-12-01 | 西南石油大学 | Image segmentation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Venkateswari et al. | License Plate cognizance by Ocular Character Perception' | |
| CN110795595B (en) | Video structured storage method, device, equipment and medium based on edge calculation | |
| US20070292029A1 (en) | Cascade plate recognition system | |
| CN106384513B (en) | A kind of fake-licensed car capture system and method based on intelligent transportation | |
| CN109492577B (en) | Gesture recognition method and device and electronic equipment | |
| CN110070029B (en) | Gait recognition method and device | |
| CN111898581B (en) | Animal detection method, apparatus, electronic device, and readable storage medium | |
| CN109410278B (en) | Target positioning method, device and system | |
| CN107911663A (en) | A kind of elevator passenger hazardous act intelligent recognition early warning system based on Computer Vision Detection | |
| JP2018022475A (en) | Method and apparatus for updating background model | |
| CN107786848A (en) | The method, apparatus of moving object detection and action recognition, terminal and storage medium | |
| CN111898485A (en) | Parking space vehicle detection processing method and device | |
| CN112949439A (en) | Method and system for monitoring invasion of personnel in key area of oil tank truck | |
| CN115424170A (en) | Garbage throwing detection system and method | |
| CN112298844A (en) | Garbage classification monitoring method and device | |
| CN114758271A (en) | Video processing method, device, computer equipment and storage medium | |
| CN106327876B (en) | A kind of fake-licensed car capture system and method based on automobile data recorder | |
| CN117292338B (en) | Vehicle accident identification and analysis method based on video stream analysis | |
| CN113920585A (en) | Behavior recognition method and device, equipment and storage medium | |
| CN113537088A (en) | Digital image sensor hardware fingerprint matching method and system based on small sample learning | |
| CN116012949B (en) | People flow statistics and identification method and system under complex scene | |
| CN110837760A (en) | Target detection method, training method and device for target detection | |
| CN113259630B (en) | Multi-camera pedestrian track aggregation system and method | |
| KR102367584B1 (en) | Automatic video surveillance system using skeleton video analysis technique | |
| CN113158720B (en) | Video abstraction method and device based on dual-mode feature and attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |