CN114821759A - Garbage throwing detection method, device, equipment and storage medium - Google Patents
Garbage throwing detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114821759A CN114821759A CN202110047738.3A CN202110047738A CN114821759A CN 114821759 A CN114821759 A CN 114821759A CN 202110047738 A CN202110047738 A CN 202110047738A CN 114821759 A CN114821759 A CN 114821759A
- Authority
- CN
- China
- Prior art keywords
- key point
- garbage
- arm swing
- information
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000010813 municipal solid waste Substances 0.000 title claims abstract description 331
- 238000001514 detection method Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 10
- 210000004247 hand Anatomy 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 210000003423 ankle Anatomy 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 210000000707 wrist Anatomy 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 238000003706 image smoothing Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, and discloses a garbage throwing detection method, a device, equipment and a storage medium, wherein the method comprises the following steps: determining the position information of a staying pedestrian and a garbage can in a garbage throwing area; acquiring skeleton key point information of a staying pedestrian, and determining arm swing amplitude information according to the skeleton key point information; and carrying out garbage throwing detection according to the position information of the garbage can and the arm swing amplitude information. Compared with the prior art, the method needs an administrator to manually snapshot the garbage throwing image, so that the detection efficiency of garbage throwing is low, and the garbage throwing detection is carried out according to the position information of the garbage can and the arm swing amplitude information, so that the detection efficiency of garbage throwing is improved, and the false alarm rate of garbage throwing is reduced.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a garbage throwing detection method, a garbage throwing detection device, garbage throwing detection equipment and a storage medium.
Background
In the face of increasing garbage yield, garbage resource utilization is realized to the maximum extent through garbage classification management, and the reduction of garbage disposal quantity is a very important measure. At present, garbage classification management regulations are established in multiple cities such as Beijing, Shenzhen, Nanjing and the like, and garbage classification is enforced. In the prior art, due to the influence of the installation position of the camera and the body posture of a person throwing the garbage, hands throwing the garbage cannot be correctly snapped, and the wrong report is easy to occur, so that the false report rate of garbage throwing is high.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a garbage throwing detection method, a garbage throwing detection device, garbage throwing detection equipment and a garbage throwing detection storage medium, and aims to solve the technical problem of reducing the false alarm rate of garbage throwing.
In order to achieve the above object, the present invention provides a garbage throwing detection method, including:
determining the position information of a staying pedestrian and a garbage can in a garbage throwing area;
acquiring skeleton key point information of the staying pedestrians, and determining arm swing amplitude information according to the skeleton key point information;
and carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
Optionally, the step of obtaining the skeleton key point information of the staying pedestrian and determining the arm swing amplitude information according to the skeleton key point information includes:
acquiring skeleton key point information of the staying pedestrians, and acquiring coordinates of each key point and confidence corresponding to each key point according to the skeleton key point information;
selecting a plurality of target key point coordinates from each key point according to the confidence coefficient;
and determining arm swing amplitude information according to the coordinates of the plurality of target key points.
Optionally, the step of determining arm swing information according to coordinates of a plurality of target key points includes:
selecting a palm key point coordinate and an arm key point coordinate from the multiple target key point coordinates;
and determining arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
Optionally, the step of selecting the palm key point coordinates and the arm key point coordinates from the multiple target key point coordinates includes:
selecting palm key point coordinates from the target key point coordinates, and determining garbage can coordinates according to the garbage can position information;
judging whether the palm key point coordinates meet preset throwing conditions or not according to the garbage can coordinates;
and when the palm key point coordinates meet the preset releasing conditions, selecting arm key point coordinates from the target key point coordinates.
Optionally, the step of determining arm swing information according to the arm key point coordinates and the palm key point coordinates includes:
determining an arm swing amplitude angle according to the arm key point coordinates and the palm key point coordinates;
judging whether the arm swing angle is larger than or equal to a preset swing threshold value or not;
and when the arm swing angle is greater than or equal to the preset swing threshold, determining arm swing information according to the arm swing angle.
Optionally, the step of determining arm swing information according to the arm swing angle includes:
acquiring arm swing time corresponding to the staying pedestrian;
determining the arm swing angular velocity according to the arm swing angle and the arm swing time;
judging whether the swing amplitude angular velocity of the arm is greater than or equal to a preset swing amplitude angular velocity threshold value or not;
and when the arm swing angular velocity is greater than or equal to the preset swing amplitude angular velocity threshold, determining arm swing amplitude information according to the arm swing amplitude angle and the arm swing amplitude angular velocity.
Optionally, after the step of determining whether the swing angular velocity of the arm is greater than or equal to a preset swing angular velocity threshold, the method further includes:
and when the arm swing angular velocity is smaller than the preset swing angular velocity threshold, returning to the step of determining the pedestrians staying in the garbage throwing area.
Optionally, the step of performing garbage throwing detection according to the garbage can position information and the arm swing information includes:
acquiring the staying time of the staying pedestrians in the garbage throwing area;
judging whether the stay time is less than or equal to a preset stay threshold value or not;
and when the stay time length is less than or equal to the preset stay threshold, carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
Optionally, after the step of performing garbage throwing detection according to the garbage can position information and the arm swing information, the method further includes:
when detecting that pedestrians stay for garbage throwing, acquiring a garbage image;
and determining garbage classification information according to the garbage image, and performing garbage throwing prompting on the staying pedestrians according to the garbage classification information.
Optionally, the step of determining spam classification information according to the spam image includes:
preprocessing the garbage image to obtain a garbage image to be processed;
performing convolution processing on the garbage image to be processed to obtain a convolution garbage image;
performing pooling treatment on the convolution rubbish image to obtain a rubbish characteristic image;
and determining garbage classification information according to the garbage feature image.
Optionally, the step of determining spam classification information according to the spam feature image includes:
determining spam characteristic information according to the spam characteristic image;
and classifying the garbage characteristic information to obtain garbage classification information.
Optionally, before the step of determining the position information of the staying pedestrians and the trash can in the trash throwing area, the method further includes:
acquiring a panoramic image of the trash can, and calibrating the position of the trash can according to the panoramic image of the trash can to obtain a vertex pixel coordinate;
and determining a trash can calibration area according to the vertex pixel coordinates, and determining a trash throwing area according to the trash can calibration area.
In addition, in order to achieve the above object, the present invention further provides a garbage throwing detection device, including:
the determining module is used for determining the position information of the staying pedestrians and the garbage can in the garbage throwing area;
the acquisition module is used for acquiring skeleton key point information of the staying pedestrians and determining arm swing amplitude information according to the skeleton key point information;
and the detection module is used for carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
Optionally, the obtaining module is further configured to obtain skeleton key point information of the staying pedestrian, and obtain coordinates of each key point and a confidence corresponding to each key point according to the skeleton key point information;
the acquisition module is further used for selecting a plurality of target key point coordinates from each key point according to the confidence coefficient;
the acquisition module is further used for determining arm swing amplitude information according to the coordinates of the plurality of target key points.
Optionally, the obtaining module is further configured to select a palm key point coordinate and an arm key point coordinate from the multiple target key point coordinates;
the acquisition module is further configured to determine arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
Optionally, the obtaining module is further configured to select a palm key point coordinate from the multiple target key point coordinates, and determine a trash can coordinate according to the trash can position information;
the acquisition module is further used for judging whether the palm key point coordinates meet preset throwing conditions or not according to the garbage can coordinates;
the acquisition module is further used for selecting the arm key point coordinates from the target key point coordinates when the palm key point coordinates meet the preset releasing conditions.
Optionally, the obtaining module is further configured to determine an arm swing angle according to the arm key point coordinates and the palm key point coordinates;
the obtaining module is further configured to determine whether the arm swing angle is greater than or equal to a preset swing threshold;
the obtaining module is further configured to determine arm swing information according to the arm swing angle when the arm swing angle is greater than or equal to the preset swing threshold.
Optionally, the obtaining module is further configured to obtain an arm swing time corresponding to the staying pedestrian;
the obtaining module is further configured to determine an arm swing angular velocity according to the arm swing angle and the arm swing time;
the acquisition module is further used for judging whether the arm swing amplitude angular velocity is greater than or equal to a preset swing amplitude angular velocity threshold value;
the obtaining module is further configured to determine arm swing information according to the arm swing angle and the arm swing angular velocity when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold.
In addition, in order to achieve the above object, the present invention further provides a garbage throwing detection device, including: a memory, a processor and a garbage placement detection program stored on the memory and executable on the processor, the garbage placement detection program being configured to implement the steps of the garbage placement detection method as described above.
In addition, to achieve the above object, the present invention further provides a storage medium, wherein the storage medium stores a garbage throw detection program, and the garbage throw detection program, when executed by a processor, implements the steps of the garbage throw detection method as described above.
The method comprises the steps of firstly determining the position information of a staying pedestrian and a garbage can in a garbage throwing area, then obtaining the skeleton key point information of the staying pedestrian, determining the arm swing amplitude information according to the skeleton key point information, and then carrying out garbage throwing detection on the arm swing amplitude information according to the garbage can position information and the skeleton key point information. Compared with the prior art, the garbage throwing detection method has the advantages that the garbage throwing detection efficiency is low due to the fact that the garbage throwing images need to be manually snapped, and the garbage throwing detection is carried out according to the position information of the garbage can and the arm swing amplitude information, so that the garbage throwing detection efficiency is improved, and the false alarm rate of garbage throwing is reduced.
Drawings
Fig. 1 is a schematic structural diagram of a garbage placement detection device in a hardware operating environment according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a garbage input detection method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of human skeleton key points according to a first embodiment of the garbage disposal detection method of the present invention;
FIG. 4 is a graph showing an angular velocity of an arm swing according to the first embodiment of the garbage disposal detection method of the present invention;
fig. 5 is a schematic flow chart of a garbage input detection method according to a second embodiment of the present invention;
fig. 6 is a block diagram of a garbage throw detection device according to a first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a garbage placement detection device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the trash input detecting device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a keyboard (K board), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a wireless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or a nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001 described previously.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the debris delivery detection device and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a data storage module, a network communication module, a user interface module, and a garbage drop detection program.
In the trash drop detection device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 of the trash drop detection device of the present invention may be disposed in the trash drop detection device, and the trash drop detection device calls the trash drop detection program stored in the memory 1005 through the processor 1001 and executes the trash drop detection method provided by the embodiment of the present invention.
An embodiment of the present invention provides a garbage throwing detection method, and referring to fig. 2, fig. 2 is a schematic flow diagram of a first embodiment of the garbage throwing detection method according to the present invention.
In this embodiment, the garbage throwing detection method includes the following steps:
step S10: and determining the position information of the staying pedestrians and the garbage can in the garbage throwing area.
It is easy to understand that the execution subject of this embodiment may be a garbage throwing detection device having functions of data processing, network communication, program operation, and the like, or may be other computer devices having similar functions, and the present embodiment is not limited thereto.
It is to be understood that the trash input area may be a trash input area and/or a trash input exit area, and the trash can position information may be spatial coordinate information of a trash can, and the present embodiment is not limited thereto.
Further, in order to accurately determine a garbage throwing area, before the step of determining the position information of the staying pedestrians and the garbage can in the garbage throwing area, a panoramic image of the garbage can be obtained, the position of the garbage can is calibrated according to the panoramic image of the garbage can, vertex pixel coordinates are obtained, then the garbage can calibration area is determined according to the vertex pixel coordinates, the garbage throwing area is determined according to the garbage can calibration area, and the like.
The garbage throwing Area can be defined in a mode that the position of a garbage can of the throwing point is integrally calibrated by adopting a quadrangle, and pixel coordinates of four vertexes are obtained and are marked as Area _ bin. And then a garbage throwing-in Area and a garbage throwing-out Area are defined by adopting a quadrangle and are respectively marked as Area _ in and Area _ out, wherein the sizes of the garbage throwing-in Area and the garbage throwing-out Area can be selected according to the field situation, and the like.
Assuming that the width of the garbage throwing-in and entering Area is 2-4 times of the width of the Area _ bin and the height is 1.2 times of the height of the Area _ bin, the width of the garbage throwing-in and leaving Area can be 3-5 times of the width of the Area _ bin and the height is 1.4 times of the height of the Area _ bin, and the like.
It should be noted that the garbage input area and the garbage input exit area may be the same area, but due to an image detection error or a small positional change of a pedestrian at the boundary, the detection result may show that the pedestrian repeatedly enters and exits in a short time, and such a shake or switching brings much inconvenience to the subsequent processing. Therefore, the area set for the refuse input/output region needs to be larger than the area set for the refuse input/output region, and thus, image detection errors and the like can be reduced.
It should be further explained that the stay pedestrians are the to-be-detected people entering the garbage throwing area, and the like, because more pedestrians can walk back and forth from the garbage throwing area, in order to accurately identify the stay pedestrians needing garbage throwing, a proper intelligent model can be selected according to the field resource conditions, and the intelligent model can detect and track the pedestrians located in the throwing area.
Because the target detection YOLO V5 model performs pedestrian detection, the inference time is 0.007 seconds at the fastest speed, namely 140 frames per second, the size of the weight file is only 27MB, the intelligent model can be a target detection YO LO V5 model, and then the target detection YOLO V5 model can be used for pedestrian detection. And then, the pedestrian tracking can adopt a multi-target tracking algorithm Deepsort, and further data association is carried out by utilizing the motion model and the appearance information.
If the multi-target tracking algorithm detects the pedestrian i, a unique identifier is generated for the pedestrian i and is recorded as an ID i And as long as no tracking is lost, ID i Remain unchanged. And establishing a pedestrian ID dynamic list ID _ list ═ ID 1 ,ID 2 ,......]When detecting that a pedestrian enters the detection Area _ in, adding the ID of the pedestrian to the list; when it is detected that the pedestrian goes out of the detection Area _ out, the ID thereof is deleted from the category. The time for the pedestrian i to enter and leave the throwing area can be recorded as t i,in And t i,out To perform storage, etc.
Step S20: and acquiring skeleton key point information of the staying pedestrians, and determining arm swing amplitude information according to the skeleton key point information.
The skeleton key point information may be coordinate information of a plurality of joint points where pedestrians stay, may be palm coordinates, elbow coordinates, shoulder coordinates, and the like, and the arm swing information may be an arm swing angle, an arm swing angular velocity, and the like, which is not limited in this embodiment.
Further, in order to accurately acquire arm swing information, acquire skeleton key point information of a staying pedestrian, and determine the arm swing information according to the skeleton key point information, the method includes the steps of: obtaining skeleton key point information of the staying pedestrians, wherein the skeleton key point information comprises coordinates of each key point and confidence degrees corresponding to the key points, selecting a plurality of target key point coordinates from the key points according to the confidence degrees, and determining arm swing amplitude information and the like according to the target key point coordinates.
Assuming that the skeleton key point information includes a palm coordinate, an elbow coordinate and a shoulder coordinate, the confidence corresponding to the palm coordinate is 0.9, the confidence corresponding to the elbow coordinate is 0.1, and the confidence corresponding to the shoulder coordinate is 0.7, if the preset confidence threshold is 0.6 and the confidence corresponding to the palm coordinate and the shoulder coordinate is greater than the preset confidence threshold, the palm coordinate and the shoulder coordinate are used as the target key point coordinate, and the confidence corresponding to the elbow coordinate is less than the preset confidence threshold, the elbow coordinate and the like need to be obtained again by using skeleton key point obtaining software, wherein the preset confidence threshold may be set by a user in a self-defined manner, may be 0.6, may also be 0.8 and the like, which is not limited in this embodiment.
The method for determining the arm swing information according to the coordinates of the plurality of target key points can also be used for selecting the coordinates of the palm key point and the coordinates of the arm key point from the coordinates of the plurality of target key points, and then determining the arm swing information according to the coordinates of the arm key point and the coordinates of the palm key point, and the like.
The processing mode of selecting the palm key point coordinates and the arm key point coordinates from the target key point coordinates can be that the palm key point coordinates are selected from the target key point coordinates, the garbage can coordinates are determined according to the garbage can position information, and whether the palm key point coordinates meet preset throwing conditions or not is judged according to the garbage can coordinates; and when the palm key point coordinates meet the preset releasing conditions, selecting arm key point coordinates and the like from the target key point coordinates.
The method for determining the arm swing information according to the arm key point coordinates and the palm key point coordinates may be that an arm swing angle is determined according to the arm key point coordinates and the palm key point coordinates, whether the arm swing angle is greater than or equal to a preset swing threshold value is judged, and when the arm swing angle is greater than or equal to the preset swing threshold value, the arm swing information and the like are determined according to the arm swing angle, wherein the preset swing threshold value may be set by a user in a self-defined manner, and may be 60 degrees, or 80 degrees, or may be 100 degrees.
The step of determining the arm swing information according to the arm swing angle may be to obtain arm swing time corresponding to the staying pedestrian, determine arm swing angular velocity according to the arm swing angle and the arm swing time, determine whether the arm swing angular velocity is greater than or equal to a preset swing angular velocity threshold, determine arm swing information and the like according to the arm swing angle and the arm swing angular velocity when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold, and return to the step of determining the staying pedestrian in the garbage throwing area when the arm swing angular velocity is less than the preset swing angular velocity threshold, where the preset swing angular velocity may be set by a user in a customized manner, may be 4, may also be 5, and the like.
Assuming that the arm swing angle is 100 degrees according to the arm key point coordinates and the angle between the palm key point coordinates and the ground, namely the arm swing angle, judging whether the arm swing angle is 100 degrees or more than or equal to a preset swing threshold value of 90 degrees, when the arm swing angle is 100 degrees or more than the preset swing threshold value 90 degrees, acquiring the arm swing time corresponding to the staying pedestrian, if the arm swing time is 20s and is more than the preset arm swing time threshold value 4s, determining arm swing angular velocity 5 according to the arm swing angle of 100 degrees and arm swing time of 20s, judging whether the arm swing angular velocity is greater than or equal to a preset swing angular velocity threshold 4, and when the arm swing amplitude angular velocity is greater than or equal to a preset swing amplitude angular velocity threshold, determining arm swing amplitude information according to the arm swing amplitude angle and the arm swing amplitude angular velocity, determining the arm swing amplitude information according to the arm swing amplitude angle, and the like.
Step S30: and carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
The step of performing garbage throwing detection according to the garbage bin position information and the arm swing amplitude information may be to obtain a staying time length of a pedestrian staying in a garbage throwing area, then judge whether the staying time length is less than or equal to a preset staying threshold, and perform garbage throwing detection on the arm swing amplitude information according to the garbage bin position information and the bone key point information when the staying time length is less than or equal to the preset staying threshold, wherein the preset staying threshold may be set by a user in a self-defined manner, may be 5min, may also be 6min, and the like.
In a specific implementation, for each detected and tracked pedestrian, 17 key point coordinates of the human body are output by using a human body key point detection model AlphaPose from top to bottom, as shown in fig. 3, fig. 3 is a schematic diagram of human skeleton key points of a first embodiment of the garbage throwing detection method of the present invention, where 1 is a neck key point, 2 is a right shoulder key point, 3 is a right elbow key point, 4 is a right wrist key point, 5 is a left shoulder key point, 6 is a left elbow key point, 7 is a left wrist key point, 8 is a right hip key point, 9 is a right knee key point, 10 is a right ankle key point, 11 is a left hip key point, 12 is a right ankle key point, 11 is a left hip key point, and 12 is a left elbow key pointThe left knee keypoint, 13 is the left ankle keypoint. Each of the skeletal key points corresponds to a coordinate p i ＝(x i ,y i ) And score c i ∈[0,1]I-0, 1.. and 16, which respectively represent the pixel position in the skeleton key point diagram and the confidence of the skeleton point position, then judging whether the staying pedestrian has the garbage throwing action, and detecting whether the two-hand skeleton point is located above the garbage can, if (x) 4 ,y 4 ) Belongs to Area _ bin and (x) 7 ,y 7 ) The left Area _ bin is not established, the skeleton points of the two hands are not detected above the kitchen garbage bin, the swing amplitude (angle) of the arms of the pedestrian is calculated in real time through the positions of the skeleton points of the two elbows and the two hands, and the maximum swing amplitude and the swing angular speed are judged to exceed the threshold value.
The manner of calculating the swing angular velocity may be to calculate and store each time frame t j (Vector)
The angle with the vertical y-axis is recorded
When the pedestrian is detected to leave the detection area, the maximum swing amplitude of the hands is calculated
If max _ degree<degr ee thresh Detecting and tracking the pedestrian in the perimeter, otherwise, taking c from near to far on the left and right sides of the angle taking the max _ degree value i >c thresh The slope k of two straight lines is calculated by linear fitting 1 And k 2 I.e., angular velocity, as shown in fig. 4. FIG. 4 is a graph showing the swing angular velocity of the arm according to the first embodiment of the garbage disposal detection method of the present invention, wherein the point "x" is a point c i <c thresh And the fitting is not participated, and the x axis is the video frame time, namely the frame number, so that the time consumption of an algorithm introduced by calculating astronomical time is avoided. Let k be max (| k) 1 ∣,∣k 2 | if k) is present<k thresh Detecting and tracking the pedestrians in the perimeter, otherwise, calculating the staying time of the pedestrians in the garbage throwing areaAnd judging whether the threshold value is smaller than the threshold value. Note t i ＝t i,out -t i,in If t is i >t thresh And detecting and tracking the pedestrian in the perimeter, otherwise, giving an alarm to prompt 'please correctly throw garbage', and deleting the pedestrian ID from the ID _ list. Judging whether all video frames are analyzed completely, otherwise, detecting and tracking pedestrians in the perimeter, wherein t i The unit of (d) is the number of video frames, etc.
Further, in order to reduce the false alarm rate of garbage throwing, after the step of detecting garbage throwing according to the garbage bin position information and the arm swing amplitude information, preprocessing a garbage image to obtain a garbage image to be processed, performing convolution processing on the garbage image to be processed to obtain a convolution garbage image, performing pooling processing on the convolution garbage image to obtain a garbage characteristic image, determining garbage classification information according to the garbage characteristic image, wherein the garbage classification information can be dry garbage, harmful garbage, wet garbage and the like, and then judging whether pedestrians stop the pedestrians to correctly throw the garbage or not according to the garbage classification information.
The step of determining the garbage classification information according to the garbage feature image may be to determine the garbage feature information according to the garbage feature image, and then perform classification processing on the garbage feature information to obtain the garbage classification information and the like.
The preprocessing refers to the enhancement processing of the image, and the enhancement processing comprises the processes of noise elimination, image smoothing and image sharpening, and the enhancement processing is realized by an image recognition processor.
The characteristic image is obtained by training a neural network model through an image recognition processor, wherein the neural network model is a trained neural network model and comprises the following 15 layers: the system comprises four convolution layers, four down-sampling layers, three full-connection layers, three activation layers and a classification layer which are mutually crossed, wherein the full-connection layers are used for adjusting parameters and enabling a network to be more stable, and the activation layers are used for improving the network speed; in the neural network model, after the preprocessed image is subjected to convolution and down-sampling processing by intersection, weight parameters in a10 th activation layer are extracted by an image recognition processor to serve as the characteristics of the image.
The features not only need to be able to better describe the images and reduce overfitting, but also need to be able to better distinguish different classes of images.
The convolutional neural network image training model constructed by the method has 15 layers in total, the structure of the convolutional neural network image training model improves the recognition rate, and the convolutional neural network image training model has better stability for recognizing garbage. The preset classification network used by the invention can be a plurality of models, different classifiers such as a support vector machine, a radial basis classifier and a BP neural network can be selected for comparison according to the requirements of users and the number of images, the effect is optimal according to an experimental support vector machine, and the recognition rate is highest at the fastest speed.
The characteristic image recognition is realized through a preset classification network model, the classification network model is a network model trained through characteristic images, after the characteristic images are obtained, the characteristic images are directly input into the classification network model to be recognized, then the recognition results are returned to an image recognition processor, and the image recognition processor judges whether the recoverable garbage is recoverable according to the recognition results, judges the type of the recoverable garbage and the like.
In this embodiment, the position information of a staying pedestrian and a trash can in a trash throwing area is determined, then the skeleton key point information of the staying pedestrian is obtained, the arm swing amplitude information is determined according to the skeleton key point information, and then trash throwing detection is performed according to the position information of the trash can and the arm swing amplitude information. Compared with the prior art, the garbage throwing detection method has the advantages that the garbage throwing detection efficiency is low due to the fact that the garbage throwing images need to be manually snapped, and the garbage throwing detection is carried out according to the position information of the garbage can and the arm swing amplitude information, so that the garbage throwing detection efficiency is improved, and the false alarm rate of garbage throwing is reduced.
Referring to fig. 5, fig. 5 is a flowchart illustrating a garbage input detection method according to a second embodiment of the present invention.
Based on the first embodiment, in this embodiment, the step S20 further includes:
step S201: and acquiring skeleton key point information of the staying pedestrians, and acquiring coordinates of each key point and confidence corresponding to each key point according to the skeleton key point information.
The skeleton key point information may be coordinate information of a plurality of joint points where pedestrians stay, may be palm coordinates, elbow coordinates, shoulder coordinates, and the like, and the arm swing information may be an arm swing angle, an arm swing angular velocity, and the like, which is not limited in this embodiment.
Assuming that the skeleton key point information includes a palm coordinate, an elbow coordinate and a shoulder coordinate, the confidence corresponding to the palm coordinate is 9, the confidence corresponding to the elbow coordinate is 4, the confidence corresponding to the shoulder coordinate is 7, and the like.
Step S202: and selecting a plurality of target key point coordinates from each key point according to the confidence degree.
Further, in order to accurately acquire arm swing information, acquire skeleton key point information of a staying pedestrian, and determine the arm swing information according to the skeleton key point information, the method includes the steps of: obtaining skeleton key point information of the staying pedestrians, determining coordinates of each key point and confidence degrees corresponding to each key point according to the skeleton key point information, and selecting a plurality of target key point coordinates from each key point according to the confidence degrees.
Assuming that the skeleton key point information includes a palm coordinate, an elbow coordinate and a shoulder coordinate, the confidence corresponding to the palm coordinate is 0.9, the confidence corresponding to the elbow coordinate is 0.4, and the confidence corresponding to the shoulder coordinate is 0.7, if the preset confidence threshold is 0.6, and the confidence corresponding to the palm coordinate and the shoulder coordinate is greater than the preset confidence threshold, the palm coordinate and the shoulder coordinate are taken as target key point coordinates, and the confidence corresponding to the elbow coordinate is less than the preset confidence threshold, the elbow coordinate and the like need to be re-acquired by using skeleton key point acquisition software, wherein the preset confidence threshold may be set by a user in a self-defined manner, may be 0.6, may also be 0.8, and the present embodiment is not limited.
The method for determining the arm swing information according to the coordinates of the plurality of target key points can also be used for selecting the coordinates of the palm key point and the coordinates of the arm key point from the coordinates of the plurality of target key points, and then determining the arm swing information according to the coordinates of the arm key point and the coordinates of the palm key point, and the like.
The processing mode of selecting the palm key point coordinates and the arm key point coordinates from the target key point coordinates can be that the palm key point coordinates are selected from the target key point coordinates, the garbage can coordinates are determined according to the garbage can position information, and whether the palm key point coordinates meet preset throwing conditions or not is judged according to the garbage can coordinates; and when the palm key point coordinates meet the preset releasing conditions, selecting arm key point coordinates and the like from the target key point coordinates.
Step S203: and determining arm swing amplitude information according to the coordinates of the plurality of target key points.
The target key point coordinates may be arm key point coordinates and palm key point coordinates, and the manner of determining arm swing information according to the arm key point coordinates and the palm key point coordinates may be, determining an arm swing angle according to the arm key point coordinates and the palm key point coordinates, determining whether the arm swing angle is greater than or equal to a preset swing threshold, and determining arm swing information according to the arm swing angle when the arm swing angle is greater than or equal to the preset swing threshold, and the like, where the preset swing threshold may be set by a user in a user-defined manner, and may be 60 degrees, or 80 degrees, or may be 100 degrees, and the like.
The step of determining the arm swing information according to the arm swing angle may be to obtain arm swing time corresponding to the staying pedestrian, determine arm swing angular velocity according to the arm swing angle and the arm swing time, determine whether the arm swing angular velocity is greater than or equal to a preset swing angular velocity threshold, determine arm swing information and the like according to the arm swing angle and the arm swing angular velocity when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold, and return to the step of determining the staying pedestrian in the garbage throwing area when the arm swing angular velocity is less than the preset swing angular velocity threshold, where the preset swing angular velocity may be set by a user in a customized manner, may be 4, may also be 5, and the like.
Assuming that the arm swing angle is 100 degrees according to the arm key point coordinates and the angle between the palm key point coordinates and the ground, namely the arm swing angle, judging whether the arm swing angle is 100 degrees or more than or equal to a preset swing threshold value of 90 degrees, when the arm swing angle is 100 degrees or more than the preset swing threshold value 90 degrees, acquiring the arm swing time corresponding to the staying pedestrian, if the arm swing time is 20s and is more than the preset arm swing time threshold value 4s, determining arm swing angular velocity 5 according to the arm swing angle of 100 degrees and arm swing time of 20s, judging whether the arm swing angular velocity is greater than or equal to a preset swing angular velocity threshold 4, and when the arm swing amplitude angular velocity is greater than or equal to a preset swing amplitude angular velocity threshold, determining arm swing amplitude information according to the arm swing amplitude angle and the arm swing amplitude angular velocity, determining the arm swing amplitude information according to the arm swing amplitude angle, and the like.
In a specific implementation, for each detected and tracked pedestrian, a human body key point detection model alphaPose from top to bottom is adopted to output coordinates of 17 key points of a human body, as shown in fig. 3, fig. 3 is a schematic diagram of human body skeleton key points of a first embodiment of the garbage throwing detection method of the present invention, wherein each point in the skeleton key points corresponds to a coordinate p i ＝(x i ,y i ) And score c i ∈[0,1]I-0, 1.. and 16, which respectively represent the pixel position in the skeleton key point diagram and the confidence of the skeleton point position, then judging whether the staying pedestrian has the garbage throwing action, and detecting whether the two-hand skeleton point is located above the garbage can, if (x) 4 ,y 4 ) Belongs to A rea _ bin and (x) 7 ,y 7 ) The left Area _ bin is not established, the skeleton points of the two hands are not detected above the kitchen garbage bin, the swing amplitude (angle) of the arms of the pedestrian is calculated in real time through the positions of the skeleton points of the two elbows and the two hands, and the maximum swing amplitude and the swing angular speed are judged to exceed the threshold value.
The manner of calculating the swing angular velocity may be to calculate and store each time frame t j (Vector)
The angle with the vertical y-axis is recorded
Calculating the maximum pendulum of both hands when it is detected that the pedestrian walks out of the detection areaWeb with two or more webs
If max _ degre e<degree thresh Detecting and tracking the pedestrian in the perimeter, otherwise, taking c from near to far on the left and right sides of the angle taking the max _ degree value i >c thresh The slope k of two straight lines is calculated by linear fitting 1 And k 2 Namely, the angular velocity, and the arm swing angle and the arm swing angular velocity are used as the arm swing information, and the like.
In this embodiment, first, skeleton key point information of a staying pedestrian is obtained, including coordinates of each key point and a confidence corresponding to the key point, then a plurality of target key point coordinates are selected from each key point according to the confidence, and then arm swing amplitude information is determined according to the plurality of target key point coordinates, so that the arm swing amplitude information can be accurately obtained.
Referring to fig. 6, fig. 6 is a block diagram illustrating a structure of a garbage disposal detecting apparatus according to a first embodiment of the present invention.
As shown in fig. 6, a garbage throwing detection device according to an embodiment of the present invention includes:
a determining module 6001, configured to determine information about positions of a staying pedestrian and a trash can in a trash throwing area;
an obtaining module 6002, configured to obtain skeleton key point information of the staying pedestrian, and determine arm swing amplitude information according to the skeleton key point information;
and the detecting module 6003 is configured to perform garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
In this embodiment, the position information of a staying pedestrian and a trash can in a trash throwing area is determined, then the skeleton key point information of the staying pedestrian is obtained, the arm swing amplitude information is determined according to the skeleton key point information, and then trash throwing detection is performed according to the position information of the trash can and the arm swing amplitude information. Compared with the prior art, the garbage throwing detection method has the advantages that the garbage throwing detection efficiency is low due to the fact that the garbage throwing images need to be manually snapped, and the garbage throwing detection is carried out according to the position information of the garbage can and the arm swing amplitude information, so that the garbage throwing detection efficiency is improved, and the false alarm rate of garbage throwing is reduced.
Further, the obtaining module 6002 is further configured to obtain skeleton key point information of the staying pedestrian, and obtain coordinates of each key point and a confidence corresponding to each key point according to the skeleton key point information;
the obtaining module 6002 is further configured to select a plurality of target keypoint coordinates from the keypoints according to the confidence;
the obtaining module 6002 is further configured to determine arm swing amplitude information according to the coordinates of the multiple target key points.
Further, the obtaining module 6002 is further configured to select a palm key point coordinate and an arm key point coordinate from the multiple target key point coordinates;
the obtaining module 6002 is further configured to determine arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
Further, the obtaining module 6002 is further configured to select a palm key point coordinate from the multiple target key point coordinates, and determine a trash can coordinate according to the trash can position information;
the obtaining module 6002 is further configured to determine whether the palm key point coordinate meets a preset releasing condition according to the trash can coordinate;
the obtaining module 6002 is further configured to select an arm key point coordinate from each target key point coordinate when the palm key point coordinate meets the preset launching condition.
Further, the obtaining module 6002 is further configured to determine an arm swing angle according to the arm key point coordinate and the palm key point coordinate;
the obtaining module 6002 is further configured to determine whether the arm swing angle is greater than or equal to a preset swing threshold;
the obtaining module 6002 is further configured to determine arm swing amplitude information according to the arm swing amplitude angle when the arm swing amplitude angle is greater than or equal to the preset swing amplitude threshold.
Further, the acquiring module 6002 is further configured to acquire an arm swing time corresponding to the staying pedestrian;
the obtaining module 6002 is further configured to determine an arm swing angular velocity according to the arm swing angle and the arm swing time;
the obtaining module 6002 is further configured to determine whether the arm swing amplitude angular velocity is greater than or equal to a preset swing amplitude angular velocity threshold;
the obtaining module 6002 is further configured to determine arm swing information according to the arm swing angle and the arm swing angular velocity when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold.
Further, the obtaining module 6002 is further configured to return to the operation of determining the pedestrian staying in the trash throwing area when the arm swing angular velocity is smaller than the preset swing angular velocity threshold.
Further, the detecting module 6003 is further configured to obtain a staying time period of the staying pedestrian in the garbage throwing area;
the detecting module 6003 is further configured to determine whether the staying time is less than or equal to a preset staying threshold;
the detecting module 6003 is further configured to, when the staying duration is less than or equal to the preset staying threshold, perform garbage throwing detection according to the garbage bin position information and the arm swing amplitude information.
Further, the garbage throwing detection device also comprises a prompt module;
the prompting module is used for acquiring a garbage image when detecting that a pedestrian stays for garbage throwing;
the prompting module is further used for determining garbage classification information according to the garbage image and prompting the staying pedestrians to throw garbage according to the garbage classification information.
Further, the prompt module is further configured to preprocess the garbage image to obtain a garbage image to be processed;
the prompting module is further used for carrying out convolution processing on the garbage image to be processed to obtain a convolution garbage image;
the prompting module is further used for performing pooling treatment on the convolution rubbish image to obtain a rubbish characteristic image;
the prompt module is further used for determining the garbage classification information according to the garbage feature image.
Further, the prompt module is further configured to determine spam feature information according to the spam feature image;
the prompt module is further used for classifying the garbage characteristic information to obtain garbage classification information.
Further, the determining module 6001 is further configured to obtain a panoramic image of the trash can, calibrate the position of the trash can according to the panoramic image of the trash can, and obtain vertex pixel coordinates;
the determining module 6001 is further configured to determine a calibration area of the trash can according to the vertex pixel coordinates, and determine a trash throwing area according to the calibration area of the trash can.
Other embodiments or specific implementation manners of the garbage throwing detection device of the present invention may refer to the above method embodiments, and are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., a rom/ram, a magnetic disk, an optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
The invention discloses A1 and a garbage throwing detection method, which comprises the following steps:
determining the position information of a staying pedestrian and a garbage can in a garbage throwing area;
acquiring skeleton key point information of the staying pedestrians, and determining arm swing amplitude information according to the skeleton key point information;
and carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
A2, the method of claim a1, wherein the step of obtaining skeleton key point information of the staying pedestrian and determining arm swing information according to the skeleton key point information comprises:
acquiring skeleton key point information of the staying pedestrians, and acquiring coordinates of each key point and confidence corresponding to each key point according to the skeleton key point information;
selecting a plurality of target key point coordinates from each key point according to the confidence coefficient;
and determining arm swing amplitude information according to the coordinates of the plurality of target key points.
A3, the method of claim a2, the step of determining arm swing information from a plurality of target keypoint coordinates, comprising:
selecting a palm key point coordinate and an arm key point coordinate from the multiple target key point coordinates;
and determining arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
A4, the method of claim A3, the step of selecting palm keypoint coordinates and arm keypoint coordinates from a plurality of target keypoint coordinates, comprising:
selecting palm key point coordinates from the target key point coordinates, and determining garbage can coordinates according to the garbage can position information;
judging whether the palm key point coordinates meet preset throwing conditions or not according to the garbage can coordinates;
and when the palm key point coordinates meet the preset releasing conditions, selecting arm key point coordinates from the target key point coordinates.
A5, the method of claim A3, the step of determining arm swing information from the arm keypoint coordinates and the palm keypoint coordinates comprising:
determining an arm swing amplitude angle according to the arm key point coordinates and the palm key point coordinates;
judging whether the arm swing angle is larger than or equal to a preset swing threshold value or not;
and when the arm swing angle is greater than or equal to the preset swing threshold, determining arm swing information according to the arm swing angle.
A6, the method of claim a5, the step of determining arm swing information from the arm swing angle comprising:
acquiring arm swing time corresponding to the staying pedestrian;
determining the arm swing angular velocity according to the arm swing angle and the arm swing time;
judging whether the swing amplitude angular velocity of the arm is greater than or equal to a preset swing amplitude angular velocity threshold value or not;
and when the arm swing angular velocity is greater than or equal to the preset swing amplitude angular velocity threshold, determining arm swing amplitude information according to the arm swing amplitude angle and the arm swing amplitude angular velocity.
A7, the method according to claim a6, wherein the step of determining whether the swing angular velocity of the arm is greater than or equal to a preset swing angular velocity threshold further comprises:
and when the swing angular velocity of the arm is smaller than the preset swing angular velocity threshold, returning to the step of determining the pedestrians staying in the garbage throwing area.
A8, the method according to any one of claims a1-a7, wherein the step of performing garbage throw detection according to the garbage can position information and the arm swing information comprises:
acquiring the staying time of the staying pedestrians in the garbage throwing area;
judging whether the stay time is less than or equal to a preset stay threshold value or not;
and when the stay time length is less than or equal to the preset stay threshold, carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
A9, the method according to claim A8, further comprising, after the step of performing garbage throw detection based on the garbage can position information and the arm swing information:
when detecting that pedestrians stay for garbage throwing, acquiring a garbage image;
and determining garbage classification information according to the garbage image, and carrying out garbage putting prompt on the staying pedestrians according to the garbage classification information.
A10, the method of claim A9, the step of determining spam classification information from the spam images comprising:
preprocessing the garbage image to obtain a garbage image to be processed;
performing convolution processing on the garbage image to be processed to obtain a convolution garbage image;
performing pooling treatment on the convolution rubbish image to obtain a rubbish characteristic image;
and determining garbage classification information according to the garbage feature image.
A11, the method of claim A10, the step of determining spam classification information from the spam feature images comprising:
determining garbage characteristic information according to the garbage characteristic image;
and classifying the garbage characteristic information to obtain garbage classification information.
A12, the method of claim a1, the step of determining the position information of the staying pedestrians and the trash can in the trash launch area further comprising:
acquiring a panoramic image of the trash can, and calibrating the position of the trash can according to the panoramic image of the trash can to obtain a vertex pixel coordinate;
and determining a trash can calibration area according to the vertex pixel coordinates, and determining a trash throwing area according to the trash can calibration area.
The invention discloses B13 and a garbage throwing detection device, which comprises:
the determining module is used for determining the position information of the staying pedestrians and the garbage can in the garbage throwing area;
the acquisition module is used for acquiring skeleton key point information of the staying pedestrians and determining arm swing amplitude information according to the skeleton key point information;
and the detection module is used for carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
B14, the device of claim B13, the obtaining module is further configured to obtain skeleton key point information of the staying pedestrian, and obtain coordinates of each key point and a confidence corresponding to each key point according to the skeleton key point information;
the acquisition module is further used for selecting a plurality of target key point coordinates from each key point according to the confidence coefficient;
the acquisition module is further used for determining arm swing amplitude information according to the coordinates of the plurality of target key points.
B15 the apparatus of claim B14, the means for obtaining further configured to select palm keypoint coordinates and arm keypoint coordinates from a plurality of target keypoint coordinates;
the acquisition module is further configured to determine arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
B16, the apparatus of claim B15, the obtaining module further configured to select palm keypoint coordinates from a plurality of target keypoint coordinates and determine trash can coordinates according to the trash can position information;
the acquisition module is further used for judging whether the palm key point coordinates meet preset throwing conditions or not according to the garbage can coordinates;
the acquisition module is further used for selecting the arm key point coordinates from the target key point coordinates when the palm key point coordinates meet the preset releasing conditions.
B17, the apparatus according to claim B15, the obtaining module further configured to determine an arm swing angle according to the arm key point coordinates and the palm key point coordinates;
the obtaining module is further configured to determine whether the arm swing angle is greater than or equal to a preset swing threshold;
the obtaining module is further configured to determine arm swing information according to the arm swing angle when the arm swing angle is greater than or equal to the preset swing threshold.
B18, the device according to claim B17, the obtaining module is further configured to obtain arm swing time corresponding to the staying pedestrian;
the obtaining module is further configured to determine an arm swing angular velocity according to the arm swing angle and the arm swing time;
the acquisition module is further used for judging whether the arm swing amplitude angular velocity is greater than or equal to a preset swing amplitude angular velocity threshold value;
the obtaining module is further configured to determine arm swing information according to the arm swing angle and the arm swing angular velocity when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold.
The invention discloses C19 and a garbage throwing detection device, which comprises: a memory, a processor and a garbage placement detection program stored on the memory and executable on the processor, the garbage placement detection program being configured to implement the steps of the garbage placement detection method as described above.
The invention discloses D20 and a storage medium, which is characterized in that a garbage throwing detection program is stored on the storage medium, and the steps of the garbage throwing detection method are realized when the garbage throwing detection program is executed by a processor.
Claims (10)
1. A garbage throwing detection method is characterized by comprising the following steps:
determining the position information of a staying pedestrian and a garbage can in a garbage throwing area;
acquiring skeleton key point information of the staying pedestrians, and determining arm swing amplitude information according to the skeleton key point information;
and carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
2. The method of claim 1, wherein the step of obtaining skeletal key point information of the stopped pedestrian and determining arm swing information from the skeletal key point information comprises:
acquiring skeleton key point information of the staying pedestrians, and acquiring coordinates of each key point and confidence corresponding to each key point according to the skeleton key point information;
selecting a plurality of target key point coordinates from each key point according to the confidence coefficient;
and determining arm swing amplitude information according to the coordinates of the plurality of target key points.
3. The method of claim 2, wherein the step of determining arm swing information based on a plurality of target keypoint coordinates comprises:
selecting a palm key point coordinate and an arm key point coordinate from the multiple target key point coordinates;
and determining arm swing amplitude information according to the arm key point coordinates and the palm key point coordinates.
4. The method of claim 3, wherein the step of selecting palm keypoint coordinates and arm keypoint coordinates from a plurality of target keypoint coordinates comprises:
selecting palm key point coordinates from the target key point coordinates, and determining garbage can coordinates according to the garbage can position information;
judging whether the palm key point coordinates meet preset throwing conditions or not according to the garbage can coordinates;
and when the palm key point coordinates meet the preset releasing conditions, selecting arm key point coordinates from the target key point coordinates.
5. The method of claim 3, wherein the step of determining arm swing information from the arm keypoint coordinates and the palm keypoint coordinates comprises:
determining an arm swing amplitude angle according to the arm key point coordinates and the palm key point coordinates;
judging whether the arm swing angle is larger than or equal to a preset swing threshold value or not;
and when the arm swing angle is greater than or equal to the preset swing threshold, determining arm swing information according to the arm swing angle.
6. The method of claim 5, wherein the step of determining arm swing information from the arm swing angle comprises:
acquiring arm swing time corresponding to the staying pedestrian;
determining the arm swing angular velocity according to the arm swing angle and the arm swing time;
judging whether the swing amplitude angular velocity of the arm is greater than or equal to a preset swing amplitude angular velocity threshold value or not;
and when the arm swing angular velocity is greater than or equal to the preset swing angular velocity threshold, determining arm swing information according to the arm swing angle and the arm swing angular velocity.
7. The method according to claim 6, wherein the step of determining whether the arm swing angular velocity is greater than or equal to a preset swing angular velocity threshold further comprises:
and when the arm swing angular velocity is smaller than the preset swing angular velocity threshold, returning to the step of determining the pedestrians staying in the garbage throwing area.
8. The utility model provides a detection device is put in to rubbish, its characterized in that, detection device is put in to rubbish includes:
the determining module is used for determining the position information of the staying pedestrians and the garbage can in the garbage throwing area;
the acquisition module is used for acquiring skeleton key point information of the staying pedestrians and determining arm swing amplitude information according to the skeleton key point information;
and the detection module is used for carrying out garbage throwing detection according to the garbage can position information and the arm swing amplitude information.
9. A refuse chute detection device, characterized in that the device comprises: memory, a processor and a garbage placement detection program stored on the memory and executable on the processor, the garbage placement detection program being configured to implement the steps of the garbage placement detection method as claimed in any one of claims 1 to 7.
10. A storage medium having stored thereon a spam detection program which, when executed by a processor, implements the steps of the spam detection method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110047738.3A CN114821759A (en) | 2021-01-13 | 2021-01-13 | Garbage throwing detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110047738.3A CN114821759A (en) | 2021-01-13 | 2021-01-13 | Garbage throwing detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114821759A true CN114821759A (en) | 2022-07-29 |
Family
ID=82524566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110047738.3A Pending CN114821759A (en) | 2021-01-13 | 2021-01-13 | Garbage throwing detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114821759A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128397A (en) * | 2021-04-16 | 2021-07-16 | 广州中大中鸣科技有限公司 | Garbage classification throwing monitoring method, system and device and storage medium |
-
2021
- 2021-01-13 CN CN202110047738.3A patent/CN114821759A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128397A (en) * | 2021-04-16 | 2021-07-16 | 广州中大中鸣科技有限公司 | Garbage classification throwing monitoring method, system and device and storage medium |
| CN113128397B (en) * | 2021-04-16 | 2024-01-05 | 广州中大中鸣科技有限公司 | Monitoring method, system, device and storage medium for garbage classification delivery |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111666857B (en) | Human behavior recognition method, device and storage medium based on environment semantic understanding | |
| US20240037977A1 (en) | Image analysis using neural networks for pose and action identification | |
| CN107066983B (en) | Identity verification method and device | |
| WO2019033525A1 (en) | Au feature recognition method, device and storage medium | |
| CN111012261A (en) | Sweeping method and system based on scene recognition, sweeping equipment and storage medium | |
| CN109325456B (en) | Target identification method, target identification device, target identification equipment and storage medium | |
| CN112241731A (en) | Attitude determination method, device, equipment and storage medium | |
| CN107679448A (en) | Eyeball action-analysing method, device and storage medium | |
| US20200356764A1 (en) | Method and apparatus for recognizing illegal behavior in unattended scenario | |
| CN109063678B (en) | Face image recognition method, device and storage medium | |
| CN109657537A (en) | Image-recognizing method, system and electronic equipment based on target detection | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| CN107633205A (en) | lip motion analysis method, device and storage medium | |
| CN111643014A (en) | Intelligent cleaning method and device, intelligent cleaning equipment and storage medium | |
| WO2022174523A1 (en) | Method for extracting gait feature of pedestrian, and gait recognition method and system | |
| CN108074571A (en) | Sound control method, system and the storage medium of augmented reality equipment | |
| CN110765924A (en) | Living body detection method and device and computer-readable storage medium | |
| CN114821759A (en) | Garbage throwing detection method, device, equipment and storage medium | |
| Yan et al. | Human-object interaction recognition using multitask neural network | |
| CN112150349A (en) | Image processing method and device, computer equipment and storage medium | |
| CN111582382B (en) | State identification method and device and electronic equipment | |
| Viswanatha et al. | Implementation of Smart Security System in Agriculture fields Using Embedded Machine Learning | |
| CN114694263B (en) | Action recognition method, device, equipment and storage medium | |
| CN107330361B (en) | Method for judging skin care manipulation of beauty equipment, beauty equipment and storage medium | |
| CN113705312A (en) | Hemoglobin detection method, device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |