CN112861716A - Illegal article placement monitoring method, system, equipment and storage medium - Google Patents
Illegal article placement monitoring method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN112861716A CN112861716A CN202110167710.3A CN202110167710A CN112861716A CN 112861716 A CN112861716 A CN 112861716A CN 202110167710 A CN202110167710 A CN 202110167710A CN 112861716 A CN112861716 A CN 112861716A
- Authority
- CN
- China
- Prior art keywords
- monitoring
- target
- illegal
- target detection
- placement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000003860 storage Methods 0.000 title claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 70
- 238000012545 processing Methods 0.000 claims abstract description 52
- 238000007781 pre-processing Methods 0.000 claims abstract description 41
- 239000000463 material Substances 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 21
- 238000012806 monitoring device Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 36
- 230000006399 behavior Effects 0.000 claims description 26
- 230000006870 function Effects 0.000 claims description 16
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 9
- 230000010354 integration Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000004873 anchoring Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 19
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention provides a method for monitoring illegal article placement, which comprises the following steps: s1, acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images; step S2, carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing; step S3, judging whether an illegal placing behavior of the goods exists or not according to the target detection result and whether the duration time of the illegal placing behavior of the goods is longer than the preset time, and if yes, generating a evidence obtaining material. The invention also provides a monitoring system, a monitoring device and a computer readable storage medium. Compared with the related technology, the network intelligence of the evidence obtaining work for the illegal placing behavior by adopting the technical scheme of the invention is good.
Description
[ technical field ] A method for producing a semiconductor device
The invention relates to the technical field of artificial intelligence, in particular to a method, a system and a device for monitoring illegal article placement and a computer-readable storage medium.
[ background of the invention ]
With the progress and development of society, various image pickup apparatuses have been popularized in our lives. In order to strengthen the construction of urban and community rail transit security work, valuable information is quickly obtained from monitoring image data obtained by edge monitoring equipment. In recent years, various intelligent products including an artificial intelligence technology as a core are gradually coming into the field of view of the public. The tendency of putting work and protecting driving for the society is great by the artificial intelligence technology, and as an important branch in the field of artificial intelligence, computer vision is mature day by day, in particular to a target detection technology based on deep learning.
However, the current target detection technology is generally applied to vehicle detection and fault location. The illegal articles are generally placed by law enforcement of urban security management personnel, and the network intelligence is not high.
Therefore, there is a need to provide a new method, system and device to solve the above technical problems.
[ summary of the invention ]
The invention aims to overcome the technical problems and provides a monitoring method, a monitoring system, monitoring equipment and a computer-readable storage medium for illegal article placement, which are good in network intelligence and are used for evidence obtaining work of illegal placement behaviors.
In order to achieve the above object, the present invention provides a method for monitoring illegal object placement, which comprises the following steps:
s1, acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images;
step S2, carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing; the method specifically comprises the following steps:
step S21, preprocessing the pre-arranged training set images through a preprocessing model to obtain training images;
step S22, training the training image as an input image of a convolutional neural network to obtain a target monitoring model;
step S23, taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through a main network of the target monitoring model to obtain a high-level feature map;
step S24, performing feature enhancement processing on the advanced feature map through the target monitoring model, and generating a one-dimensional vector;
step S25, the one-dimensional vector is subjected to prediction processing through the target detection module to generate and output a target detection result;
step S3, judging whether an illegal placing behavior of the goods exists or not according to the target detection result and whether the duration time of the illegal placing behavior of the goods is longer than the preset time, and if yes, generating a evidence obtaining material.
Preferably, the step S21 includes the following steps:
step S211, selecting four training set images, respectively carrying out zooming, rotating and color gamut changing on each training set image, respectively placing the four processed training set images in four directions, and combining the four processed training set images with an anchoring frame to generate a new image;
step S212, calculating the proportion of the new image to be zoomed, calculating the size of the new image zoomed and calculating the black edge filling value of the new image, and then processing the new image according to the calculation result and generating an input image of a convolutional neural network;
wherein the preprocessing model is used for processing the step S211 and the step S212.
Preferably, the step S23 includes the following steps:
s231, sequentially carrying out slicing operation, integration and splicing operation, convolution operation, batch normalization and activation of a Leaky _ relu function or a Mish function on the input image through a Focus structure, and generating a feature map;
step S232, forming one deep neural network from the plurality of CSP structures, and processing the feature map through the deep neural network to generate a high-level feature map.
Preferably, the step S24 includes the following steps:
step S241, forming an SPP structure by the pooling operation 13 × 13, the pooling operation 9 × 9 and the pooling operation 5 × 5, processing the high-level feature map by the SPP structure, and splicing the output features of the processed SPP structure to generate a first feature;
step S242, performing enhancement processing on the first features through a feature pyramid formed by an FPN structure to generate second features, and performing feature fusion processing on the second features through a PAN structure to generate the one-dimensional vector;
wherein the feature enhancement model is used for processing the step S241 and the step S242.
Preferably, the step S25 includes the following steps:
step S251, calculating the class probability and the loss of the target score of the one-dimensional vector through a binary cross entropy and a logs loss function, narrowing the difference between a predicted value and a true value through the calculated loss, screening the predicted value with the maximum probability and outputting the predicted value;
step S252, updating weights in a network layer when the deep neural network is trained by using Adam or SGD as a gradient optimization function;
step S253, generating and outputting the target detection result;
wherein the one-dimensional vector is a product of a sum of 52 classes, 1 probability, and 4 coordinates and a 3-anchor box, and the prediction model is used to process the step S251 and the step S252.
Preferably, the step S3 includes the following steps:
step S31, judging whether the target is in a static state according to the adjacent frames of the same target in the target frames, if so, entering step S32;
step S32, judging whether the target is located in an illegal area, if so, entering step S33;
step S33, determining whether the target of the first frame and the last frame in the target frames within a preset time is the same, and if so, determining that there is an illegal placement behavior of the article.
Preferably, after step S3, the method for monitoring placement of illegal articles further includes the following steps:
step S4, storing the evidence-obtaining material and giving an alarm; wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
The invention also provides a monitoring device, which comprises a processor and a memory, wherein the processor is used for reading the program in the memory and executing the steps in the illegal object placement monitoring method.
The invention also provides a computer-readable storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the steps in the method for monitoring placement of an illegal item as described in any one of the above.
The present invention also provides a monitoring system, comprising:
the video preprocessing module is used for acquiring a monitoring video and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images;
the target detection module is used for carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing, and specifically comprises the following steps: the image preprocessing module is used for preprocessing the pre-arranged training set images through a preprocessing model to obtain training images; training the training image as an input image of a convolutional neural network to obtain a target monitoring model; taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through a backbone network of the target monitoring model to obtain a high-level feature map; performing feature enhancement processing on the high-level feature map through the target monitoring model, and generating a one-dimensional vector; performing prediction processing on the one-dimensional vector through the target detection module to generate and output a target detection result;
the illegal article placement judging module is used for judging whether an article illegal placement behavior exists or not according to the target detection result and whether the duration time of the article illegal placement behavior is longer than the preset time, and if yes, generating a evidence obtaining material;
the evidence obtaining storage and alarm module is used for storing the evidence obtaining materials and giving an alarm; wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
Compared with the prior art, the method for monitoring illegal article placement comprises the following steps: acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images; carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing; and judging whether an illegal placing behavior of the object exists or not and judging whether the duration time of the illegal placing behavior of the object is longer than the preset time according to the target detection result, and if so, generating a evidence obtaining material. Therefore, the illegal article placement monitoring method can judge and alarm illegal placement of articles in a real scene, and evidence collection work of illegal placement behaviors is realized through 'cloud monitoring' and 'non-contact' law enforcement means. The target detection function is realized by adopting a deep convolutional neural network architecture, the target detection of the image is to position and classify and identify the target in the image, and the target in the image can be locked by utilizing the result of the target detection, so that the monitoring video can be favorably analyzed by workers. The requirements of the construction of a new generation of smart city are combined, the synchronous identification snapshot and intelligent association are carried out on the placement of the articles in the violation area, the resource force of video monitoring of each department and each unit is integrated, the interconnection and intercommunication of facilities and the integration and application of large image data are promoted, and the full city coverage, full network sharing, full time availability and full process controllability of video monitoring are realized. In summary, the monitoring method, the monitoring system, the monitoring device and the computer-readable storage medium of the present invention have good network intelligence for evidence collection of illegal placement behaviors.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive efforts,
wherein:
FIG. 1 is a block flow diagram of a method for monitoring illegal item placement according to the present invention;
FIG. 2 is a block flow diagram of step S21 of the illegal item placement monitoring method according to the present invention;
FIG. 3 is a block flow diagram of one embodiment of FIG. 2;
FIG. 4 is a block flow diagram of step S23 of the illegal placement monitoring method according to the present invention;
FIG. 5 is a block flow diagram of step S24 of the illegal placement monitoring method according to the present invention;
FIG. 6 is a schematic diagram of the FPN structure and PAN structure of FIG. 5;
FIG. 7 is a block flow diagram of step S25 of the illegal placement monitoring method according to the present invention;
FIG. 8 is a block flow diagram of one embodiment of FIG. 7;
FIG. 9 is a block flow diagram of step S3 of the illegal placement monitoring method according to the present invention;
FIG. 10 is a block flow diagram of one embodiment of FIG. 9;
FIG. 11 is a block diagram of a monitoring system according to the present invention;
fig. 12 is a block diagram of a illegal item placement determination module of the monitoring system according to the present invention;
fig. 13 is a block diagram of a forensic storage and alarm module of a monitoring system according to the present invention.
[ detailed description ] embodiments
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "comprising" and "having," and any variations thereof, in the description and claims of this application and the description of the figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Reference herein to "an embodiment or this implementation" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, the present invention provides a method for monitoring illegal placement of articles.
The method for monitoring the illegal article placement comprises the following steps:
the invention provides a method for monitoring illegal article placement, which comprises the following steps:
and step S1, acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images.
And step S2, carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing.
In this embodiment, in step S2, a deep convolutional neural network architecture is adopted to implement a target detection function, and the target detection of the image is to locate and classify and identify the target in the image, and the target in the image can be locked by using the result of the target detection, which is beneficial for the staff to perform analysis work on the monitored video.
The step S2 includes the following specific steps:
and step S21, preprocessing the pre-arranged training set images through a preprocessing model to obtain training images.
Referring to fig. 2, the step S21 includes the following steps:
step S211, selecting four training set images, respectively performing scaling, rotation, and color gamut changing on each training set image, and respectively placing the four processed training set images in four directions and combining the four training set images with an anchor frame to generate a new image.
Step S212, calculating the scale of the new image to be zoomed, calculating the size of the new image zoomed and calculating the black edge filling value of the new image, and then processing the new image according to the calculation result and generating the input image of the convolutional neural network.
Wherein the preprocessing model is used for processing the step S211 and the step S212.
Referring to fig. 3, fig. 3 is a block flow diagram of step S21 of the illegal item placement monitoring method according to an embodiment of the present invention.
In this embodiment, an example 210 of the step S21 includes the following steps:
step S2101, four images are read.
Step S2102 is to perform rotation scaling and color gamut adjustment change on the four images.
And step S2103, respectively placing the four images in four directions.
Step S2104 combines four images and combines the frames.
And step S2105, generating and outputting a new image.
Step S2106, a new image scaling is calculated.
Step S2107 calculates the scaled image size.
Step S2108 calculates the scaled image size.
And step S2109, calculating a black border filling value.
Step S2110, outputting the scaled fixed-size image.
And step S22, training the training image as an input image of the convolutional neural network to obtain a target monitoring model.
Step S23, taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through the backbone network of the target monitoring model to obtain a high-level feature map.
Referring to fig. 4, the step S23 includes the following steps:
and S231, sequentially carrying out slicing operation, integration and splicing operation, convolution operation, batch normalization and activation of a Leaky _ relu function or a Mish function on the input image through a Focus structure, and generating a feature map.
Step S232, forming one deep neural network from the plurality of CSP structures, and processing the feature map through the deep neural network to generate a high-level feature map.
And step S24, performing feature enhancement processing on the high-level feature map through the target monitoring model, and generating a one-dimensional vector.
Referring to fig. 5, the step S24 includes the following steps:
and step S241, forming an SPP structure by the pooling operation 13 × 13, the pooling operation 9 × 9 and the pooling operation 5 × 5, processing the high-level feature map by the SPP structure, and splicing the output features of the processed SPP structure to generate a first feature.
Step S242, performing enhancement processing on the first feature through a feature pyramid composed of FPN structures to generate a second feature, and performing feature fusion processing on the second feature through a PAN structure to generate the one-dimensional vector.
Wherein the feature enhancement model is used for processing the step S241 and the step S242.
Referring to fig. 6, fig. 6 is a schematic structural diagram of the FPN structure and the PAN structure in fig. 5.
And step S25, performing prediction processing on the one-dimensional vector by the target detection module to generate and output the target detection result.
Referring to fig. 7, the step S25 includes the following steps:
and step S251, calculating the class probability and the loss of the target score of the one-dimensional vector through the binary cross entropy and the logs loss function, narrowing the difference between a predicted value and a true value through the calculated loss, screening the predicted value with the maximum probability and outputting the predicted value.
And step S252, updating weights in the network layer when the deep neural network is trained by using Adam or SGD as a gradient optimization function.
And step S253, generating and outputting the target detection result.
Wherein the one-dimensional vector is a product of a sum of 52 classes, 1 probability, and 4 coordinates and a 3-anchor box, and the prediction model is used to process the step S251 and the step S252.
Referring to fig. 8, fig. 8 is a block flow diagram of an embodiment 250 of step S25 of the illegal item placement monitoring method according to the present invention.
In this embodiment, an example 250 of step S25 includes the following steps:
and step S2501, inputting the one-dimensional vector.
In step S2502, calculation processing is performed by a loss function.
Step S2503, non-local maximum suppression processing.
Step S2504, generates and outputs a predicted value. And the predicted value is the target detection result.
Step S3, judging whether an illegal placing behavior of the goods exists or not according to the target detection result and whether the duration time of the illegal placing behavior of the goods is longer than the preset time, and if yes, generating a evidence obtaining material.
Referring to fig. 9, the step S3 includes the following steps:
step S31, determining whether the target is in a static state according to the adjacent frames of the same target in the target frames, if yes, entering step S32.
And step S32, judging whether the target is located in the violation area, if so, entering step S33.
Step S33, determining whether the target of the first frame and the last frame in the target frames within a preset time is the same, and if so, determining that there is an illegal placement behavior of the article.
Referring to fig. 10, fig. 10 is a block flow diagram of step S25 of the illegal item placement monitoring method according to an embodiment of the present invention.
In the present embodiment, as shown in fig. 10, initially, a video stream (rstp format), target detection information (txt format), and violation region information (txt format) are input. And the generate forensic material phase outputs forensic information (txt format), alarm information (txt format), and forensic information (MP4 format).
In this embodiment, after step S3, the method for monitoring placement of an illegal item further includes the following steps:
and step S4, storing the evidence-obtaining material and giving an alarm.
Wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
In summary, the illegal article placement monitoring method completes discrimination and alarm of illegal placement of articles in a real scene, and achieves evidence collection work of illegal placement behaviors through 'cloud monitoring' and 'non-contact' law enforcement means. The system combines the requirements of the construction of a new generation of smart city, synchronously identifies, captures and intelligently associates the placement of articles in the violation area, integrates the video monitoring resource strength of each department and each unit, promotes the interconnection and intercommunication of facilities and the integration and application of image big data, and realizes the full-city coverage, full-network sharing, full-time availability and full-process controllability of video monitoring. Therefore, the monitoring method of the invention has good network intelligence for evidence collection of illegal putting behaviors.
Referring to fig. 11, the present invention further provides a monitoring system 100. The invention also provides a monitoring system which comprises a video preprocessing module 1, a target detection module 2, an illegal article placement judgment module 3 and a evidence obtaining storage and alarm module 4.
The video preprocessing module 1 is used for acquiring a monitoring video and preprocessing the monitoring video, wherein the preprocessing includes frame extraction to correspondingly acquire a plurality of target images.
The target detection module 2 is configured to perform target detection processing on the surveillance video and generate a target detection result according to the target detection processing, and specifically includes: the image preprocessing module is used for preprocessing the pre-arranged training set images through a preprocessing model to obtain training images; training the training image as an input image of a convolutional neural network to obtain a target monitoring model; taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through a backbone network of the target monitoring model to obtain a high-level feature map; performing feature enhancement processing on the high-level feature map through the target monitoring model, and generating a one-dimensional vector; and performing prediction processing on the one-dimensional vector through the target detection module 2 to generate and output a target detection result.
The illegal article placement determination module 3 is used for determining whether an illegal article placement behavior exists or not according to the target detection result and whether the duration time of the illegal article placement behavior is longer than the preset time, and if yes, generating a evidence obtaining material.
Referring to fig. 12, fig. 12 is a block diagram illustrating a structure of an illegal object placement determination module of a monitoring system according to the present invention.
In this embodiment, the illegal article placement determination module 3 includes an illegal region setting submodule 31, a video image transmission submodule 32, an image information transmission submodule 33, an illegal article screening submodule 34, an illegal article placement determination submodule 35, a novel forensic output submodule 36, and a forensic video output submodule 37. The above sub-modules are all functional modules commonly used in the field.
The evidence obtaining storage and alarm module 4 is used for storing the evidence obtaining materials and giving an alarm. Wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
Referring to fig. 13, fig. 13 is a block diagram of a forensic storage and alarm module of a monitoring system according to the present invention.
In this embodiment, the evidence obtaining storage and alarm module 4 includes an alarm information output sub-module 41, an evidence obtaining information storage sub-module 42, an evidence obtaining license plate storage sub-module 43, an evidence obtaining video storage sub-module 44, and an evidence obtaining video delivery sub-module 45. The above sub-modules are all functional modules commonly used in the field.
It should be noted that the video preprocessing module 1, the target detection module 2, the illegal object placement determination module 3, and the evidence obtaining storage and alarm module 4 are all modules and components commonly used in the art, and specific models need to be selected according to actual design needs of products, which are not described in detail herein.
The invention also provides a monitoring device, which comprises a processor and a memory, wherein the processor is used for reading the program in the memory and executing the steps in the illegal object placement monitoring method.
As will be understood by those skilled in the art, the monitoring device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable gate array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The memory includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of the monitoring device, such as a hard disk or a memory of the monitoring device. In other embodiments, the memory may also be an external storage device of the monitoring device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the monitoring device. Of course, the memory may also comprise both an internal memory unit of the monitoring device and an external memory device thereof. In this embodiment, the memory is generally used to store an operating system and various types of application software installed in the monitoring device, for example, a program code of a monitoring method for illegal item placement of the monitoring device. In addition, the memory may also be used to temporarily store various types of data that have been output or are to be output.
The processor may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor is typically used to control the overall operation of the monitoring device. In this embodiment, the processor is configured to execute the program code stored in the memory or process data, for example, execute the program code of the monitoring method for monitoring illegal item placement of the monitoring device.
The invention also provides a computer-readable storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the steps in the method for monitoring placement of an illegal item as described in any one of the above.
One of ordinary skill in the art will appreciate that all or part of the processes in the method for monitoring illegal object placement of the monitoring device according to the embodiments may be implemented by a computer program instructing associated hardware, and the program may be stored in a computer-readable storage medium, and when executed, may include processes such as those of the embodiments of the methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The present embodiment mentioned in the examples of the present invention is for convenience of description. The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Compared with the prior art, the method for monitoring illegal article placement comprises the following steps: acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images; carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing; and judging whether an illegal placing behavior of the object exists or not and judging whether the duration time of the illegal placing behavior of the object is longer than the preset time according to the target detection result, and if so, generating a evidence obtaining material. Therefore, the illegal article placement monitoring method can judge and alarm illegal placement of articles in a real scene, and evidence collection work of illegal placement behaviors is realized through 'cloud monitoring' and 'non-contact' law enforcement means. The target detection function is realized by adopting a deep convolutional neural network architecture, the target detection of the image is to position and classify and identify the target in the image, and the target in the image can be locked by utilizing the result of the target detection, so that the monitoring video can be favorably analyzed by workers. The requirements of the construction of a new generation of smart city are combined, the synchronous identification snapshot and intelligent association are carried out on the placement of the articles in the violation area, the resource force of video monitoring of each department and each unit is integrated, the interconnection and intercommunication of facilities and the integration and application of large image data are promoted, and the full city coverage, full network sharing, full time availability and full process controllability of video monitoring are realized. In summary, the monitoring method, the monitoring system, the monitoring device and the computer-readable storage medium of the present invention have good network intelligence for evidence collection of illegal placement behaviors.
While the foregoing is directed to embodiments of the present invention, it will be understood by those skilled in the art that various changes may be made without departing from the spirit and scope of the invention.
Claims (10)
1. A method for monitoring illegal article placement is characterized by comprising the following steps:
s1, acquiring a monitoring video, and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images;
step S2, carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing; the method specifically comprises the following steps:
step S21, preprocessing the pre-arranged training set images through a preprocessing model to obtain training images;
step S22, training the training image as an input image of a convolutional neural network to obtain a target monitoring model;
step S23, taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through a main network of the target monitoring model to obtain a high-level feature map;
step S24, performing feature enhancement processing on the advanced feature map through the target monitoring model, and generating a one-dimensional vector;
step S25, the one-dimensional vector is subjected to prediction processing through the target detection module to generate and output a target detection result;
step S3, judging whether an illegal placing behavior of the goods exists or not according to the target detection result and whether the duration time of the illegal placing behavior of the goods is longer than the preset time, and if yes, generating a evidence obtaining material.
2. The method for monitoring placement of illegal articles according to claim 1, wherein the step S21 comprises the following steps:
step S211, selecting four training set images, respectively carrying out zooming, rotating and color gamut changing on each training set image, respectively placing the four processed training set images in four directions, and combining the four processed training set images with an anchoring frame to generate a new image;
step S212, calculating the proportion of the new image to be zoomed, calculating the size of the new image zoomed and calculating the black edge filling value of the new image, and then processing the new image according to the calculation result and generating an input image of a convolutional neural network;
wherein the preprocessing model is used for processing the step S211 and the step S212.
3. The method for monitoring placement of illegal articles according to claim 1, wherein the step S23 comprises the following steps:
s231, sequentially carrying out slicing operation, integration and splicing operation, convolution operation, batch normalization and activation of a Leaky _ relu function or a Mish function on the input image through a Focus structure, and generating a feature map;
step S232, forming one deep neural network from the plurality of CSP structures, and processing the feature map through the deep neural network to generate a high-level feature map.
4. The method for monitoring placement of illegal articles according to claim 1, wherein the step S24 comprises the following steps:
step S241, forming an SPP structure by the pooling operation 13 × 13, the pooling operation 9 × 9 and the pooling operation 5 × 5, processing the high-level feature map by the SPP structure, and splicing the output features of the processed SPP structure to generate a first feature;
step S242, performing enhancement processing on the first features through a feature pyramid formed by an FPN structure to generate second features, and performing feature fusion processing on the second features through a PAN structure to generate the one-dimensional vector;
wherein the feature enhancement model is used for processing the step S241 and the step S242.
5. The method for monitoring placement of illegal articles according to claim 1, wherein the step S25 comprises the following steps:
step S251, calculating the class probability and the loss of the target score of the one-dimensional vector through a binary cross entropy and a logs loss function, narrowing the difference between a predicted value and a true value through the calculated loss, screening the predicted value with the maximum probability and outputting the predicted value;
step S252, updating weights in a network layer when the deep neural network is trained by using Adam or SGD as a gradient optimization function;
step S253, generating and outputting the target detection result;
wherein the one-dimensional vector is a product of a sum of 52 classes, 1 probability, and 4 coordinates and a 3-anchor box, and the prediction model is used to process the step S251 and the step S252.
6. The method for monitoring placement of illegal articles according to claim 1, wherein the step S3 comprises the following steps:
step S31, judging whether the target is in a static state according to the adjacent frames of the same target in the target frames, if so, entering step S32;
step S32, judging whether the target is located in an illegal area, if so, entering step S33;
step S33, determining whether the target of the first frame and the last frame in the target frames within a preset time is the same, and if so, determining that there is an illegal placement behavior of the article.
7. The method for monitoring placement of illegal items according to claim 1, wherein after step S3, the method for monitoring placement of illegal items further comprises the following steps:
step S4, storing the evidence-obtaining material and giving an alarm; wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
8. A monitoring system, the system comprising:
the video preprocessing module is used for acquiring a monitoring video and preprocessing the monitoring video, wherein the preprocessing comprises frame extraction to correspondingly acquire a plurality of target images;
the target detection module is used for carrying out target detection processing on the monitoring video and generating a target detection result according to the target detection processing, and specifically comprises the following steps: the image preprocessing module is used for preprocessing the pre-arranged training set images through a preprocessing model to obtain training images; training the training image as an input image of a convolutional neural network to obtain a target monitoring model; taking the target image obtained by frame extraction in the monitoring video as the input of the target monitoring model, and performing feature extraction on the target image through a backbone network of the target monitoring model to obtain a high-level feature map; performing feature enhancement processing on the high-level feature map through the target monitoring model, and generating a one-dimensional vector; performing prediction processing on the one-dimensional vector through the target detection module to generate and output a target detection result;
the illegal article placement judging module is used for judging whether an article illegal placement behavior exists or not according to the target detection result and whether the duration time of the article illegal placement behavior is longer than the preset time, and if yes, generating a evidence obtaining material;
the evidence obtaining storage and alarm module is used for storing the evidence obtaining materials and giving an alarm; wherein, the material of collecting evidence is including the license plate of collecting evidence, the video of collecting evidence and the information of collecting evidence, warning notice is including alarm information and the video of collecting evidence.
9. A monitoring device comprising a processor and a memory, the processor being configured to read a program in the memory and execute the steps of the method of monitoring placement of an offending item as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores a computer program comprising program instructions which, when executed by a processor, implement the steps in the method for monitoring the presence of an offending item as claimed in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110167710.3A CN112861716A (en) | 2021-02-05 | 2021-02-05 | Illegal article placement monitoring method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110167710.3A CN112861716A (en) | 2021-02-05 | 2021-02-05 | Illegal article placement monitoring method, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112861716A true CN112861716A (en) | 2021-05-28 |
Family
ID=75988925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110167710.3A Pending CN112861716A (en) | 2021-02-05 | 2021-02-05 | Illegal article placement monitoring method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112861716A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993047A (en) * | 2017-12-28 | 2019-07-09 | 杭州海康威视系统技术有限公司 | City huddles violation recognition methods, device and the electronic equipment of material |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
| CN111428644A (en) * | 2020-03-25 | 2020-07-17 | 北京以萨技术股份有限公司 | Zebra crossing region monitoring method, system and medium based on deep neural network |
| CN111539359A (en) * | 2020-04-28 | 2020-08-14 | 浙江工商大学 | Illegal parking detection method based on deep learning |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| CN111597945A (en) * | 2020-05-11 | 2020-08-28 | 济南博观智能科技有限公司 | Target detection method, device, equipment and medium |
| CN112149511A (en) * | 2020-08-27 | 2020-12-29 | 深圳市点创科技有限公司 | Method, terminal and device for detecting violation of driver based on neural network |
| CN112287899A (en) * | 2020-11-26 | 2021-01-29 | 山东捷讯通信技术有限公司 | Unmanned aerial vehicle aerial image river drain detection method and system based on YOLO V5 |
-
2021
- 2021-02-05 CN CN202110167710.3A patent/CN112861716A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993047A (en) * | 2017-12-28 | 2019-07-09 | 杭州海康威视系统技术有限公司 | City huddles violation recognition methods, device and the electronic equipment of material |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| CN111428644A (en) * | 2020-03-25 | 2020-07-17 | 北京以萨技术股份有限公司 | Zebra crossing region monitoring method, system and medium based on deep neural network |
| CN111539359A (en) * | 2020-04-28 | 2020-08-14 | 浙江工商大学 | Illegal parking detection method based on deep learning |
| CN111597945A (en) * | 2020-05-11 | 2020-08-28 | 济南博观智能科技有限公司 | Target detection method, device, equipment and medium |
| CN112149511A (en) * | 2020-08-27 | 2020-12-29 | 深圳市点创科技有限公司 | Method, terminal and device for detecting violation of driver based on neural network |
| CN112287899A (en) * | 2020-11-26 | 2021-01-29 | 山东捷讯通信技术有限公司 | Unmanned aerial vehicle aerial image river drain detection method and system based on YOLO V5 |
Non-Patent Citations (2)
| Title |
|---|
| WILLIAM: "(1 封私信) 如何评价YOLOv5？", 《HTTPS://WWW.ZHIHU.COM/QUESTION/399884529/ANSWER/1374024055?UTM_ID=0》, pages 1 - 14 * |
| 新智元: "深入浅出Yolo系列之Yolov5核心基础知识完整讲解", 《HTTPS://ZHUANLAN.ZHIHU.COM/P/260400612》, pages 1 - 19 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390262B (en) | Video analysis method, device, server and storage medium | |
| Chi et al. | Automated object identification using optical video cameras on construction sites | |
| CN106878670B (en) | A kind of method for processing video frequency and device | |
| CN110650316A (en) | Intelligent patrol and early warning processing method and device, electronic equipment and storage medium | |
| CN107105207A (en) | Target monitoring method, target monitoring device and video camera | |
| CN107705574A (en) | A kind of precisely full-automatic capturing system of quick road violation parking | |
| CN104200466B (en) | A kind of method for early warning and video camera | |
| KR102217253B1 (en) | Apparatus and method for analyzing behavior pattern | |
| CN106791655B (en) | A kind of method for processing video frequency and device | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| CN109766779A (en) | It hovers personal identification method and Related product | |
| CN110659391A (en) | Video detection method and device | |
| CN111666821B (en) | Method, device and equipment for detecting personnel aggregation | |
| CN114332702A (en) | Target area detection method and device, storage medium and electronic equipment | |
| CN115002414A (en) | Monitoring method, monitoring device, server and computer readable storage medium | |
| CN110852179A (en) | Method for detecting suspicious personnel intrusion based on video monitoring platform | |
| CN115294528A (en) | Pedestrian safety monitoring method and device | |
| CN102496024B (en) | Method for detecting incident triggered by characteristic frame in intelligent monitor | |
| CN114926791A (en) | Method and device for detecting abnormal lane change of vehicles at intersection, storage medium and electronic equipment | |
| Filonenko et al. | Illegally parked vehicle detection using adaptive dual background model | |
| CN102244769A (en) | Object and key person monitoring system and method thereof | |
| CN106713864B (en) | A kind of method for processing video frequency and device | |
| CN113191273A (en) | Oil field well site video target detection and identification method and system based on neural network | |
| CN117237935A (en) | Method and device for identifying space object, electronic equipment and storage medium | |
| CN112861716A (en) | Illegal article placement monitoring method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |