CN112861711A - Regional intrusion detection method and device, electronic equipment and storage medium - Google Patents
Regional intrusion detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112861711A CN112861711A CN202110164424.1A CN202110164424A CN112861711A CN 112861711 A CN112861711 A CN 112861711A CN 202110164424 A CN202110164424 A CN 202110164424A CN 112861711 A CN112861711 A CN 112861711A
- Authority
- CN
- China
- Prior art keywords
- detection
- target
- frame set
- detection frame
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 562
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000012216 screening Methods 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims description 36
- 238000012545 processing Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 9
- 239000000203 mixture Substances 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims 1
- 230000009467 reduction Effects 0.000 description 14
- 238000013528 artificial neural network Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 10
- 238000011176 pooling Methods 0.000 description 8
- 238000003062 neural network model Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000009545 invasion Effects 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Geometry (AREA)
- Bioinformatics & Computational Biology (AREA)
Abstract
The embodiment of the invention provides a method, a device and electronic equipment for detecting regional intrusion, wherein the method comprises the following steps: acquiring video data, wherein the video data comprises continuous multi-frame images; respectively detecting target people through a first detection model and a second detection model based on continuous multi-frame images to obtain a corresponding first detection result and a second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set; extracting feature information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set; screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person; and judging whether the corresponding target personnel invade the preset area or not based on the target detection frame. The accuracy rate of the intrusion detection of the pedestrian area is improved.
Description
Technical Field
The present invention relates to the field of artificial intelligence, and in particular, to a method and an apparatus for detecting intrusion in a region, an electronic device, and a storage medium.
Background
In recent years, with the development of deep learning technology in the field of artificial intelligence and the improvement of computing power of related hardware equipment, computer vision-based applications are falling to the ground and further developed in various fields. For example, the area intrusion detection technology is increasingly widely applied in the security field, and accurate and efficient pedestrian detection and identification results have important auxiliary functions in assisting security personnel to patrol, improving human-computer interaction experience and the like.
In the prior art, a detection model based on a convolutional neural network has a good effect on solving the problem of image classification and identification, but for continuous detection of pedestrian regional intrusion, accurate prediction and further tracking processing are required to be performed on the position and the size of each pedestrian while the existence of the pedestrian in an image is judged so as to prompt the pedestrian in time, the existing detection model can have the condition of inaccurate frame prediction, and the re-identification and tracking are difficult to perform after a detection target is lost, so that the accuracy of pedestrian regional intrusion detection is not high.
Disclosure of Invention
The embodiment of the invention provides a regional intrusion detection method, which can improve the accuracy of pedestrian regional intrusion detection.
In a first aspect, an embodiment of the present invention provides a method for detecting an area intrusion, including:
acquiring video data, wherein the video data comprises continuous multi-frame images, and each frame of image comprises at least one target person;
respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set;
extracting feature information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set;
screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person;
and judging whether the corresponding target personnel invade a preset area or not based on the target detection frame.
Optionally, the screening the first detection frame set by the second detection frame set to obtain a target detection frame set includes:
overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set, and calculating the overlapping area;
if the overlapping area is larger than a preset overlapping threshold value, putting the corresponding detection frame in the second detection frame set into the target detection frame set;
otherwise, putting the corresponding detection frame in the first detection frame set into the target detection frame set.
Optionally, the preset matching algorithm includes a hungarian algorithm, and the matching the feature information of the target person with the target detection box set through the preset matching algorithm to obtain the target detection box corresponding to the target person includes:
carrying out graph structure processing on the characteristic information of the target person and the target detection frame set to obtain graph structure combined data;
and matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target personnel.
Optionally, after the feature information of the target person is extracted through a third network model based on the continuous multi-frame image and the first detection frame set, before the first detection frame set is screened through the second detection frame set to obtain a target detection frame set, the method further includes:
and carrying out de-duplication and filtering processing on the first detection frame set and the second detection frame set.
Optionally, the first detection model and the third detection model are constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and are pre-trained by corresponding training data sets.
Optionally, the second detection model is constructed by a gaussian mixture background modeling method.
Optionally, the pre-training of the first detection model and the third detection model includes:
acquiring video data;
acquiring a multi-frame image from the video data, acquiring a pedestrian from the image and marking the pedestrian to obtain a pedestrian data set;
dividing the pedestrian data set into a training set, a verification set and a test set;
and constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
In a second aspect, an embodiment of the present invention provides an area intrusion detection apparatus, including:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring video data, and the video data comprises continuous multi-frame images which comprise at least one target person;
the detection module is used for respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set;
the extracting module is used for extracting the characteristic information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set;
the screening and matching module is used for screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person;
and the judging module is used for judging whether the corresponding target personnel invade the preset area or not based on the target detection frame.
Optionally, the screening and matching module includes:
the overlapping submodule is used for overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set and calculating the overlapping area;
the first input sub-module is used for inputting the corresponding detection frame in the second detection frame set into the target detection frame set if the overlapping area is larger than a preset overlapping threshold value;
and the second putting sub-module is used for putting the corresponding detection frame in the first detection frame set into the target detection frame set if the detection frame is not in the target detection frame set.
Optionally, the screening and matching module further includes:
the processing submodule is used for carrying out graph structure processing on the characteristic information of the target personnel and the target detection frame set to obtain graph structure combined data;
and the matching submodule is used for matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target personnel.
Optionally, after the extracting module, the apparatus further includes:
and the de-duplication and filtering processing module is used for carrying out de-duplication and filtering processing on the first detection frame set and the second detection frame set.
Optionally, the first detection model and the third detection model are constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and are pre-trained by corresponding training data sets.
Optionally, the second detection model is constructed by a gaussian mixture background modeling method.
Optionally, the apparatus further includes a pre-training module of the first detection model and the third detection model, where the pre-training module includes:
the first obtaining submodule is used for obtaining video data;
the second acquisition submodule is used for acquiring multi-frame images from the video data, acquiring pedestrians from the images and marking the pedestrians to obtain a pedestrian data set;
the dividing submodule is used for dividing the pedestrian data set into a training set, a verification set and a test set;
and the training submodule is used for constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
In a third aspect, an embodiment of the present invention provides an electronic device, including: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps in the area intrusion detection method provided by the embodiment of the invention.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps in the area intrusion detection method provided by the embodiment of the present invention.
In the embodiment of the invention, video data is acquired, wherein the video data comprises continuous multi-frame images, and the images comprise at least one target person; respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set; extracting feature information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set; screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person; and judging whether the corresponding target personnel invade a preset area or not based on the target detection frame. The method comprises the steps of obtaining continuous multi-frame images of video data including at least one target pedestrian, using a first detection model and a second detection model to respectively detect and obtain a first detection frame set and a second detection frame set of the target person from the continuous multi-frame images, screening the first detection frame set through the second detection frame set, obtaining a more accurate human body detection frame as the target detection frame set, then extracting feature information of the target person from the continuous multi-frame images through a third detection model, matching the feature information with the target detection frame set to obtain a final target detection frame, continuously identifying and tracking the target person corresponding to corresponding feature information through the target detection frame, judging whether the corresponding target person invades a preset area, and accordingly improving accuracy of pedestrian invasion area detection.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for detecting an intrusion into a region according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a first detection model according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a third detection model according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for screening test frames according to an embodiment of the present invention;
FIG. 5 is a flow chart of a matching method provided by an embodiment of the invention;
fig. 6 is a schematic structural diagram of an area intrusion detection apparatus according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a screening and matching module according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of another screening and matching module provided in an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a method for detecting an intrusion into a region according to an embodiment of the present invention, as shown in fig. 1, including the following steps:
101. video data is acquired, the video data comprises a plurality of continuous frames of images, and each frame of image comprises at least one target person.
In the embodiment of the invention, the regional intrusion detection method can be applied to a pedestrian regional intrusion identification detection scene under video monitoring. The electronic device on which the provided regional intrusion detection method operates can acquire the video data and transmit the data in a wired connection mode or a wireless connection mode. It should be noted that the Wireless connection manner may include, but is not limited to, a 3G/4G connection, a WiFi (Wireless-Fidelity) connection, a bluetooth connection, a wimax (Worldwide Interoperability for Microwave Access) connection, a Zigbee (low power local area network protocol) connection, a uwb (ultra wideband) connection, and other Wireless connection manners known now or developed in the future.
The video data can be shot, collected and transmitted in real time through video collecting equipment, can also be actively uploaded from a terminal through manpower, and the like, and then the obtained image data is stored or directly input into electronic equipment on which a regional intrusion detection method is operated for detection. The video acquisition equipment can comprise a camera and electronic equipment which is provided with the camera and can acquire video images. The video includes a plurality of frames of continuous image data, or may be a plurality of frames of image sequences acquired at certain intervals, where the plurality of frames of continuous image data may be an image sequence including at least one target person and requiring pedestrian area intrusion detection, or may include a plurality of target persons, for example, 2 or 3 target persons.
102. On the basis of continuous multi-frame images, the target person is detected through the first detection model and the second detection model respectively, and corresponding first detection results and second detection results are obtained, wherein the first detection results comprise a first detection frame set, and the second detection results comprise a second detection frame set.
In the embodiment of the present invention, target detection may be performed on target persons from the consecutive multiple frames of images through a first detection model and a second detection model, and human detection frames of all target persons in each frame of image are correspondingly output, that is, the first detection frame set including all target persons of all frame of image and the second detection frame set including all target persons of all frame of image. The first detection model and/or the second detection model may be a trained neural network model capable of performing target recognition and positioning, such as an existing fast R-CNN, YOLO, or the like, or may be a self-designed neural network model, or may be a non-neural network model; at least one target person is detected from the multi-frame images through the first detection model and the second detection model, and a human body detection frame (namely, rectangular frame coordinates of the target person in the images) of the corresponding target person can be obtained.
Further, the first detection model is constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and is pre-trained by a corresponding training data set. Specifically, referring to fig. 2, fig. 2 is a schematic structural diagram of a first detection model according to an embodiment of the present invention, as shown in fig. 2, the first detection model includes a multi-layer neural network, and a network structure of the multi-layer neural network includes an input image layer, a standard convolution module, and three feature dimension reduction modules, where an input image of the input image layer has a size of 416 × 416 RGB images, and then the number of channels is first increased to 64 through the standard convolution module with a convolution kernel of 3 × 3, the standard convolution module includes two sets of standard convolution structures, a last two-dimensional convolution of a first set of standard convolution structures of the standard convolution module increases the channels to 32, and a last two-dimensional convolution of a second set of standard convolution structures increases the channels to 64, so that a feature size is 104 × 104; after the standard convolution module, three characteristic dimension reduction modules are continuously arranged, each characteristic dimension reduction module is composed of eight groups of grouping convolutions, 1 × 1 standard convolutions and average pooling layers to form a residual convolution structure, and is connected with a two-dimensional convolution used for increasing the number of channels, and feature graphs (the scales of the feature graphs are respectively 52 × 52, 26 × 26 and 13 × 13) output by the last three characteristic dimension reduction modules are collected and output after channel mixing and superposition, and the number of model parameters constructed by the network structure is small, so that the operation speed of the model can be increased; through the network structure of the first detection model, more detailed pedestrian characteristics can be extracted, and the accuracy rate of pedestrian detection is increased.
The second detection model is constructed by a mixed Gaussian background modeling method. The detection problem of moving objects is mainly divided into two categories, camera fixation and camera motion. In the field of intelligent security, a camera is basically a fixed point location, so that the mixed Gaussian background modeling is very suitable for separating a background and a foreground from a video image sequence under the condition, and suppressing a non-pedestrian target in the foreground to obtain a more accurate boundary frame comprising a plurality of target personnel outlines of a plurality of frames of images, and the boundary frame is used as the second detection frame set.
103. And extracting the characteristic information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set.
In the embodiment of the present invention, a corresponding target person may be obtained from the continuous multiple frames of images through the first detection frame set output by the first detection model, and then feature information of the obtained target person, for example, face information, may be extracted through a third detection model. The third detection model can be constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and is pre-trained by a corresponding training data set. Specifically, referring to fig. 3, fig. 3 is a schematic structural diagram of a third detection model according to an embodiment of the present invention, as shown in fig. 3, the third detection model includes a multi-layer neural network, and a network structure of the third detection model includes an input image layer, a standard convolution module, two feature dimension reduction modules, and a global mean pooling layer, where the input image of the input image layer is a target person intercepted by the first detection module, each target person is normalized to an image size of 128 × 64 in width, promoted to 256 by the standard convolution module, promoted to 1024 dimensions by the two lightweight feature dimension reduction modules, then one feature is extracted from each feature channel by the global mean pooling layer, and the target person features of 1024 dimensions are summed up, and each feature dimension reduction module is composed of two groups of convolution, a residual convolution structure composed of a standard convolution of 1 × 1 and a mean pooling layer, the two-dimensional convolution for increasing the number of channels is connected at the back, the number of parameters of the model constructed by the network structure is small, and the running speed of the model can be increased; and finally, the target person features are distributed to two branches, one branch utilizes cross entropy cross Encopy loss to train the target person feature classification, the other branch utilizes triple loss to shorten the distances of all features of the target person features with consistent features, and the distances of the features of the target persons with different features are pushed far, so that all feature information of different target persons is separated.
It should be noted that, the first detection model and the third detection model are trained in advance, and the pre-training step may include:
acquiring video data;
acquiring a multi-frame image from the video data, acquiring a pedestrian from the image and marking the pedestrian to obtain a pedestrian data set;
dividing the pedestrian data set into a training set, a verification set and a test set;
and constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model. And then, verifying and testing the trained model through the verification set and the test set until the model meets the requirements.
104. And screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person.
Optionally, referring to fig. 4, fig. 4 is a flowchart of a method for screening detection frames according to an embodiment of the present invention, and as shown in fig. 4, the screening the first detection frame set by the second detection frame set to obtain the target detection frame set includes the following steps:
401. overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set, and calculating the overlapping area;
402. if the overlapping area is larger than a preset overlapping threshold value, putting the corresponding detection frame in the second detection frame set into the target detection frame set;
403. otherwise, putting the corresponding detection frame in the first detection frame set into the target detection frame set.
In an embodiment of the present invention, the second detection frame set is a set of multiple target person human body detection frames including multiple frames of images output by the second detection model constructed by the gaussian mixture background modeling method, and the first detection frame set is a set of multiple target person human body detection frames including multiple frames of images output by the first detection model constructed based on the neural network. Then performing or operation (i.e. taking the union of the two detection frames) on the human detection frames in the second detection frame set and the human detection frames in the first detection frame set, calculating the overlapping area IoU of the two detection frames (i.e. taking the intersection of the two detection frames), if the overlapping area IoU of a certain human detection frame is larger than a preset overlapping threshold, deleting the corresponding human detection frame in the first detection frame set, only reserving the corresponding human detection frame in the second detection frame set, otherwise, if the overlapping area IoU of a certain human detection frame is smaller than the preset overlapping threshold, deleting the corresponding human detection frame in the second detection frame set, only reserving the corresponding human detection frame in the first detection frame set, then putting the reserved human detection frame into a target detection frame set until the last human detection frame in the second detection frame set and/or the first detection frame set, the screening is ended. The screening method which integrates the first detection model based on the neural network and the second detection model based on the mixed Gaussian background modeling can greatly improve the accuracy of pedestrian target detection.
Optionally, referring to fig. 5, fig. 5 is a flowchart of a matching method provided in an embodiment of the present invention, as shown in fig. 5, where the preset matching algorithm includes a hungarian algorithm, and the matching the feature information of the target person with the target detection box set by using the preset matching algorithm to obtain the target detection box corresponding to the target person includes the following steps:
501. carrying out graph structure processing on the characteristic information of the target personnel and the target detection frame set to obtain graph structure combined data;
502. and matching the combined data of the graph structures through a Hungarian algorithm to obtain a target detection box of the target personnel.
In the embodiment of the present invention, feature information, such as face information, of all target persons corresponding to all target person human body detection boxes in the first detection box set may be extracted from the multi-frame image through the third detection model and the first detection box set, then the target detection box set and the feature information of all target persons are subjected to graph structure processing to obtain graph structure combined data, the graph structure combined data may be specifically processed into a bipartite graph, that is, the feature information of the target persons may be regarded as one point, the feature information of all target persons forms one point set of the bipartite graph, all target person human body detection boxes in the target detection box set form another point set of the bipartite graph, then the two point sets may be matched by using the hungarian algorithm to obtain a plurality of continuous target detection boxes corresponding to each target person, therefore, the corresponding target personnel can be continuously tracked through the target detection frame, and the accuracy of pedestrian tracking is improved.
105. And judging whether the corresponding target personnel invade the preset area or not based on the target detection frame.
In the embodiment of the present invention, the preset area may be an artificially set area, a dangerous area, or another area where entry is prohibited, and may be defined in a video image acquired by an image pickup device, then the corresponding target person is continuously tracked by the multiple continuous target detection frames corresponding to each target person obtained through the above steps, and whether the corresponding target person enters the preset area is determined by an overlapping area between the target detection frame and the preset area, and a corresponding prompt is performed, for example, when the overlapping area between the target detection frame and the preset area is greater than zero, a corresponding warning message is sent to remind the relevant person. Furthermore, the overlapping area can be constructed to classify the intrusion level of the alarm information, and the larger the overlapping area is, the deeper the pedestrian invades, the higher the intrusion level is, and the stronger the alarm information can be.
In summary, the embodiment of the invention obtains the first detection frame set and the second detection frame set of the target person from the continuous multi-frame images by obtaining the continuous multi-frame images including the video data of at least one target pedestrian, using the first detection model and the second detection model to respectively detect and obtain the first detection frame set and the second detection frame set of the target person from the continuous multi-frame images, and the first detection frame set is subjected to detection frame screening through the second detection frame set, so that more accurate human body detection frames can be obtained as a target detection frame set, then extracting the characteristic information of the target person from the continuous multi-frame images through a third detection model, matching the characteristic information with the target detection frame set to obtain a final target detection frame, the target personnel corresponding to the corresponding characteristic information can be continuously identified and tracked through the target detection frame, whether the corresponding target personnel invade the preset area or not is judged, and therefore the accuracy of pedestrian area invasion detection is improved.
It should be noted that the method for detecting regional intrusion provided by the embodiment of the present invention may be applied to devices such as a mobile phone, a monitor, a computer, and a server that can perform regional intrusion detection.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an area intrusion detection apparatus according to an embodiment of the present invention, and as shown in fig. 6, the area intrusion detection apparatus 600 includes:
the acquiring module 601 is configured to acquire video data, where the video data includes a plurality of consecutive frames of images, and the images include at least one target person.
In the embodiment of the invention, the area intrusion detection device can be applied to a pedestrian area intrusion identification detection scene under video monitoring. The electronic equipment on which the provided regional intrusion detection device operates can acquire the video data and transmit the data in a wired connection mode or a wireless connection mode. It should be noted that the Wireless connection manner may include, but is not limited to, a 3G/4G connection, a WiFi (Wireless-Fidelity) connection, a bluetooth connection, a wimax (Worldwide Interoperability for Microwave Access) connection, a Zigbee (low power local area network protocol) connection, a uwb (ultra wideband) connection, and other Wireless connection manners known now or developed in the future.
The video data can be shot, collected and transmitted in real time through video collecting equipment, can also be actively uploaded from a terminal through manpower, and the like, and then the obtained image data is stored or directly input into the electronic equipment on which the regional intrusion detection device operates to detect. The video acquisition equipment can comprise a camera and electronic equipment which is provided with the camera and can acquire video images. The video includes a plurality of frames of continuous image data, or may be a plurality of frames of image sequences acquired at certain intervals, where the plurality of frames of continuous image data may be an image sequence including at least one target person and requiring pedestrian area intrusion detection, or may include a plurality of target persons, for example, 2 or 3 target persons.
The detection module 602 is configured to detect the target person through a first detection model and a second detection model respectively based on the consecutive multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, where the first detection result includes a first detection frame set, and the second detection result includes a second detection frame set.
In the embodiment of the present invention, target detection may be performed on target persons from the consecutive multiple frames of images through a first detection model and a second detection model, and human detection frames of all target persons in each frame of image are correspondingly output, that is, the first detection frame set including all target persons of all frame of image and the second detection frame set including all target persons of all frame of image. The first detection model and/or the second detection model may be a trained neural network model capable of performing target recognition and positioning, such as an existing fast R-CNN, YOLO, or the like, or may be a self-designed neural network model, or may be a non-neural network model; at least one target person is detected from the multi-frame images through the first detection model and the second detection model, and a human body detection frame (namely, rectangular frame coordinates of the target person in the images) of the corresponding target person can be obtained.
Further, the first detection model is constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and is pre-trained by a corresponding training data set. Specifically, referring to fig. 2, the first detection model includes a multi-layer neural network, and a network structure of the multi-layer neural network includes an input image layer, a standard convolution module and three feature dimension reduction modules, wherein the size of an input image of the input image layer is 416 × 416, then the number of channels is first increased to 64 by the standard convolution module with a convolution kernel of 3 × 3, the standard convolution module specifically includes two groups of standard convolution structures, the last two-dimensional convolution of the first group of standard convolution structures of the standard convolution module increases the number of channels to 32, and the last two-dimensional convolution of the second group of standard convolution structures increases the number of channels to 64, so that the size of the feature map becomes 104 × 104; after the standard convolution module, three characteristic dimension reduction modules are continuously arranged, each characteristic dimension reduction module is composed of eight groups of grouping convolutions, 1 × 1 standard convolutions and average pooling layers to form a residual convolution structure, and is connected with a two-dimensional convolution used for increasing the number of channels, and feature graphs (the scales of the feature graphs are respectively 52 × 52, 26 × 26 and 13 × 13) output by the last three characteristic dimension reduction modules are collected and output after channel mixing and superposition, and the number of model parameters constructed by the network structure is small, so that the operation speed of the model can be increased; through the network structure of the first detection model, more detailed pedestrian characteristics can be extracted, and the accuracy rate of pedestrian detection is increased.
Optionally, the second detection model is constructed by a gaussian mixture background modeling method. The detection problem of moving objects is mainly divided into two categories, camera fixation and camera motion. In the field of intelligent security, a camera is basically a fixed point location, so that the mixed Gaussian background modeling is very suitable for separating a background and a foreground from a video image sequence under the condition, and suppressing a non-pedestrian target in the foreground to obtain a more accurate boundary frame comprising a plurality of target personnel outlines of a plurality of frames of images, and the boundary frame is used as the second detection frame set.
An extracting module 603, configured to extract, based on the consecutive multi-frame images and the first detection frame set, feature information of the target person through a third detection model.
In the embodiment of the present invention, a corresponding target person may be obtained from the continuous multiple frames of images through the first detection frame set output by the first detection model, and then feature information of the obtained target person, for example, face information, may be extracted through a third detection model.
Optionally, the third detection model may be constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and pre-trained by a corresponding training data set. Specifically, referring to fig. 3, the third detection model includes a multi-layer neural network, and the network structure includes an input image layer, a standard convolution module, two feature dimension reduction modules, and a global mean pooling layer, where the input image of the input image layer is the target person intercepted by the first detection module, each target person is normalized to an image size 128 × 64 in width, and is promoted to 256 by the standard convolution module, and then the dimension is promoted to 1024 by the two lightweight feature dimension reduction modules, and then a feature is extracted from each feature channel by the global mean pooling layer, and the target person features of 1024 dimensions are counted, and each feature dimension reduction module is composed of two groups of convolutions, a residual convolution structure composed of 1 × 1 standard convolution and averaging pooling layer, and a two-dimensional convolution for increasing the number of channels is connected to the subsequent two-dimensional convolution, and the number of model parameters constructed by the network structure is small, the running speed of the model can be improved; and finally, the target person features are distributed to two branches, one branch utilizes cross entropy cross Encopy loss to train the target person feature classification, the other branch utilizes triple loss to shorten the distances of all features of the target person features with consistent features, and the distances of the features of the target persons with different features are pushed far, so that all feature information of different target persons is separated.
It should be noted that, the first detection model and the third detection model are trained in advance, and correspondingly, the area intrusion detection apparatus further includes a pre-training module of the first detection model and the third detection model, where the pre-training module includes:
the first obtaining submodule is used for obtaining video data;
the second acquisition submodule is used for acquiring multi-frame images from the video data, acquiring pedestrians from the images and marking the pedestrians to obtain a pedestrian data set;
the dividing submodule is used for dividing the pedestrian data set into a training set, a verification set and a test set;
and the training submodule is used for constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
And a screening and matching module 604, configured to screen the first detection frame set through the second detection frame set to obtain a target detection frame set, and match the feature information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person.
Optionally, as shown in fig. 7, the screening and matching module 604 includes:
an overlap sub-module 6041, configured to overlap the detection frames in the second detection frame set and the detection frames in the first detection frame set, and calculate an overlap area;
a first putting sub-module 6042, configured to put a corresponding detection frame in the second detection frame set into the target detection frame set if the overlapping area is greater than a preset overlapping threshold;
and a second placing sub-module 6043, configured to place the corresponding detection frame in the first detection frame set into the target detection frame set otherwise.
Optionally, after the extracting module 603, the apparatus further includes a de-duplication and filtering processing module, configured to perform de-duplication and filtering processing on the first detection frame set and the second detection frame set.
In an embodiment of the present invention, the second detection frame set is a set of multiple target person human body detection frames including multiple frames of images output by the second detection model constructed by the gaussian mixture background modeling method, and the first detection frame set is a set of multiple target person human body detection frames including multiple frames of images output by the first detection model constructed based on the neural network. Then performing or operation (i.e. taking the union of the two detection frames) on the human detection frames in the second detection frame set and the human detection frames in the first detection frame set, calculating the overlapping area IoU of the two detection frames (i.e. taking the intersection of the two detection frames), if the overlapping area IoU of a certain human detection frame is larger than a preset overlapping threshold, deleting the corresponding human detection frame in the first detection frame set, only reserving the corresponding human detection frame in the second detection frame set, otherwise, if the overlapping area IoU of a certain human detection frame is smaller than the preset overlapping threshold, deleting the corresponding human detection frame in the second detection frame set, only reserving the corresponding human detection frame in the first detection frame set, then putting the reserved human detection frame into a target detection frame set until the last human detection frame in the second detection frame set and/or the first detection frame set, the screening is ended. The screening method which integrates the first detection model based on the neural network and the second detection model based on the mixed Gaussian background modeling can greatly improve the accuracy of pedestrian target detection.
Optionally, as shown in fig. 8, the screening and matching module 604 further includes:
the processing submodule 6044 is configured to perform graph structure processing on the feature information of the target person and the target detection box set to obtain graph structure joint data;
and the matching submodule 6045 is used for matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target person.
In the embodiment of the present invention, feature information, such as face information, of all target persons corresponding to all target person human body detection boxes in the first detection box set may be extracted from the multi-frame image through the third detection model and the first detection box set, then the target detection box set and the feature information of all target persons are subjected to graph structure processing to obtain graph structure combined data, the graph structure combined data may be specifically processed into a bipartite graph, that is, the feature information of the target persons may be regarded as one point, the feature information of all target persons forms one point set of the bipartite graph, all target person human body detection boxes in the target detection box set form another point set of the bipartite graph, then the two point sets may be matched by using the hungarian algorithm to obtain a plurality of continuous target detection boxes corresponding to each target person, therefore, the corresponding target personnel can be continuously tracked through the target detection frame, and the accuracy of pedestrian tracking is improved.
And a determining module 605, configured to determine whether the corresponding target person invades the preset area based on the target detection frame.
In the embodiment of the present invention, the preset area may be an artificially set area, a dangerous area, or another area where entry is prohibited, and may be defined in a video image acquired by an image pickup device, then the corresponding target person is continuously tracked by the multiple continuous target detection frames corresponding to each target person obtained through the above steps, and whether the corresponding target person enters the preset area is determined by an overlapping area between the target detection frame and the preset area, and a corresponding prompt is performed, for example, when the overlapping area between the target detection frame and the preset area is greater than zero, a corresponding warning message is sent to remind the relevant person. Furthermore, the overlapping area can be constructed to classify the intrusion level of the alarm information, and the larger the overlapping area is, the deeper the pedestrian invades, the higher the intrusion level is, and the stronger the alarm information can be.
It should be noted that the device for detecting intrusion into a region provided by the embodiment of the present invention may be applied to a mobile phone, a monitor, a computer, a server, and other devices that can perform intrusion detection into a region.
The regional intrusion detection device provided by the embodiment of the invention can realize each process realized by the regional intrusion detection method in the embodiment of the method, and can achieve the same beneficial effect. To avoid repetition, further description is omitted here.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 9, the electronic device 900 includes: a memory 902, a processor 901 and a computer program stored on the memory 902 and executable on the processor 901, wherein:
the processor 901 is used for calling the computer program stored in the memory 902 and executing the following steps:
acquiring video data, wherein the video data comprises continuous multi-frame images, and the images comprise at least one target person;
respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set;
extracting feature information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set;
screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person;
and judging whether the corresponding target personnel invade a preset area or not based on the target detection frame.
Optionally, the screening, performed by the processor 901, the first detection frame set by using the second detection frame set to obtain a target detection frame set includes:
overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set, and calculating the overlapping area;
if the overlapping area is larger than a preset overlapping threshold value, putting the corresponding detection frame in the second detection frame set into the target detection frame set;
otherwise, putting the corresponding detection frame in the first detection frame set into the target detection frame set.
Optionally, the preset matching algorithm executed by the processor 901 includes a hungarian algorithm, and the obtaining of the target detection box corresponding to the target person by matching the feature information of the target person with the target detection box set through the preset matching algorithm includes:
carrying out graph structure processing on the characteristic information of the target person and the target detection frame set to obtain graph structure combined data;
and matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target personnel.
Optionally, after the extracting, by a third network model, the feature information of the target person based on the continuous multi-frame images and the first detection frame set, before the screening, by the second detection frame set, the first detection frame set to obtain a target detection frame set, the method executed by the processor 901 further includes:
and carrying out de-duplication and filtering processing on the first detection frame set and the second detection frame set.
Optionally, in the steps executed by the processor 901, the first detection model and the third detection model are constructed by a residual convolution, a standard convolution and a channel mixing algorithm, and are pre-trained by a corresponding training data set.
Optionally, in the step executed by the processor 901, the second detection model is constructed by a gaussian mixture background modeling method.
Optionally, in the steps executed by the processor 901, the pre-training of the first detection model or the third detection model includes:
acquiring video data;
acquiring a multi-frame image from the video data, acquiring a pedestrian from the image and marking the pedestrian to obtain a pedestrian data set;
dividing the pedestrian data set into a training set, a verification set and a test set;
and constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
The electronic device may be a device that can be applied to a mobile phone, a monitor, a computer, a server, and the like that can perform regional intrusion detection.
The electronic device provided by the embodiment of the invention can realize each process realized by the regional intrusion detection method in the method embodiment, can achieve the same beneficial effects, and is not repeated here to avoid repetition.
The memory 902 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 902 may be an internal storage unit of the electronic device 900, such as a hard disk or a memory of the electronic device 900. In other embodiments, the memory 902 may also be an external storage device of the electronic device 900, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the electronic device 900. Of course, the memory 902 may also include both internal and external memory units of the electronic device 900. In this embodiment, the memory 902 is generally used for storing an operating system installed in the electronic device 900 and various application software, such as a program code of a local intrusion detection method. In addition, the memory 902 may also be used to temporarily store various types of data that have been output or are to be output.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the method for detecting an intrusion into a region provided in the embodiment of the present invention, and can achieve the same technical effect, and in order to avoid repetition, the computer program is not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (16)
1. A method for regional intrusion detection, comprising the steps of:
acquiring video data, wherein the video data comprises continuous multi-frame images, and each frame of image comprises at least one target person;
respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set;
extracting feature information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set;
screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person;
and judging whether the corresponding target personnel invade a preset area or not based on the target detection frame.
2. The method of claim 1, wherein the screening the first set of detection boxes through the second set of detection boxes to obtain a set of target detection boxes comprises:
overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set, and calculating the overlapping area;
if the overlapping area is larger than a preset overlapping threshold value, putting the corresponding detection frame in the second detection frame set into the target detection frame set;
otherwise, putting the corresponding detection frame in the first detection frame set into the target detection frame set.
3. The method as claimed in claim 2, wherein the preset matching algorithm comprises a hungarian algorithm, and the matching the feature information of the target person with the target detection box set through the preset matching algorithm to obtain the target detection box corresponding to the target person comprises:
carrying out graph structure processing on the characteristic information of the target person and the target detection frame set to obtain graph structure combined data;
and matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target personnel.
4. The method of claim 1, wherein after the extracting, by a third network model, the feature information of the target person based on the consecutive multi-frame images and the first detection frame set, before the screening, by the second detection frame set, the first detection frame set to obtain a target detection frame set, the method further comprises:
and carrying out de-duplication and filtering processing on the first detection frame set and the second detection frame set.
5. The method of claim 4, wherein the first detection model and the third detection model are constructed by residual convolution, standard convolution and channel mixing algorithms and pre-trained by corresponding training data sets.
6. The method of claim 5, wherein the second detection model is constructed by a Gaussian mixture background modeling method.
7. The method of claim 6, wherein the pre-training of the first detection model and the third detection model comprises:
acquiring video data;
acquiring a multi-frame image from the video data, acquiring a pedestrian from the image and marking the pedestrian to obtain a pedestrian data set;
dividing the pedestrian data set into a training set, a verification set and a test set;
and constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
8. An area intrusion detection device, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring video data, and the video data comprises continuous multi-frame images which comprise at least one target person;
the detection module is used for respectively detecting the target person through a first detection model and a second detection model based on the continuous multi-frame images to obtain a corresponding first detection result and a corresponding second detection result, wherein the first detection result comprises a first detection frame set, and the second detection result comprises a second detection frame set;
the extracting module is used for extracting the characteristic information of the target person through a third detection model based on the continuous multi-frame images and the first detection frame set;
the screening and matching module is used for screening the first detection frame set through the second detection frame set to obtain a target detection frame set, and matching the characteristic information of the target person with the target detection frame set through a preset matching algorithm to obtain a target detection frame corresponding to the target person;
and the judging module is used for judging whether the corresponding target personnel invade the preset area or not based on the target detection frame.
9. The apparatus of claim 8, wherein the screening and matching module comprises:
the overlapping submodule is used for overlapping the detection frames in the second detection frame set with the detection frames in the first detection frame set and calculating the overlapping area;
the first input sub-module is used for inputting the corresponding detection frame in the second detection frame set into the target detection frame set if the overlapping area is larger than a preset overlapping threshold value;
and the second putting sub-module is used for putting the corresponding detection frame in the first detection frame set into the target detection frame set if the detection frame is not in the target detection frame set.
10. The apparatus of claim 9, wherein the screening and matching module further comprises:
the processing submodule is used for carrying out graph structure processing on the characteristic information of the target personnel and the target detection frame set to obtain graph structure combined data;
and the matching submodule is used for matching the graph structure joint data through the Hungarian algorithm to obtain a target detection box of the target personnel.
11. The apparatus of claim 10, wherein after the extraction module, the apparatus further comprises:
and the de-duplication and filtering processing module is used for carrying out de-duplication and filtering processing on the first detection frame set and the second detection frame set.
12. The apparatus of claim 11, in which the first detection model and the third detection model are constructed by residual convolution, standard convolution, and channel mixing algorithms and pre-trained by corresponding training data sets.
13. The apparatus of claim 12, wherein the second detection model is constructed by a gaussian mixture background modeling method.
14. The apparatus of claim 13, further comprising a pre-training module of the first detection model and the third detection model, the pre-training module comprising:
the first obtaining submodule is used for obtaining video data;
the second acquisition submodule is used for acquiring multi-frame images from the video data, acquiring pedestrians from the images and marking the pedestrians to obtain a pedestrian data set;
the dividing submodule is used for dividing the pedestrian data set into a training set, a verification set and a test set;
and the training submodule is used for constructing a first detection model or a third detection model and pre-training the first detection model or the third detection model through the pedestrian data set to obtain the trained first detection model or the trained third detection model.
15. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, the processor implementing the steps in the method of regional intrusion detection according to any of claims 1 to 7 when executing the computer program.
16. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for regional intrusion detection according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110164424.1A CN112861711A (en) | 2021-02-05 | 2021-02-05 | Regional intrusion detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110164424.1A CN112861711A (en) | 2021-02-05 | 2021-02-05 | Regional intrusion detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112861711A true CN112861711A (en) | 2021-05-28 |
Family
ID=75988673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110164424.1A Pending CN112861711A (en) | 2021-02-05 | 2021-02-05 | Regional intrusion detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112861711A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116862980A (en) * | 2023-06-12 | 2023-10-10 | 上海玉贲智能科技有限公司 | Target detection frame position optimization correction method, system, medium and terminal for image edge |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107645652A (en) * | 2017-10-27 | 2018-01-30 | 深圳极视角科技有限公司 | A kind of illegal geofence system based on video monitoring |
| CN109257569A (en) * | 2018-10-24 | 2019-01-22 | 广东佳鸿达科技股份有限公司 | Security protection video monitoring analysis method |
| CN109598743A (en) * | 2018-11-20 | 2019-04-09 | 北京京东尚科信息技术有限公司 | Pedestrian target tracking, device and equipment |
| CN109816690A (en) * | 2018-12-25 | 2019-05-28 | 北京飞搜科技有限公司 | Multi-target tracking method and system based on depth characteristic |
| CN110210474A (en) * | 2019-04-30 | 2019-09-06 | 北京市商汤科技开发有限公司 | Object detection method and device, equipment and storage medium |
| CN110427905A (en) * | 2019-08-08 | 2019-11-08 | 北京百度网讯科技有限公司 | Pedestrian tracting method, device and terminal |
| CN110738125A (en) * | 2019-09-19 | 2020-01-31 | 平安科技（深圳）有限公司 | Method, device and storage medium for selecting detection frame by using Mask R-CNN |
| CN110838133A (en) * | 2019-09-27 | 2020-02-25 | 深圳云天励飞技术有限公司 | Multi-target tracking method and related equipment |
| CN110889334A (en) * | 2019-11-06 | 2020-03-17 | 江河瑞通（北京）技术有限公司 | Personnel intrusion identification method and device |
| DE102018220274A1 (en) * | 2018-11-26 | 2020-05-28 | Osram Gmbh | Methods for the joint detection, tracking and classification of objects |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| CN111696130A (en) * | 2019-03-12 | 2020-09-22 | 北京京东尚科信息技术有限公司 | Target tracking method, target tracking apparatus, and computer-readable storage medium |
| CN111784750A (en) * | 2020-06-22 | 2020-10-16 | 深圳日海物联技术有限公司 | Method, device and equipment for tracking moving object in video image and storage medium |
| CN112116635A (en) * | 2020-09-17 | 2020-12-22 | 赵龙 | Visual tracking method and device based on rapid human body movement |
| CN112233097A (en) * | 2020-10-19 | 2021-01-15 | 中国科学技术大学 | Road scene other vehicle detection system and method based on space-time domain multi-dimensional fusion |
-
2021
- 2021-02-05 CN CN202110164424.1A patent/CN112861711A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107645652A (en) * | 2017-10-27 | 2018-01-30 | 深圳极视角科技有限公司 | A kind of illegal geofence system based on video monitoring |
| CN109257569A (en) * | 2018-10-24 | 2019-01-22 | 广东佳鸿达科技股份有限公司 | Security protection video monitoring analysis method |
| CN109598743A (en) * | 2018-11-20 | 2019-04-09 | 北京京东尚科信息技术有限公司 | Pedestrian target tracking, device and equipment |
| DE102018220274A1 (en) * | 2018-11-26 | 2020-05-28 | Osram Gmbh | Methods for the joint detection, tracking and classification of objects |
| CN109816690A (en) * | 2018-12-25 | 2019-05-28 | 北京飞搜科技有限公司 | Multi-target tracking method and system based on depth characteristic |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| CN111696130A (en) * | 2019-03-12 | 2020-09-22 | 北京京东尚科信息技术有限公司 | Target tracking method, target tracking apparatus, and computer-readable storage medium |
| CN110210474A (en) * | 2019-04-30 | 2019-09-06 | 北京市商汤科技开发有限公司 | Object detection method and device, equipment and storage medium |
| CN110427905A (en) * | 2019-08-08 | 2019-11-08 | 北京百度网讯科技有限公司 | Pedestrian tracting method, device and terminal |
| CN110738125A (en) * | 2019-09-19 | 2020-01-31 | 平安科技（深圳）有限公司 | Method, device and storage medium for selecting detection frame by using Mask R-CNN |
| CN110838133A (en) * | 2019-09-27 | 2020-02-25 | 深圳云天励飞技术有限公司 | Multi-target tracking method and related equipment |
| CN110889334A (en) * | 2019-11-06 | 2020-03-17 | 江河瑞通（北京）技术有限公司 | Personnel intrusion identification method and device |
| CN111784750A (en) * | 2020-06-22 | 2020-10-16 | 深圳日海物联技术有限公司 | Method, device and equipment for tracking moving object in video image and storage medium |
| CN112116635A (en) * | 2020-09-17 | 2020-12-22 | 赵龙 | Visual tracking method and device based on rapid human body movement |
| CN112233097A (en) * | 2020-10-19 | 2021-01-15 | 中国科学技术大学 | Road scene other vehicle detection system and method based on space-time domain multi-dimensional fusion |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116862980A (en) * | 2023-06-12 | 2023-10-10 | 上海玉贲智能科技有限公司 | Target detection frame position optimization correction method, system, medium and terminal for image edge |
| CN116862980B (en) * | 2023-06-12 | 2024-01-23 | 上海玉贲智能科技有限公司 | Target detection frame position optimization correction method, system, medium and terminal for image edge |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109325964B (en) | Face tracking method and device and terminal | |
| CN107358149B (en) | Human body posture detection method and device | |
| CN105279484B (en) | Method for checking object and object test equipment | |
| CN108268867B (en) | License plate positioning method and device | |
| CN108256404B (en) | Pedestrian detection method and device | |
| WO2021051601A1 (en) | Method and system for selecting detection box using mask r-cnn, and electronic device and storage medium | |
| CN110781836A (en) | Human body recognition method and device, computer equipment and storage medium | |
| US20060067562A1 (en) | Detection of moving objects in a video | |
| CN108009466B (en) | Pedestrian detection method and device | |
| CN111860318A (en) | Construction site pedestrian loitering detection method, device, equipment and storage medium | |
| CN110533950A (en) | Detection method, device, electronic equipment and the storage medium of parking stall behaviour in service | |
| CN111210399B (en) | Imaging quality evaluation method, device and equipment | |
| CN111160169B (en) | Face detection method, device, equipment and computer readable storage medium | |
| CN112085952A (en) | Vehicle data monitoring method and device, computer equipment and storage medium | |
| CN108197544B (en) | Face analysis method, face filtering method, face analysis device, face filtering device, embedded equipment, medium and integrated circuit | |
| CN110659391A (en) | Video detection method and device | |
| CN112434566A (en) | Passenger flow statistical method and device, electronic equipment and storage medium | |
| CN111126411B (en) | Abnormal behavior identification method and device | |
| CN112307994A (en) | Obstacle identification method based on sweeper, electronic device and storage medium | |
| CN113205510B (en) | Railway intrusion foreign matter detection method, device and terminal | |
| CN114359618A (en) | Training method of neural network model, electronic equipment and computer program product | |
| CN112861711A (en) | Regional intrusion detection method and device, electronic equipment and storage medium | |
| CN111950507B (en) | Data processing and model training method, device, equipment and medium | |
| CN113673308A (en) | Object identification method, device and electronic system | |
| CN116823884A (en) | Multi-target tracking method, system, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |