CN113470009B - Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium - Google Patents
Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113470009B CN113470009B CN202110843093.4A CN202110843093A CN113470009B CN 113470009 B CN113470009 B CN 113470009B CN 202110843093 A CN202110843093 A CN 202110843093A CN 113470009 B CN113470009 B CN 113470009B
- Authority
- CN
- China
- Prior art keywords
- detection area
- umbrella
- determining
- area
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 818
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 230000006399 behavior Effects 0.000 claims abstract description 86
- 238000012549 training Methods 0.000 claims description 50
- 238000012795 verification Methods 0.000 claims description 44
- 230000015654 memory Effects 0.000 claims description 22
- 238000002372 labelling Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 10
- 238000007635 classification algorithm Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 2
- 238000012216 screening Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 240000004282 Grewia occidentalis Species 0.000 description 2
- 230000004075 alteration Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- RKTYLMNFRDHKIL-UHFFFAOYSA-N copper;5,10,15,20-tetraphenylporphyrin-22,24-diide Chemical compound [Cu+2].C1=CC(C(=C2C=CC([N-]2)=C(C=2C=CC=CC=2)C=2C=CC(N=2)=C(C=2C=CC=CC=2)C2=CC=C3[N-]2)C=2C=CC=CC=2)=NC1=C3C1=CC=CC=C1 RKTYLMNFRDHKIL-UHFFFAOYSA-N 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, electronic equipment and a storage medium for detecting and identifying illegal umbrella opening, wherein the method comprises the following steps: acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image; for each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area; and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area. Therefore, the false detection of normal umbrella supporting behaviors such as supporting the umbrella by pedestrians and supporting the umbrella by driving non-motor vehicles is avoided. Therefore, the illegal umbrella opening detection scheme provided by the embodiment of the invention has better accuracy.
Description
Technical Field
The present invention relates to the field of target detection technologies, and in particular, to a method and apparatus for detecting and identifying an illegal umbrella, an electronic device, and a storage medium.
Background
With the increasing development of urban security, the construction of urban security and urban capacity is increasingly focused. The detection of the illegal umbrella is an important link of the construction of the smart city, the aesthetic degree of the city is seriously affected, and the illegal umbrella is mainly from store gates, side of kiosks, mobile booths, street-keeping booths and the like at present. Traditional processing for detecting illegal umbrella opening is often on-site inspection of a city management or manual statistics of specific positions of illegal umbrella opening through observing videos, and specific city management personnel are dispatched to on-site law enforcement. The traditional method needs to consume a large amount of manpower and material resources for urban management, the implementation effect is general, management is often relatively low-efficiency, timely feedback cannot be achieved, and convenience in evidence collection and law enforcement are not achieved.
With the rapid development of artificial intelligence technology, multiple cities are put into construction of smart cities by adopting an AI technology, and illegal umbrella supporting behaviors in videos are detected by a deep learning method. After training the detection model, inputting the acquired image into the detection model, and outputting a detection result of whether the illegal umbrella is opened or not. But because of the variety of umbrella types, including round, tetragonal, and of different sizes, and in addition similar shapes to umbrellas, including sunshades on windows, sunshades on non-motor vehicles, etc. The prior art scheme is easy to generate false detection, such as false detection of pedestrian umbrella opening as illegal umbrella opening and the like.
Therefore, the prior art has the problem of poor detection accuracy of illegal umbrella opening.
Disclosure of Invention
The embodiment of the invention provides a method, a device, electronic equipment and a storage medium for detecting and identifying illegal umbrella opening, which are used for solving the problem of poor detecting and identifying accuracy of illegal umbrella opening in the prior art.
The embodiment of the invention provides a method for detecting and identifying illegal umbrella opening, which comprises the following steps:
Acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image;
For each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
Further, the identifying the image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image includes:
Inputting an image in the video to be detected into a pre-trained target detection model, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image based on the target detection model; the target detection model is obtained by training according to the sample image and the corresponding labeling image aiming at each sample image in the first training set, wherein the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
Further, after determining each umbrella detection area in the image, before determining the intersection ratio of the umbrella detection area and each human detection area for each umbrella detection area, the method further includes:
Inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, determining the type information of the umbrella detection area based on the verification model, reserving the umbrella detection area if the type information of the umbrella detection area is umbrella, and filtering the umbrella detection area if the type information of the umbrella detection area is umbrella.
Further, the training process of the verification model comprises the following steps:
inputting the detection area image and corresponding label information into the verification model aiming at each detection area image in the second training set, and training the verification model by adopting a fine granularity classification algorithm of countermeasure learning; wherein the tag information includes type information of a corresponding detection area image.
Further, according to the intersection ratio of each umbrella detection area to each human detection area, the intersection ratio of each umbrella detection area to each non-motor vehicle detection area, determining a target umbrella detection area with illegal umbrella opening behavior from each umbrella detection area comprises:
determining whether the umbrella detection area is a fixed position area or a moving position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image which is separated from the image by a preset frame number for each umbrella detection area;
If the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area;
if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
Further, if the umbrella detection area is a fixed position area, determining that the maximum intersection ratio of the umbrella detection area and the human body detection area is smaller than a preset first intersection ratio threshold value, and determining that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behavior, the method further includes:
judging whether the umbrella detection area is positioned in a preset rule area range, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behaviors;
If the umbrella detection area is a moving position area, after judging that the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and the method further comprises the following steps:
judging whether the umbrella detection areas are all located in a preset rule area range in the image with the preset frame number, and if so, determining that the umbrella detection areas are target umbrella detection areas with illegal umbrella supporting behaviors.
Further, after the determining that the target umbrella detection area of the illegal umbrella opening behavior exists, the method further comprises:
And outputting alarm information for prompting the existence of illegal umbrella opening behaviors, and carrying the position information of the target umbrella detection area in the alarm information.
In another aspect, an embodiment of the present invention provides a device for detecting and identifying an illegal umbrella, where the device includes:
The first determining module is used for acquiring a video to be detected, identifying images in the video to be detected and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the images;
The second determining module is used for determining the intersection ratio of the umbrella detection area and each human detection area according to each umbrella detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
The detection module is used for determining a target umbrella detection area with illegal umbrella opening behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
Further, the first determining module is specifically configured to input an image in the video to be detected into a target detection model that is trained in advance, and determine each umbrella detection area, each human body detection area and each non-motor vehicle detection area in the image based on the target detection model; the target detection model is obtained by training according to the sample image and the corresponding labeling image aiming at each sample image in the first training set, wherein the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
Further, the apparatus further comprises:
The screening module is used for inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, determining the type information of the umbrella detection area based on the verification model, reserving the umbrella detection area if the type information of the umbrella detection area is umbrella, and otherwise filtering the umbrella detection area.
Further, the apparatus further comprises:
The first training module is used for inputting the sample image and the corresponding labeling image into the target detection model aiming at each sample image in the first training set, and training the target detection model; the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
Further, the apparatus further comprises:
The second training module is used for inputting the detection area image and the corresponding label information into the verification model aiming at each detection area image in the second training set, and training the verification model by adopting a fine granularity classification algorithm of countermeasure learning; wherein the tag information includes type information of a corresponding detection area image.
Further, the detection module is specifically configured to determine, for each umbrella detection area, whether the umbrella detection area is a fixed position area or a mobile position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image spaced by a preset frame number from the image; if the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area; if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
Further, the apparatus further comprises:
The judging module is used for judging whether the umbrella detection area is located in a preset regular area range or not if the umbrella detection area is a fixed position area, judging that the maximum intersection ratio of the umbrella detection area and the human body detection area is smaller than a preset first intersection ratio threshold value, and if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors;
The judging module is further configured to judge whether the umbrella detection areas are all located within a preset regular area in the image of the preset frame number if the maximum intersection ratio of the umbrella detection areas and the non-motor vehicle detection areas of the preset class is greater than a preset second intersection ratio threshold, and if yes, determine that the umbrella detection areas are target umbrella detection areas with illegal umbrella supporting behaviors.
Further, the apparatus further comprises:
And the alarm module is used for outputting alarm information for prompting the existence of illegal umbrella opening behaviors and carrying the position information of the target umbrella detection area in the alarm information.
On the other hand, the embodiment of the invention provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing any of the method steps described above when executing a program stored on a memory.
In another aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored therein, which when executed by a processor, implements the method steps of any of the above.
The embodiment of the invention provides a method, a device, electronic equipment and a storage medium for detecting and identifying illegal umbrella opening, wherein the method comprises the following steps: acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image; for each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area; and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
The technical scheme has the following advantages or beneficial effects:
In the embodiment of the invention, an image in a video to be detected is identified, and after each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined, the intersection ratio of the umbrella detection area and each human body detection area is respectively determined for each umbrella detection area, and the intersection ratio of the umbrella detection area and each non-motor vehicle detection area is determined. And screening each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, and the intersection ratio of each umbrella detection area to each non-motor vehicle detection area, so as to filter out the normal umbrella supporting behavior of pedestrians and non-motor vehicles, and finally determining the target umbrella detection area with illegal umbrella supporting behavior through screening. Therefore, the false detection of the normal umbrella opening behavior on the non-motor vehicle as the illegal umbrella opening behavior is avoided. Therefore, the scheme for detecting and identifying the illegal umbrella opening provided by the embodiment of the invention has better accuracy.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the detecting and identifying process of the illegal umbrella opening provided in embodiment 1 of the present invention;
FIG. 2 is a flowchart of umbrella screening according to embodiment 3 of the present invention;
FIG. 3 is a flow chart of detecting illegal umbrella opening provided in embodiment 4 of the present invention;
FIG. 4 is a flow chart of detecting illegal umbrella opening provided in embodiment 5 of the present invention;
FIG. 5 is a flow chart of another detecting a offending umbrella provided in embodiment 5 of the present invention;
FIG. 6 is a flowchart of detecting an offending umbrella according to embodiment 5 of the present invention;
FIG. 7 is a schematic structural diagram of a detecting and identifying device for an illegal umbrella according to embodiment 6 of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to embodiment 7 of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the attached drawings, wherein it is apparent that the embodiments described are only some, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1:
fig. 1 is a schematic diagram of a process for detecting and identifying an illegal umbrella according to an embodiment of the present invention, where the process includes the following steps:
S101: and acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image.
S102: and determining the intersection ratio of the umbrella detection area and each human detection area according to each umbrella detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area.
S103: and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
The detection and identification method of the illegal umbrella is applied to electronic equipment, and the electronic equipment can be PC, tablet personal computer and other equipment, and also can be intelligent image acquisition equipment. If the electronic equipment is intelligent image acquisition equipment, the intelligent image acquisition equipment directly performs the detection and identification process of the follow-up illegal umbrella opening after acquiring the video to be detected. If the electronic equipment is a PC, a tablet personal computer and the like, after the intelligent image acquisition equipment acquires the video to be detected, the video to be detected is sent to the electronic equipment, and then the electronic equipment performs the subsequent detection and identification processes of the illegal umbrella opening.
In the embodiment of the invention, after the electronic equipment acquires the video to be detected, the image in the video to be detected is identified, and each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined. Specifically, the method can be used for identifying the image in the video to be detected based on the deep neural network model, can be used for identifying the image in the video to be detected based on the fractal characteristic, and can be used for identifying the image in the video to be detected based on the wavelet moment. Thereby determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image. The determined umbrella detection area, human body detection area and non-motor vehicle detection area are areas which are contained by the circumscribed connection of each umbrella, human body and non-motor vehicle in the image.
For each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area. And then screening each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, and determining the target umbrella detection area with illegal umbrella supporting behaviors.
For example, a preset intersection ratio threshold value is stored in the electronic device, and if the maximum intersection ratio of the umbrella detection area and the human body detection area is larger than the preset intersection ratio threshold value for each umbrella detection area, the umbrella detection area is considered to be the detection area of the pedestrian umbrella, and the umbrella detection area is not taken as the target umbrella detection area with illegal umbrella opening behaviors. Or if the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area is larger than a preset intersection ratio threshold value, the umbrella detection area is considered to be the umbrella detection area on the non-motor vehicle, and the umbrella detection area is not taken as the target umbrella detection area with illegal umbrella supporting behaviors. And if the maximum intersection ratio of the umbrella detection area and the human body detection area is not greater than a preset intersection ratio threshold value and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area is not greater than the preset intersection ratio threshold value, taking the umbrella detection area as a target umbrella detection area with illegal umbrella opening behaviors.
In the embodiment of the invention, an image in a video to be detected is identified, and after each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined, the intersection ratio of the umbrella detection area and each human body detection area is respectively determined for each umbrella detection area, and the intersection ratio of the umbrella detection area and each non-motor vehicle detection area is determined. And screening each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, and the intersection ratio of each umbrella detection area to each non-motor vehicle detection area, so as to filter out the normal umbrella supporting behavior of pedestrians and non-motor vehicles, and finally determining the target umbrella detection area with illegal umbrella supporting behavior through screening. Therefore, the false detection of the normal umbrella opening behavior on the non-motor vehicle as the illegal umbrella opening behavior is avoided. Therefore, the scheme for detecting and identifying the illegal umbrella opening provided by the embodiment of the invention has better accuracy.
Example 2:
In order to determine each umbrella detection area, human body detection area and non-motor vehicle detection area in the image, on the basis of the above embodiment, in the embodiment of the present invention, the identifying the image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image includes:
Inputting an image in the video to be detected into a pre-trained target detection model, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image based on the target detection model; the target detection model is obtained by training according to the sample image and the corresponding labeling image aiming at each sample image in the first training set, wherein the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
In the embodiment of the invention, the electronic equipment stores a target detection model which is trained in advance, and the target detection model is trained according to the sample images in the training set and the corresponding labeling images.
Specifically, the training process of the target detection model includes:
Inputting the sample image and the corresponding labeling image into the target detection model aiming at each sample image in the first training set, and training the target detection model; the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
The electronic device stores a first training set comprising a plurality of sample images, each sample image having a corresponding annotation image. The labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image. And inputting the sample image and the corresponding labeling image into a target detection model aiming at each sample image in the first training set, wherein the target detection model outputs a training image corresponding to the sample image, the training image comprises position information of each umbrella detection area, each human body detection area and each non-motor vehicle detection area, and then a loss value is calculated according to the position information of each umbrella detection area, each human body detection area and each non-motor vehicle detection area in the training image and the position information of each umbrella detection area, each human body detection area and each non-motor vehicle detection area in the labeling image corresponding to the sample image. And adjusting parameter values of all parameters in the target detection model according to the loss value. When the convergence condition is reached, the training of the target detection model is completed.
And inputting the images in the video to be detected into a pre-trained target detection model, and outputting each umbrella detection area, human body detection area and non-motor vehicle detection area in the images by the target detection model.
Example 3:
in order to make the identification of the illegal umbrella more accurate, in the embodiments of the present invention, after determining each umbrella detection area in the image, before determining the intersection ratio of the umbrella detection area and each person detection area for each umbrella detection area, the method further includes:
Inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, determining the type information of the umbrella detection area based on the verification model, reserving the umbrella detection area if the type information of the umbrella detection area is umbrella, and filtering the umbrella detection area if the type information of the umbrella detection area is umbrella.
In the embodiment of the invention, the electronic equipment stores a pre-trained verification model, and the verification model is used for verifying and screening each umbrella detection area in the identified image.
Specifically, after the electronic device identifies each umbrella detection area in the image, inputting the umbrella detection area into a pre-trained verification model for each umbrella detection area, and determining type information of the umbrella detection area based on the verification model. If the type information of the umbrella detection area is umbrella, reserving the umbrella detection area, otherwise filtering the umbrella detection area.
The training process of the verification model comprises the following steps:
inputting the detection area image and corresponding label information into the verification model aiming at each detection area image in the second training set, and training the verification model by adopting a fine granularity classification algorithm of countermeasure learning; wherein the tag information includes type information of a corresponding detection area image.
The electronic device stores a second training set, the second training set including a plurality of detection area images, each detection area image having corresponding tag information. Wherein the tag information includes type information of the corresponding detection area image. And inputting the detection region image and the corresponding label information into a verification model aiming at each detection region image in the second training set, outputting training label information corresponding to the detection region image by the verification model, and calculating a loss value according to the training label information and the label information corresponding to the detection region image. And adjusting parameter values of all parameters in the verification model according to the loss value. When the convergence condition is reached, the verification model training is completed.
And inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, and outputting type information of the umbrella detection area by the verification model.
Specifically, the verification model is used for improving the detection precision of the umbrella and eliminating the target frame similar to the umbrella. The verification model is a fine-grained umbrella classification model, the input of the model is a picture with the size of 128 x 128 pixels, and the model is very small. And introducing an countermeasure learning training strategy into model training, wherein the countermeasure pictures are 8 x8 and 16 x 16 region confusion, reconstructing the confusion region when the picture downsampling is 8 times and 16 times, and the model can learn information of fine distinction between different categories. The verification model loss comprises three parts: classification loss, counterlearning loss, and regional reconstruction loss. The overall overview of the quadratic model check is: each umbrella detection area identified is first acquired, then each umbrella detection area is cut on an original image, then a picture with the size of 128 pixels is scaled to be 128 pixels, and then the picture is sent to a verification model. And the verification model outputs category information, when the output category information and the detection category information are consistent, the umbrella detection area is reserved, and otherwise, the umbrella detection area is abandoned.
FIG. 2 is a flowchart of umbrella screening according to an embodiment of the present invention, including the following steps:
S201: inputting an image in a video to be detected into a pre-trained target detection model, and determining each umbrella detection area in the image based on the target detection model.
S202: inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, determining the type information of the umbrella detection area based on the verification model, reserving the umbrella detection area if the type information of the umbrella detection area is umbrella, and filtering the umbrella detection area if not.
Example 4:
In order to make detection of the illegal umbrella more accurate, in the embodiments of the present invention, determining, from the each umbrella detection area, a target umbrella detection area in which the illegal umbrella-supporting behavior exists according to the intersection ratio of the each umbrella detection area to the each human detection area, the intersection ratio of the each umbrella detection area to the each non-motor vehicle detection area includes:
determining whether the umbrella detection area is a fixed position area or a moving position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image which is separated from the image by a preset frame number for each umbrella detection area;
If the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area;
if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
In the embodiment of the invention, different strategies are adopted for detecting whether the umbrella is illegal to open or not for the umbrella with the moving position and the umbrella with the fixed position in the video, so that the illegal umbrella opening detection is more accurate.
First, for each umbrella detection area, whether the umbrella detection area is a fixed position area or a moving position area is determined based on the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image spaced apart from the image by a preset number of frames.
The electronic equipment acquires a video to be detected, identifies each frame of image in the video to be detected, and determines each umbrella detection area and ID corresponding to each umbrella detection area in each frame of image. Two umbrella detection areas with the same ID in two images separated by a preset frame number are determined. And determining whether the umbrella detection area is a fixed position area or a mobile position area according to the position information of the two umbrella detection areas with the same ID in the image. The method comprises the steps of storing a preset distance threshold in the electronic equipment, determining the distance between two umbrella detection areas with the same ID according to the position information of the two umbrella detection areas with the same ID in an image, if the distance is larger than the preset distance threshold, determining the umbrella detection area as a mobile position area, otherwise, determining the umbrella detection area as a fixed position area.
If the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area. Therefore, the umbrella detection area of the pedestrian umbrella or the sunshade on the non-motor vehicle in the static state is filtered, and the target umbrella detection area of the illegal umbrella is reserved.
If the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area. Among them, the preset class of non-motor vehicles are non-motor vehicles commonly used by the mobile vendors, such as electric tricycles. The electronic equipment acquires a video to be detected, identifies each frame of image in the video to be detected, and determines each non-motor vehicle detection area in each frame of image and non-motor vehicle type information corresponding to each non-motor vehicle detection area, wherein the non-motor vehicle type information is, for example, an electric bicycle, an electric tricycle and the like. For the moving umbrella detection area, if the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area of the preset category is larger than a preset second intersection ratio threshold value, the umbrella detection area is the umbrella detection area of the mobile vendor umbrella, and the umbrella detection area is determined to be the target umbrella detection area with illegal umbrella opening behaviors. Therefore, the target umbrella detection area with illegal umbrella opening behaviors is effectively and accurately determined.
Fig. 3 is a flowchart of detecting an illegal umbrella opening according to an embodiment of the present invention, and the process includes the following steps:
S301: and acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image.
S302: and determining the intersection ratio of the umbrella detection area and each human detection area according to each umbrella detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area.
S303: for each umbrella detection area, determining whether the umbrella detection area is a fixed position area or a moving position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image which is separated from the image by a preset frame number, if the umbrella detection area is determined to be the fixed position area, performing S304, and if the umbrella detection area is determined to be the moving position area, performing S305.
S304: judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area is smaller than a preset first intersection ratio threshold value or not, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
S305: judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
The fact shows that the result output by the target detection model cannot be directly used as the basis of illegal umbrella opening, because the real event contains excessive non-illegal umbrella opening events. The method mainly eliminates the determined non-illegal umbrella supporting event from detection results, such as sunshades of pedestrian umbrella supporting, electric vehicles or motorcycles, and some detection results need to filter target parking event judgment, such as movement of illegal umbrella on mobile vendor vehicles (usually tricycles) or movement of umbrella by pedestrians. In the embodiment of the invention, detection target information output by a target detection model is obtained, wherein the detection target information comprises category information, position information, target unique id, umbrella detection area, human body detection area and non-motor vehicle detection area. And respectively calculating the intersection ratio IOU of the umbrella detection area and the human body detection area, and calculating the intersection ratio IOU of the umbrella detection area and the non-motor vehicle detection area. And respectively tracking the ids by adopting a multi-target tracking algorithm, and reserving id pairs of pedestrians, non-motor vehicles and umbrellas with IOUs larger than a threshold alpha and id pairs of the maximum IOU for specific positions of target ids of umbrellas of each frame. Let the distance threshold be beta, after every interval of k frames, if(M represents the m-th frame, n represents the object with id of n, d represents the object position information), and the umbrella position is the fixed position, otherwise, the umbrella is in the moving state. For umbrellas in a fixed position state, removing umbrella ids larger than alpha in the state, wherein the umbrellas are pedestrian umbrellas in a static state or sun-shading sheds on motorcycles, and correct illegal umbrellas in the static state are reserved. For umbrellas in a moving state, the id pair of the tricycle and the illegal umbrellas is reserved, and for the subsequent entry of new umbrella targets, the judgment is also made. The non-offending umbrella determined so far can be completely found out.
The parameters alpha and beta are parameters for adjusting the sensitivity of the algorithm, the beta measures the interference factors of the external environment, when some external environments change relatively greatly, the beta can be adjusted to be larger, the alpha reflects the density of the detection target, when the target aggregation is relatively serious, the alpha can be properly increased, and a group of parameters alpha and beta can be generated through the number of target and IOU pairs.
Example 5:
In order to make detection of an illegal umbrella more accurate, in the above embodiments, in the embodiments of the present invention, if the umbrella detection area is a fixed position area, the method further includes, before determining that the umbrella detection area is a target umbrella detection area with an illegal umbrella supporting behavior after determining that the maximum intersection ratio of the umbrella detection area and the human body detection area, and the maximum intersection ratios of the umbrella detection area and the non-motor vehicle detection area are both smaller than a preset first intersection ratio threshold value:
judging whether the umbrella detection area is positioned in a preset rule area range, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behaviors;
If the umbrella detection area is a moving position area, after judging that the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and the method further comprises the following steps:
judging whether the umbrella detection areas are all located in a preset rule area range in the image with the preset frame number, and if so, determining that the umbrella detection areas are target umbrella detection areas with illegal umbrella supporting behaviors.
Considering that the illegal umbrella supporting event mainly occurs in areas such as front of store doors and on two sides of traffic roads, the electronic equipment stores a preset rule area range in the embodiment of the invention. The preset regular area range is, for example, an area range of a shop front, two sides of a traffic road and the like in the picture.
Fig. 4 is a flowchart of detecting an illegal umbrella opening, provided by an embodiment of the invention, including the following steps:
S401: if the umbrella detection area is a fixed position area, determining the maximum intersection ratio of the umbrella detection area and the human body detection area, and determining that the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area is smaller than a preset first intersection ratio threshold value.
S402: whether the umbrella detection area is within the preset rule area is judged, if yes, S403 is carried out, and if not, S404 is carried out.
S403: and determining the umbrella detection area as a target umbrella detection area with illegal umbrella opening behaviors.
S404: and determining that the umbrella detection area is not a target umbrella detection area with illegal umbrella opening behaviors.
Fig. 5 is a flowchart for detecting illegal umbrella opening, provided by the embodiment of the invention, comprising the following steps:
s501: and if the umbrella detection area is a moving position area, determining that the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value.
S502: and judging whether the umbrella detection areas are all located in the preset regular area range in the images with the preset frame number, if so, performing S403, and if not, performing S404.
S503: and determining the umbrella detection area as a target umbrella detection area with illegal umbrella opening behaviors.
S504: and determining that the umbrella detection area is not a target umbrella detection area with illegal umbrella opening behaviors.
In the embodiment of the invention, the preset rule area range can be a partial area range in the image or a full screen range. The set rule positive event has high accuracy and low false alarm, but because the umbrella opening event is a random event, the umbrella opening position is usually not fixed, and the common center point of the illegal umbrella cover is relatively high, the alarm is not triggered sufficiently, the full screen detection recall rate is high, but the false alarm can be increased. For the illegal umbrella supporting event, the rule areas are mainly arranged in front of store doors and at the two sides of traffic roads, and the rule areas can be arranged to be high enough.
In the embodiment of the invention, for the umbrella with the fixed position, whether the umbrella is in the regular area is judged, and if the umbrella is in the regular area, the target umbrella detection area with illegal umbrella opening behaviors is determined. And tracking the id of the moving umbrella, and determining that a target umbrella detection area with illegal umbrella supporting behaviors exists if the umbrella detection areas are all located in the preset regular area range in the image with the preset frame number.
In the embodiment of the present invention, after the target umbrella detection area where the illegal umbrella opening behavior exists is determined, the method further includes:
And outputting alarm information for prompting the existence of illegal umbrella opening behaviors, and carrying the position information of the target umbrella detection area in the alarm information.
In order to be convenient for reminding personnel of illegal umbrella opening behaviors so that related personnel can timely process illegal umbrella opening events, in the embodiment of the invention, after a target umbrella detection area with the illegal umbrella opening behaviors is determined, alarm information for prompting the existence of the illegal umbrella opening behaviors is output, and the position information of the target umbrella detection area is carried in the alarm information. The alarm information can be text alarm information, for example, a video picture and alarm content are displayed on a display screen of the electronic equipment, and the alarm content contains the position information of the target umbrella detection area. Preferably, the alarm content can be displayed in a superimposed manner at the position of the target umbrella detection area in the video picture. The alarm information can also be voice alarm information, for example, alarm voice is output through a loudspeaker of the electronic equipment, wherein the alarm voice carries the position information of the target umbrella detection area.
The embodiment of the invention is based on the detection and recognition of the deep learning target, and adds a secondary verification, event filtering and rule alarm module of the illegal target on the basis of detection, wherein the first two modules mainly improve the detection precision of the target, and the later gives the rule for alarming the illegal event. The whole process of the illegal umbrella opening detection scheme is as follows: the training data set of the umbrella is collected to train the target detection model of the illegal umbrella, and attention is paid to the training strategy different from other methods, wherein the umbrella data comprises the common umbrella shade for pedestrians and the illegal large umbrella, the illegal large umbrella comprises a round umbrella and a four-corner umbrella, and the umbrella carried by the pedestrians is usually round. Compared with a four-corner umbrella, the round umbrella is easier to misdetect, most of negative samples and difficult samples are round umbrellas which are supported by pedestrians and non-motor vehicles, the difficulty in the scheme is distinguishing between the positive and negative samples, and the key point is removing the misdetection.
FIG. 6 is a flowchart of detecting an offending umbrella according to an embodiment of the present invention. And outputting the video to be detected to an umbrella detection area, a human body detection area and a non-motor vehicle detection area through the target detection model. And sending the umbrella detection area into a verification model, and screening the umbrella detection area. Different strategies are adopted to detect whether the umbrella with the moving position and the umbrella with the fixed position in the video are illegal umbrella supporting. And finally detecting the illegal umbrella opening according to the preset rule area range.
The whole scheme is summarized as follows: outputting the video stream to be detected to an umbrella detection area, a human body detection area and a non-motor vehicle detection area through a target detection model, then sending the umbrella detection area to a verification model, and removing False Positives (FP) targets similar to umbrellas, such as a rain shelter of a shop lintel, a window and the like through the verification model. The remaining umbrella detection area, the human body detection area and the non-motor vehicle detection area are used for filtering umbrellas which are supported on pedestrians and non-motor vehicles through an umbrella filtering rule module. And sending the filtered target frame to an alarm rule module of the illegal umbrella opening, and finally judging whether the umbrella opening is illegal or not through detailed logic. The umbrella filtering rule module detects whether the umbrella is illegal according to the position-moving umbrella and the position-fixed umbrella in the video by adopting different strategies. And the alarm rule module performs a final process of detecting the illegal umbrella according to the preset rule area range.
Example 6:
fig. 7 is a schematic structural diagram of a detecting and identifying device for an illegal umbrella according to an embodiment of the present invention, where the device includes:
A first determining module 71, configured to acquire a video to be detected, identify an image in the video to be detected, and determine each umbrella detection area, human body detection area, and non-motor vehicle detection area in the image;
A second determining module 72, configured to determine, for each umbrella detection area, an intersection ratio of the umbrella detection area to each human detection area, and determine an intersection ratio of the umbrella detection area to each non-motor vehicle detection area;
and the detection module 73 is configured to determine, from the each umbrella detection area, a target umbrella detection area in which an illegal umbrella-opening behavior exists according to the intersection ratio of the each umbrella detection area and the each human detection area, wherein the intersection ratio of the each umbrella detection area and the each non-motor vehicle detection area.
The first determining module 71 is specifically configured to input an image in the video to be detected into a target detection model that is trained in advance, and determine, based on the target detection model, each umbrella detection area, human body detection area, and non-motor vehicle detection area in the image; the target detection model is obtained by training according to the sample image and the corresponding labeling image aiming at each sample image in the first training set, wherein the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
The apparatus further comprises:
And the screening module 74 is configured to input the umbrella detection area into a pre-trained verification model for each umbrella detection area, determine type information of the umbrella detection area based on the verification model, reserve the umbrella detection area if the type information of the umbrella detection area is umbrella, and otherwise filter the umbrella detection area.
The apparatus further comprises:
A first training module 75, configured to input, for each sample image in the first training set, the sample image and the corresponding label image into the target detection model, and train the target detection model; the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
The apparatus further comprises:
A second training module 76, configured to input, for each detection area image in the second training set, the detection area image and corresponding tag information into the verification model, and train the verification model by adopting a fine-granularity classification algorithm for countermeasure learning; wherein the tag information includes type information of a corresponding detection area image.
The detecting module 73 is specifically configured to determine, for each umbrella detecting area, whether the umbrella detecting area is a fixed position area or a moving position area according to the position information of the umbrella detecting area and the position information of the umbrella detecting area corresponding to the image spaced by a preset frame number from the image; if the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area; if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
The apparatus further comprises:
The judging module 77 is configured to judge, if the umbrella detection area is a fixed location area, a maximum intersection ratio of the umbrella detection area and a human body detection area, and the maximum intersection ratios of the umbrella detection area and a non-motor vehicle detection area are both smaller than a preset first intersection ratio threshold, and judge whether the umbrella detection area is located within a preset regular area range, if so, determine that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behavior;
The determining module 77 is further configured to determine, if the umbrella detection area is a moving position area, that a maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is greater than a preset second intersection ratio threshold, determine whether the umbrella detection areas are all located within a preset regular area in the image of the preset frame number, and if yes, determine that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behavior.
The apparatus further comprises:
And the alarm module 78 is used for outputting alarm information for prompting the existence of illegal umbrella opening behaviors and carrying the position information of the target umbrella detection area in the alarm information.
Example 7:
on the basis of the above embodiments, the embodiment of the present invention further provides an electronic device, as shown in fig. 8, including: a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804;
the memory 803 stores a computer program that, when executed by the processor 801, causes the processor 801 to perform the steps of:
Acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image;
For each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
Based on the same inventive concept, the embodiment of the invention also provides an electronic device, and because the principle of solving the problem of the electronic device is similar to that of detecting and identifying the illegal umbrella, the implementation of the electronic device can be referred to the implementation of the method, and the repetition is omitted.
The electronic device provided by the embodiment of the invention can be a desktop computer, a portable computer, a smart phone, a tablet Personal computer, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), a network side device and the like.
The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface 802 is used for communication between the electronic device and other devices described above.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (DIGITAL SIGNAL Processing unit, DSP), application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.
When a processor executes a program stored in a memory, the method and the device realize acquisition of a video to be detected, identify images in the video to be detected, and determine each umbrella detection area, human body detection area and non-motor vehicle detection area in the images; for each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area; and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area. In the embodiment of the invention, an image in a video to be detected is identified, and after each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined, the intersection ratio of the umbrella detection area and each human body detection area is respectively determined for each umbrella detection area, and the intersection ratio of the umbrella detection area and each non-motor vehicle detection area is determined. And screening each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, and the intersection ratio of each umbrella detection area to each non-motor vehicle detection area, so as to filter out the normal umbrella supporting behavior of pedestrians and non-motor vehicles, and finally determining the target umbrella detection area with illegal umbrella supporting behavior through screening. Therefore, the false detection of the normal umbrella opening behavior on the non-motor vehicle as the illegal umbrella opening behavior is avoided. Therefore, the scheme for detecting and identifying the illegal umbrella opening provided by the embodiment of the invention has better accuracy.
Example 8:
On the basis of the above embodiments, the embodiments of the present invention further provide a computer-readable storage medium having stored therein a computer program executable by an electronic device, which when run on the electronic device, causes the electronic device to perform the steps of:
Acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image;
For each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area.
Based on the same inventive concept, the embodiment of the invention further provides a computer readable storage medium, and because the principle of solving the problem when the processor executes the computer program stored on the computer readable storage medium is similar to that of the illegal umbrella opening detection and identification method, the implementation of the processor executing the computer program stored on the computer readable storage medium can refer to the implementation of the method, and the repetition is omitted.
The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memories such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc., optical memories such as CD, DVD, BD, HVD, etc., and semiconductor memories such as ROM, EPROM, EEPROM, nonvolatile memories (NAND FLASH), solid State Disks (SSD), etc.
The computer readable storage medium provided by the embodiment of the invention stores a computer program, when the computer program is executed by a processor, the video to be detected is obtained, the image in the video to be detected is identified, and each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined; for each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area; and determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area. In the embodiment of the invention, an image in a video to be detected is identified, and after each umbrella detection area, human body detection area and non-motor vehicle detection area in the image are determined, the intersection ratio of the umbrella detection area and each human body detection area is respectively determined for each umbrella detection area, and the intersection ratio of the umbrella detection area and each non-motor vehicle detection area is determined. And screening each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, and the intersection ratio of each umbrella detection area to each non-motor vehicle detection area, so as to filter out the normal umbrella supporting behavior of pedestrians and non-motor vehicles, and finally determining the target umbrella detection area with illegal umbrella supporting behavior through screening. Therefore, the false detection of the normal umbrella opening behavior on the non-motor vehicle as the illegal umbrella opening behavior is avoided. Therefore, the scheme for detecting and identifying the illegal umbrella opening provided by the embodiment of the invention has better accuracy.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. A method for detecting and identifying an offending umbrella, the method comprising:
Acquiring a video to be detected, identifying an image in the video to be detected, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image;
For each umbrella detection area, determining the intersection ratio of the umbrella detection area and each human detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area;
Determining whether the umbrella detection area is a fixed position area or a mobile position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image which is separated from the image by a preset frame number for each umbrella detection area;
If the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area;
if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
2. The method of claim 1, wherein the identifying the image in the video to be detected, determining each of the umbrella detection area, the human body detection area, and the non-motor vehicle detection area in the image comprises:
Inputting an image in the video to be detected into a pre-trained target detection model, and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the image based on the target detection model; the target detection model is obtained by training according to the sample image and the corresponding labeling image aiming at each sample image in the first training set, wherein the labeling image is labeled with the position information of each umbrella detection area, human body detection area and non-motor vehicle detection area in the corresponding sample image.
3. The method of claim 1, wherein after determining each umbrella detection region in the image, before determining an intersection ratio of the umbrella detection region and each human detection region for each umbrella detection region, respectively, the method further comprises:
Inputting the umbrella detection area into a pre-trained verification model aiming at each umbrella detection area, determining the type information of the umbrella detection area based on the verification model, reserving the umbrella detection area if the type information of the umbrella detection area is umbrella, and filtering the umbrella detection area if the type information of the umbrella detection area is umbrella.
4. A method according to claim 3, wherein the training process of the verification model comprises:
inputting the detection area image and corresponding label information into the verification model aiming at each detection area image in the second training set, and training the verification model by adopting a fine granularity classification algorithm of countermeasure learning; wherein the tag information includes type information of a corresponding detection area image.
5. The method of claim 1, wherein if the umbrella detection area is a fixed location area, determining the maximum intersection ratio of the umbrella detection area to the human body detection area, and after the maximum intersection ratios of the umbrella detection area to the non-vehicle detection area are both less than a preset first intersection ratio threshold, determining the umbrella detection area to be a target umbrella detection area with illegal umbrella opening behavior, the method further comprises:
judging whether the umbrella detection area is positioned in a preset rule area range, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella opening behaviors;
If the umbrella detection area is a moving position area, after judging that the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and the method further comprises the following steps:
judging whether the umbrella detection areas are all located in a preset rule area range in the image with the preset frame number, and if so, determining that the umbrella detection areas are target umbrella detection areas with illegal umbrella supporting behaviors.
6. The method of claim 1, wherein after said determining a target umbrella detection area for which there is an offending umbrella deployment activity, the method further comprises:
And outputting alarm information for prompting the existence of illegal umbrella opening behaviors, and carrying the position information of the target umbrella detection area in the alarm information.
7. A device for detecting and identifying an offensive umbrella, said device comprising:
The first determining module is used for acquiring a video to be detected, identifying images in the video to be detected and determining each umbrella detection area, human body detection area and non-motor vehicle detection area in the images;
The second determining module is used for determining the intersection ratio of the umbrella detection area and each human detection area according to each umbrella detection area, and determining the intersection ratio of the umbrella detection area and each non-motor vehicle detection area;
The detection module is used for determining a target umbrella detection area with illegal umbrella supporting behaviors from each umbrella detection area according to the intersection ratio of each umbrella detection area to each human detection area, wherein the intersection ratio of each umbrella detection area to each non-motor vehicle detection area;
The detection module is specifically configured to determine, for each umbrella detection area, whether the umbrella detection area is a fixed position area or a mobile position area according to the position information of the umbrella detection area and the position information of the umbrella detection area corresponding to the image spaced by a preset frame number from the image; if the umbrella detection area is a fixed position area, judging whether the maximum intersection ratio of the umbrella detection area and the human body detection area and the maximum intersection ratio of the umbrella detection area and the non-motor vehicle detection area are smaller than a preset first intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area; if the umbrella detection area is a moving position area, judging whether the maximum intersection ratio of the umbrella detection area and a preset type of non-motor vehicle detection area is larger than a preset second intersection ratio threshold value, if so, determining that the umbrella detection area is a target umbrella detection area with illegal umbrella supporting behaviors, and if not, determining that the umbrella detection area is not the target umbrella detection area.
8. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1-6 when executing a program stored on a memory.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110843093.4A CN113470009B (en) | 2021-07-26 | 2021-07-26 | Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110843093.4A CN113470009B (en) | 2021-07-26 | 2021-07-26 | Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113470009A CN113470009A (en) | 2021-10-01 |
| CN113470009B true CN113470009B (en) | 2024-05-14 |
Family
ID=77882427
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110843093.4A Active CN113470009B (en) | 2021-07-26 | 2021-07-26 | Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113470009B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113688958A (en) * | 2021-10-26 | 2021-11-23 | 城云科技（中国）有限公司 | Filtering method, device and system suitable for target identification data |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105096585A (en) * | 2014-05-09 | 2015-11-25 | 陈时军 | Traffic fine management method and apparatus based on floats |
| JP2019109656A (en) * | 2017-12-18 | 2019-07-04 | 東芝情報システム株式会社 | Information management system for light vehicle/light motorcycle |
| CN109993031A (en) * | 2017-12-29 | 2019-07-09 | 杭州海康威视数字技术股份有限公司 | A kind of animal-drawn vehicle target is driven against traffic regulations behavioral value method, apparatus and camera |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
| CN111626990A (en) * | 2020-05-06 | 2020-09-04 | 北京字节跳动网络技术有限公司 | Target detection frame processing method and device and electronic equipment |
| CN111709340A (en) * | 2020-06-09 | 2020-09-25 | 杭州云视通互联网科技有限公司 | Method and system for detecting using behavior of umbrella |
| CN112349087A (en) * | 2019-08-07 | 2021-02-09 | 北京博研智通科技有限公司 | Visual data input method based on holographic perception of intersection information |
| CN112560807A (en) * | 2021-02-07 | 2021-03-26 | 南京云创大数据科技股份有限公司 | Crowd gathering detection method based on human head detection |
| CN112651293A (en) * | 2020-10-30 | 2021-04-13 | 华设设计集团股份有限公司 | Video detection method for road illegal stall setting event |
-
2021
- 2021-07-26 CN CN202110843093.4A patent/CN113470009B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105096585A (en) * | 2014-05-09 | 2015-11-25 | 陈时军 | Traffic fine management method and apparatus based on floats |
| JP2019109656A (en) * | 2017-12-18 | 2019-07-04 | 東芝情報システム株式会社 | Information management system for light vehicle/light motorcycle |
| CN109993031A (en) * | 2017-12-29 | 2019-07-09 | 杭州海康威视数字技术股份有限公司 | A kind of animal-drawn vehicle target is driven against traffic regulations behavioral value method, apparatus and camera |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
| CN112349087A (en) * | 2019-08-07 | 2021-02-09 | 北京博研智通科技有限公司 | Visual data input method based on holographic perception of intersection information |
| CN111626990A (en) * | 2020-05-06 | 2020-09-04 | 北京字节跳动网络技术有限公司 | Target detection frame processing method and device and electronic equipment |
| CN111709340A (en) * | 2020-06-09 | 2020-09-25 | 杭州云视通互联网科技有限公司 | Method and system for detecting using behavior of umbrella |
| CN112651293A (en) * | 2020-10-30 | 2021-04-13 | 华设设计集团股份有限公司 | Video detection method for road illegal stall setting event |
| CN112560807A (en) * | 2021-02-07 | 2021-03-26 | 南京云创大数据科技股份有限公司 | Crowd gathering detection method based on human head detection |
Non-Patent Citations (4)
| Title |
|---|
| Method of Automated Detection of Traffic Violation with a Convolutional Neural Network;S.R. Ibadov et al.;EPJ Web of Conferences;全文 * |
| 低空无人机辅助城市违章建筑测量;林贤恩;;福建建材(04);全文 * |
| 基于ＹＯＬＯｖ３的特定电力作业场景下的违规操作识别算法;丘浩等;电力科学与技术学报;第36卷(第3期);全文 * |
| 基于目标检测和语义分割的人行道违规停车检测;赵逸如;刘正熙;熊运余;严广宇;;现代计算机(09);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113470009A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN111563494B (en) | Behavior identification method and device based on target detection and computer equipment | |
| CN103279756B (en) | Vehicle detection based on integrated classifier analyzes system and determination method thereof | |
| CN110738857B (en) | Vehicle violation evidence obtaining method, device and equipment | |
| CN109190488B (en) | Front vehicle door opening detection method and device based on deep learning YOLOv3 algorithm | |
| CN109993031A (en) | A kind of animal-drawn vehicle target is driven against traffic regulations behavioral value method, apparatus and camera | |
| CN109740420A (en) | Vehicle illegal recognition methods and Related product | |
| CN108804987B (en) | Door opening and closing state detection method and device and people flow detection system | |
| CN109993138A (en) | A kind of car plate detection and recognition methods and device | |
| CN102902983B (en) | A kind of taxi identification method based on support vector machine | |
| CN112597928B (en) | Event detection method and related device | |
| CN111274886B (en) | Deep learning-based pedestrian red light running illegal behavior analysis method and system | |
| CN109460787A (en) | IDS Framework method for building up, device and data processing equipment | |
| CN112163525B (en) | Event type prediction method and device, electronic equipment and storage medium | |
| CN111914656A (en) | Personnel behavior detection method and device, electronic equipment and storage medium | |
| CN110674887A (en) | End-to-end road congestion detection algorithm based on video classification | |
| CN110245673A (en) | Method for detecting parking stalls and device | |
| CN113470009B (en) | Illegal umbrella opening detection and identification method and device, electronic equipment and storage medium | |
| Kejriwal et al. | Vehicle detection and counting using deep learning basedYOLO and deep SORT algorithm for urban traffic management system | |
| CN112418303A (en) | Training method and device for recognizing state model and computer equipment | |
| CN110458013B (en) | Traffic abnormal event detection method based on instance-level attention mechanism | |
| CN114141022A (en) | Emergency lane occupation behavior detection method and device, electronic equipment and storage medium | |
| Arul et al. | Modelling and Simulation of Smart Traffic Light System for Emergency Vehicle using Image Processing Techniques | |
| Hu et al. | An image-based crash risk prediction model using visual attention mapping and a deep convolutional neural network | |
| Susanto et al. | The implementation of intelligent systems in automating vehicle detection on the road |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |