CN115205794A - Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle - Google Patents
Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle Download PDFInfo
- Publication number
- CN115205794A CN115205794A CN202210251039.5A CN202210251039A CN115205794A CN 115205794 A CN115205794 A CN 115205794A CN 202210251039 A CN202210251039 A CN 202210251039A CN 115205794 A CN115205794 A CN 115205794A
- Authority
- CN
- China
- Prior art keywords
- motor vehicle
- violation
- monitoring image
- detection
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a medium for identifying violation of a non-motor vehicle. Wherein, the method comprises the following steps: acquiring a monitoring image to be identified, and determining whether the definition of the monitoring image meets the standard; if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image; and determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle. The technical scheme can quickly and effectively identify the violation behaviors of the non-motor vehicles.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a device, equipment and a medium for identifying violation of regulations of a non-motor vehicle.
Background
The non-motor vehicles in road traffic, including shed-free tricycles, electric motorcycles and the like, generally have the conditions of poor mechanical performance and poor balance, so that traffic accidents are more likely to occur when the non-motor vehicles are illegally driven compared with other vehicles. And the harmfulness of the driver to the illegal driving of the non-motor vehicle is not generally known enough, so that the motor tricycle is convenient and lucky at times, and the motor tricycle (electric motor car) is used as a passenger-cargo dual-purpose vehicle. In case of an accident, the driver and passengers on the motor tricycle are thrown out of the motor tricycle, and the motor tricycle is more seriously injured. It is even more dangerous if the tricycle is sitting on the back without any self-protection ability. In order to further restrain the traffic violation behaviors of motor tricycles and electric tricycles and strictly prevent road traffic accidents, the common practice of traffic management departments in various regions at present is to adopt increased hands to carry out special treatment, and the limited police resources are greatly consumed.
At present, illegal behaviors of a non-motor vehicle are detected by adopting a multi-stage multi-model comprehensive detection system, and the detection of the illegal behaviors of the non-motor vehicle is divided into a method of target detection, target classification and comprehensive judgment by combining human face and license plate information. The accuracy and the real-time performance of judging the illegal driving condition of the non-motor vehicle are poor.
Disclosure of Invention
The invention provides a method, a device, equipment and a medium for identifying violation of regulations of a non-motor vehicle, which can effectively support the requirements of time dimension analysis and life cycle management of data, can reduce merging overhead for large-scale updated use scenes, enables data files to be distributed in sequence on a time range, and improves the overall performance of reading and writing of a system.
According to an aspect of the present invention there is provided a method of violation identification for a non-motorized vehicle, the method comprising:
acquiring a monitoring image to be identified, and determining whether the definition of the monitoring image meets the standard;
if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image;
and determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
Optionally, the violation detection model includes: the system comprises an input end, a Backbone network, a Neck and a Prediction module;
the input end is used for enhancing the monitoring image;
the Backbone network of the backhaul is used for carrying out multi-scale feature extraction;
the Neck is used for aggregating semantic features and position features of the detection target based on the feature map pyramid network;
the Prediction predicting module is used for predicting the normal driving score and the illegal driving score of the non-motor vehicle for the detection target.
Optionally, the training process of the input end includes:
acquiring a preset number of candidate enhancement algorithms;
screening out at least two image enhancement algorithms from the candidate enhancement algorithms;
and determining enhancement probability values of the at least two image enhancement algorithms, and performing enhancement processing on the sample by adopting at least one image enhancement algorithm based on the probability values to obtain a sample enhancement result as input data.
Optionally, the training process of the backhaul Backbone network includes:
carrying out down-sampling operation on the sample through a Focus module;
and performing multi-scale feature extraction based on the down-sampling result through a CSPDarknet53 network.
Optionally, the training process of the tack includes:
performing maximum pooling on the characteristics of the samples through an SPP module to obtain a multi-scale characteristic diagram;
detecting the multi-scale characteristic diagram through an FPN module to obtain a detection target;
and aggregating the semantic features and the position features of the detection target through a PAN module.
Optionally, after the non-motor vehicle violation detection is performed to obtain the normal driving score and the violation driving score of the non-motor vehicle of each detection target in the monitoring image, the method further includes:
adopting a non-maximum value inhibition method to inhibit non-maximum value elements in the non-motor vehicle normal driving score and non-maximum value elements in the non-motor vehicle illegal driving score;
and performing weighted calculation on the inhibition result based on the dynamic weight to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle.
Optionally, the optimization function of the violation detection model adopts a random gradient descent method with momentum added;
the learning rate is controlled by a multi-step learning rate function.
According to another aspect of the present invention there is provided a violation identification device for a non-motor vehicle comprising:
the monitoring image acquisition module is used for acquiring a monitoring image to be identified and determining whether the definition of the monitoring image meets the standard;
the violation detection module is used for determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model if the non-motor vehicles and the personnel in the monitoring image are detected, and performing non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image;
and the violation identification module is used for determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform a method of non-motor vehicle violation identification in accordance with any of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to implement a method of non-motor vehicle violation identification according to any of the embodiments of the present invention when executed.
According to the technical scheme of the embodiment of the invention, whether the definition of the monitoring image meets the standard or not is determined by acquiring the monitoring image to be identified; if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and performing non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image; and determining the violation identification result of the non-motor vehicle according to the normal running score and the violation driving score of the non-motor vehicle. The technical scheme can quickly and effectively identify the violation behaviors of the non-motor vehicle.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will be readily apparent from the following specification.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a method for violation identification of a non-motor vehicle in accordance with an embodiment of the present invention;
fig. 2 is a flowchart of an image clarity determining method in a violation identification method for a non-motor vehicle according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a violation detection model of a non-motorized vehicle provided by a second embodiment of the present invention;
FIG. 4 is a flow chart of a violation identification method for a non-motor vehicle according to a third embodiment of the present invention;
FIG. 5 is a schematic view of a violation identification device of a non-motor vehicle according to a fourth embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," "target," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a flow chart of a violation identification method for a non-motor vehicle according to an embodiment of the present invention, which is applicable to a case of performing violation identification on the non-motor vehicle, and the method can be performed by a violation identification device for the non-motor vehicle, which can be implemented in hardware and/or software, and the violation identification device for the non-motor vehicle can be configured in an electronic device with data processing capability. As shown in fig. 1, the method includes:
s110, acquiring a monitoring image to be identified, and determining whether the definition of the monitoring image meets the standard.
The monitoring image can be an image frame in a picture or a video acquired by an image acquirer installed on a road, and the image acquirer installed on the road can be a gun-type camera, a spherical camera, a hemispherical camera or other equipment with an image acquisition function. For example, the monitoring image may be a video image obtained by converting video content of a traffic intersection shot by a dome camera into continuous image frames through a video access system.
It will be appreciated that the monitoring image may or may not be present with a non-motorized vehicle. In order to identify the violation of the non-motor vehicle and improve the efficiency of the violation of the non-motor vehicle, the monitoring image to be identified should be an image frame in which the non-motor vehicle exists. Wherein, the non-motor vehicle can be a bicycle, a manpower tricycle, an animal-drawn vehicle, a motor-driven wheelchair for the disabled, an electric bicycle or the like. In the embodiment of the invention, whether the non-motor vehicle exists in the acquired monitoring image to be identified or not is determined, and the identification can be carried out through an artificial intelligence visual analysis algorithm.
In addition, the monitoring image inevitably causes distortion to a certain degree in the processes of acquisition, compression, transmission and storage, and in order to improve the violation identification accuracy of the non-motor vehicle, the definition of the acquired monitoring image to be identified needs to be judged. Wherein the definition of the image may be determined by using the resolution. For example, the definition of the monitoring image may be determined by calculating a Brenner gradient function, a Tenengrad gradient function, an SMD function, or an energy gradient function of the resolution of the monitoring image, and classifying different levels according to a preset range according to the calculated result. In the embodiment of the invention, the standard can be determined according to actual needs.
Specifically, fig. 2 is a flowchart of an image sharpness determining method in a violation identification method for a non-motor vehicle according to an embodiment of the present invention. As shown in fig. 2, the method comprises steps A1-A4:
a1, obtaining an original monitoring image, and preprocessing the original monitoring image to obtain a monitoring image to be identified, wherein the monitoring image to be identified is a gray image of the original monitoring image;
the original monitoring image is grayed to obtain the monitoring image to be identified, and the graying of the original monitoring image can be performed by directly calling a function method, a maximum value method, an average value method, a weighted average value method or a gamma correction method.
A2, respectively determining a first-order gray level image gradient value and a second-order gray level image gradient value of the monitored image to be identified;
the first-order gray image gradient value can be determined through a Tenengrad gradient function, and the second-order gray image gradient value can be determined through a Laplacian gradient function.
A3, carrying out weighted summation on the gradient value of the first-order gray level image and the gradient value of the second-order gray level image, and determining the definition of the monitoring image to be identified;
the definition of the monitoring image to be identified can be determined by the following formula:
X＝V 1 ×a+V 2 ×b；
wherein X is the definition of the monitored image to be identified, V 1 Is a first order gray scale image gradient value, V 2 And a and b are weighting coefficients. Preferably, a =0.6, b =0.4.
And A4, if the definition of the monitored image to be identified is greater than a predetermined definition threshold value, determining that the definition of the monitored image meets the standard.
The definition of the monitoring image is determined, and the image which accords with the standard definition is screened out, so that the monitoring image with fuzzy picture quality can be filtered, and the violation identification efficiency of the non-motor vehicle can be improved.
And S120, if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image.
After the monitored image to be identified which accords with the standard definition is obtained, the violation detection model can determine the non-motor vehicles and people detected by the target, and further detect the violation behaviors. The violation detection model can be used for target identification and detection of non-motor vehicle violation behaviors. For example, the violation detection model may be a convolutional neural network model, a cyclic neural network, or a deep neural network. The non-motor vehicle violation behaviors can include signal running, parking crossing, driving in a motor vehicle lane, riding a rider, reverse driving, blocking head and turning, robbing the lane and running, and disorderly parking and disorderly putting; pedestrians run signals, do not walk the sidewalk, cross the road without walking the crosswalk line, cross the road obliquely and cross the guardrail; human-powered vehicles, human-powered tricycles, animal-powered vehicles and the like.
In the embodiment of the invention, the violation detection model is trained in advance, specifically, the initial violation detection model can be trained according to the monitoring image and the recognition result which are input in advance and used for training, and the image characteristics are extracted. The image characteristics can be characteristics which can clearly distinguish the non-motor vehicle violation behaviors in the image, and if one non-motor vehicle target and two personnel targets exist in the same area in the image, the non-motor vehicle violation behaviors are indicated. In addition, the violation detection model can also detect the type of non-motor vehicles in the monitoring image. It will be appreciated that the shed tricycle in the non-motor vehicle is manned and the electric bicycle is not manned, so that the requirements for the detection of the violation of the traffic are different, and different violation detection rules can be set according to the type of non-motor vehicle detected by the violation detection model.
In the embodiment of the invention, the non-motor vehicles and the personnel in the monitoring image are determined as the detection target through the pre-trained violation detection model, and the non-motor vehicle violation detection is carried out. The monitoring image can be detected through a violation detection model to obtain all non-motor vehicles and personnel in the image; determining a non-motor vehicle area according to the position relation between the non-motor vehicle and the personnel marked in the image; and determining whether the non-motor vehicle is subjected to violation behaviors or not according to the number and the positions of the non-motor vehicles and the personnel in each non-motor vehicle area. It will be appreciated that one non-motor vehicle zone may be detected for the same surveillance image, or at least two non-motor vehicle zones may be detected.
In the embodiment of the invention, the normal driving score and the illegal driving score of the non-motor vehicle of each detection target in the monitoring image can be determined according to the confidence coefficient of the normal driving behavior of the non-motor vehicle, the confidence coefficient of the illegal driving behavior and the preset confidence coefficient which are determined by the illegal detection model, or can be directly output by the illegal detection model which is trained in advance.
S130, determining a violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
The violation identification result can be a result of identifying the driving behavior of the non-motor vehicle in the monitoring image, and can be legal, violation manned, reverse driving and the like. The violation identification result can be displayed by sending an alarm signal through the violation detection model, and the alarm content can comprise the position of the non-motor vehicle driving against the violation, the confidence of the judgment result and the position information (such as IP address and number) of the current monitoring camera.
According to the technical scheme provided by the embodiment of the invention, whether the definition of the monitoring image meets the standard or not is determined by acquiring the monitoring image to be identified; if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image; and determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle. The technical scheme can quickly and effectively identify the violation behaviors of the non-motor vehicles.
Example two
Fig. 3 is a schematic diagram of a violation detection model of a non-motor vehicle according to a second embodiment of the present invention. As shown in fig. 3, the detection model of the present embodiment specifically includes: the device comprises an input end, a Backbone network, a Neck and a Prediction module. Wherein the content of the first and second substances,
the input end 310 is configured to perform enhancement processing on the monitored image;
the backhaul Backbone network 320 is used for performing multi-scale feature extraction;
the Neck330 is used for aggregating the semantic features and the position features of the detection target based on the feature map pyramid network;
the Prediction module 340 is configured to predict the normal driving score of the non-motor vehicle and the illegal driving score of the non-motor vehicle for the detection target.
The training process of the input terminal 310 includes steps B1-B3:
b1: a preset number of candidate enhancement algorithms is obtained.
In order to guarantee the convergence of the model, the generalization capability of the model can be improved to the greatest extent, and the accuracy of the violation detection model can be improved by screening the image enhancement algorithm with better combination performance.
Among them, the candidate enhancement algorithms may be the following 26: a1= Blur, smooth filtering of image mean; a2= motion _ blu, motion blur; a3= medianburr, median filtering; a4= sharp, image sharpening enhancement; a5= FancyPCA, fancyPCA image enhancement; a6= GaussianBlur, gaussian blur; a7= GaussNoise, gaussian noise; a8= iaaadditivegassinannoise, adaptive gaussian noise; a9= isonose, camera sensor noise; a10= RandomFog, random simulated image fog enhancement; a11= CLAHE, adaptive histogram equalization; a12= huespaturitionvalue, random hue, saturation modification; a13= RandomBrightnessContrast, random brightness, contrast modification; a14= channelsuffle, random channel rearrangement; a15= optical distortion; a16= RandomGamma, random gamma noise; a17= ColorJitter, random color dithering; a18= VerticalFlip, vertical flip; a19= horizon flip, horizontal flip; a20= Transpose, image Transpose; a21= rotarandom, random angular rotation; a22 = shiftscaleprotate, affine transformation; a23= gridsistration, mesh distortion; a24 = randomgridshouffle, random grid arrangement; a25= GridDropout, grid erase; a26= elastic transform.
It should be noted that, the embodiment of the present invention does not limit the preset number of candidate enhancement algorithms, and may select the candidate enhancement algorithms according to actual needs.
B2: and screening out at least two image enhancement algorithms from the candidate enhancement algorithms.
The screening method of the image enhancement algorithm can be random combination or artificial combination. Specifically, a combination of a1 and a11 may be used, or a combination of a1, a6, a17, and a23 may be used. It should be noted that the embodiment of the present invention does not limit the kind and number of algorithms in the combined image enhancement algorithm.
In the embodiment of the invention, the candidate enhancement algorithm can be screened by a Mosaic data enhancement method.
B3: and determining enhancement probability values of the at least two image enhancement algorithms, and performing enhancement processing on the sample by adopting at least one image enhancement algorithm based on the probability values to obtain a sample enhancement result as input data.
The probability value may be a percentage of the number of images that can achieve a preset image enhancement effect in the number of all training images by using an image enhancement algorithm. For example, 100 training images are obtained, and 30 images using the gaussian blur algorithm achieve a preset image enhancement effect, so that the probability value of the gaussian blur algorithm is 0.3. The enhancement probability value may be a sum of probability values of the filtered at least two image enhancement algorithms.
By combining and screening the image enhancement algorithm and enhancing the monitored image based on the enhanced probability value, the generalization capability of the model can be improved to the greatest extent while the convergence of the model is ensured.
The training process of the Backbone network 320 includes steps C1-C2:
c1: and performing downsampling operation on the sample through a Focus module.
The Focus module may implement downsampling without information loss, specifically, before the monitoring image after image enhancement enters the Backbone network of Backbone 320, slice the picture, and take one value for every other pixel in one picture, which is similar to adjacent sampling, so that four complementary pictures are taken, but no information is lost. That is to say, the W and H information is concentrated in the channel space, the input channels are expanded by 4 times, that is, the spliced picture is changed into 12 channels relative to the original RGB three-channel mode, and the obtained new picture is subjected to convolution operation, so that a double-sampling feature map without information loss is finally obtained.
Illustratively, a monitoring image with an original size of 640 × 640 × 3 is input into a Focus module, and is first changed into a 320 × 320 × 12 feature map by a slicing operation, and then is finally changed into a 320 × 320 × 32 feature map by a convolution operation.
C2: and performing multi-scale feature extraction based on the down-sampling result through a CSPDarknet53 network.
The CSPDarknet53 network adopts a classic yolo backbone network, and a plurality of scale features of large scale, medium scale and small scale are respectively extracted based on a down-sampling result.
The training process of the tack 330 comprises the steps D1-D3:
d1: and performing maximum pooling on the features of the samples through an SPP module to obtain a multi-scale feature map.
In the SPP module, feature maps of different scales are obtained by downsampling using a maximum pooling method of k = {1 × 1,5 × 5,9 × 9,13 × 13 }.
D2: and detecting the multi-scale characteristic diagram through the FPN module to obtain a detection target.
The FPN is a characteristic diagram pyramid network, mainly solves the multi-scale problem in object detection, and greatly improves the performance of small object detection by simple network connection change under the condition of not increasing the calculated amount of an original model basically. In the embodiment of the present invention, the detection target includes both a near view target (large) and a far view target (small), so that multi-scale training is required for target detection.
D3: and aggregating the semantic features and the position features of the detection target through a PAN module.
The PAN module is a bottom-up pyramid of features behind the FPN module. Which contains two PAN structures. The FPN module conveys strong semantic features from top to bottom, and the feature pyramid conveys strong positioning features from bottom to top, and every two are connected and aggregated from the features. The feature map in the deep layer carries stronger semantic features and weaker positioning information. And the shallow feature map carries strong position information and weak semantic features. The FPN module transfers deep semantic features to a shallow layer, so that semantic expression on multiple scales is enhanced. And the PAN module conducts the shallow positioning information to the deep layer in contrast, so that the positioning capacity on multiple scales is enhanced.
The Prediction module 340 is used for predicting the normal driving score and the illegal driving score of the non-motor vehicle for the detection target.
The input vector of the Prediction module 340 is 3 branches output by the FPN module, and the Prediction head is output by two-layer convolution. The final output here is in the form: batchSize X (4 + total number of classes) feature map width. Where 4 is predicted x, y, w and h. In the embodiment of the invention, the total number of the categories is 2, and the score is the normal driving score of the non-motor vehicle and the score is the illegal driving score of the non-motor vehicle.
According to the technical scheme provided by the embodiment of the invention, the violation detection model is trained in advance, so that the image display effect can be enhanced, and the convergence speed and generalization capability of the model can be improved.
EXAMPLE III
Fig. 4 is a flowchart of a violation identification method for a non-motor vehicle according to a third embodiment of the present invention, which is optimized based on the above embodiments in the present application. The concrete optimization is as follows: after the non-motor vehicle violation detection is carried out to obtain the non-motor vehicle normal driving score and the non-motor vehicle violation driving score of each detection target in the monitoring image, the method further comprises the following steps: adopting a non-maximum value inhibition method to inhibit non-maximum value elements in the non-motor vehicle normal driving score and non-maximum value elements in the non-motor vehicle illegal driving score; and performing weighted calculation on the inhibition result based on the dynamic weight to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle. As shown in fig. 4, the method includes:
s410, acquiring a monitoring image to be identified, and determining whether the definition of the monitoring image meets the standard.
And S420, if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and performing non-motor vehicle violation detection to obtain the non-motor vehicle normal driving score and the non-motor vehicle violation driving score of each detection target in the monitoring image.
And S430, determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
And S440, adopting a non-maximum value inhibition method to inhibit non-maximum value elements in the normal driving score of the non-motor vehicle and non-maximum value elements in the illegal driving score of the non-motor vehicle.
It can be understood that when the violation detection model is used for identifying and detecting a detection target in a monitoring image, a plurality of non-maximum value elements may be obtained according to the monitoring image, and if the overlapping degree of some non-maximum value elements is high, the accuracy of the violation detection model in identifying and detecting the target is affected. Therefore, in the embodiment of the present invention, a Non Maximum Suppression (NMS) algorithm of a dynamic threshold is used to filter out overlapped targets and improve the accuracy of detection.
The nature of the non-maximum value inhibition method is to search local maximum values and inhibit non-maximum value elements. The non-maximum value suppression method is to find a bounding box boundary box with higher confidence coefficient according to the coordinate information of the score and the region.
Specifically, the method may comprise steps E1-E3:
e1: sequencing the confidence degrees of all the detection target prediction frames in the monitoring image in a descending order;
wherein the confidence level of the detection target prediction box may comprise a non-motor vehicle normal driving score confidence level and a non-motor vehicle violation driving score confidence level.
E2: taking the detection target prediction frame with the highest confidence coefficient as a correct prediction frame, and respectively calculating IoU (Intersection over Unit) values of the correct prediction frame and the rest detection target prediction frames in the monitored image;
the IoU value is a measure of the accuracy with which the corresponding object is detected in a particular data set.
E3: and if the IoU value is larger than a threshold value T, rejecting a detection target prediction frame corresponding to the IoU value.
S450, carrying out weighted calculation on the inhibition result based on the dynamic weight to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle.
In order to avoid adopting the NMS algorithm to filter out the blocked violating targets, before step B1, the confidence of the detected target prediction box may be weighted, that is, a dynamic weight (ω) is added 1 ，ω 2 ). Specifically, the confidence of the detected target prediction box is(s) 1 ，s 2 ) The weighted confidence is(s) 1 ×ω 1 ，s 2 ×ω 2 ). It can be understood that the dynamic weight value can be adjusted according to the requirements of the actual situation. Preferably, ω is 1 ＝1，ω 2 ＝0.6。
According to the technical scheme provided by the embodiment of the invention, whether the definition of the monitoring image meets the standard or not is determined by acquiring the monitoring image to be identified; if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image; determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle; adopting a non-maximum value inhibition method to inhibit non-maximum value elements in the non-motor vehicle normal driving score and non-maximum value elements in the non-motor vehicle illegal driving score; and performing weighted calculation on the inhibition result based on the dynamic weight to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle. According to the technical scheme, overlapped targets can be effectively filtered, and the detection accuracy is improved.
On the basis of the above embodiments, optionally, the optimization function of the violation detection model adopts a random gradient descent method with momentum added; the learning rate is controlled by a multi-step learning rate function.
The direction of the gradient is the direction in which the function rises fastest at a given point, and then the reverse direction of the gradient is the direction in which the function falls fastest at the given point, so that when the gradient is reduced, the weight is updated along the reverse direction of the gradient, and the global optimal solution can be effectively found. The random gradient descent method is to randomly extract one group from samples, update the group according to the gradient after training, and then extract and update the group.
In the embodiment of the invention, the optimization function of the violation detection model adopts a random gradient descent method with momentum added, if the gradient symbols at the current time and the last time are the same, the descent amplitude can be accelerated, and the problem of too slow descent in the prior art is solved; if the signs of the gradients of the current time and the last time are opposite, the gradients of the current time and the last time are mutually inhibited, and the oscillation is slowed down; in addition, due to the action of momentum, when the local optimum is achieved, the device can jump out by virtue of the momentum, and is not easy to fall into a local optimum point.
Preferably, the momentum may be 0.9.
In addition, the multiple-step learning rate function can effectively control the training effect and the convergence degree of the violation detection model. Preferably, the multi-step learning rate function may be [50,75,90].
The method has the advantages that the convergence speed and the generalization capability of the violation detection model can be effectively improved by matching and using the random gradient descent method for adding momentum and the multi-step learning rate function.
Example four
Fig. 5 is a schematic structural diagram of a violation identification device for a non-motor vehicle according to a fourth embodiment of the present invention, where the device is capable of executing a violation identification method for a non-motor vehicle according to any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method. As shown in fig. 5, the apparatus includes:
a monitoring image obtaining module 510, configured to obtain a monitoring image to be identified, and determine whether a definition of the monitoring image meets a standard;
the violation detection module 520 is used for determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model if the non-motor vehicles and the personnel in the monitoring image are detected, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image;
and the violation identification module 530 is used for determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
According to the technical scheme of the embodiment of the invention, whether the definition of the monitoring image meets the standard or not is determined by acquiring the monitoring image to be identified; if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the illegal driving scores of the non-motor vehicles of all the detection targets in the monitoring image; and determining the violation identification result of the non-motor vehicle according to the normal running score and the violation driving score of the non-motor vehicle. The technical scheme can quickly and effectively identify the violation behaviors of the non-motor vehicle.
Optionally, the violation detection model includes: the system comprises an input end, a Backbone network, a Neck and a Prediction module;
the input end is used for enhancing the monitoring image;
the Backbone network of the backhaul is used for carrying out multi-scale feature extraction;
the Neck is used for aggregating semantic features and position features of the detection target based on the feature map pyramid network;
the Prediction module is used for predicting the normal driving score and the illegal driving score of the non-motor vehicle of the detection target.
Optionally, the training process of the input end includes:
acquiring a preset number of candidate enhancement algorithms;
screening out at least two image enhancement algorithms from the candidate enhancement algorithms;
and determining enhancement probability values of the at least two image enhancement algorithms, and performing enhancement processing on the sample by adopting at least one image enhancement algorithm based on the probability values to obtain a sample enhancement result as input data.
Optionally, the training process of the backhaul Backbone network includes:
performing down-sampling operation on the sample through a Focus module;
and performing multi-scale feature extraction based on the down-sampling result through a CSPDarknet53 network.
Optionally, the training process of the tack includes:
performing maximum pooling on the characteristics of the sample through an SPP module to obtain a multi-scale characteristic map;
detecting the multi-scale characteristic diagram through the FPN module to obtain a detection target;
and aggregating the semantic features and the position features of the detection target through a PAN module.
Optionally, the apparatus further comprises:
the non-maximum value suppression module is used for suppressing non-maximum value elements in the non-motor vehicle normal driving score and non-maximum value elements in the non-motor vehicle violation driving score by adopting a non-maximum value suppression method after non-motor vehicle violation detection is carried out to obtain the non-motor vehicle normal driving score and the non-motor vehicle violation driving score of each detection target in the monitoring image;
and the inhibition post-processing module is used for carrying out weighted calculation on the inhibition result based on the dynamic weight so as to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle.
Optionally, the optimization function of the violation detection model adopts a random gradient descent method with momentum added;
the learning rate is controlled by a multi-step learning rate function.
The violation identification device of the non-motor vehicle provided by the embodiment of the invention can execute the violation identification method of the non-motor vehicle provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 6 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores computer programs executable by the at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer programs stored in the Read Only Memory (ROM) 12 or the computer programs loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
In some embodiments, the violation identification method of a non-motor vehicle may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more of the steps of the method described above for violation identification of a non-motor vehicle may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the violation identification method of the non-motor vehicle by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of violation identification for a non-motorized vehicle, the method comprising:
acquiring a monitoring image to be identified, and determining whether the definition of the monitoring image meets the standard or not;
if yes, determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image;
and determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
2. The method of claim 1 wherein the violation detection model comprises: the system comprises an input end, a Backbone network, a Neck and a Prediction module;
the input end is used for enhancing the monitoring image;
the Backbone network of the backhaul is used for carrying out multi-scale feature extraction;
the Neck is used for aggregating semantic features and position features of the detection target based on the feature map pyramid network;
the Prediction module of the Prediction is used for predicting the normal driving score and the illegal driving score of the non-motor vehicle for the detection target.
3. The method of claim 2, wherein the training process of the input comprises:
acquiring a preset number of candidate enhancement algorithms;
screening out at least two image enhancement algorithms from the candidate enhancement algorithms;
and determining enhancement probability values of the at least two image enhancement algorithms, and performing enhancement processing on the sample by adopting at least one image enhancement algorithm based on the probability values to obtain a sample enhancement result as input data.
4. The method according to claim 2, wherein the training process of the backhaul Backbone network comprises:
performing down-sampling operation on the sample through a Focus module;
and performing multi-scale feature extraction based on the down-sampling result through a CSPDarknet53 network.
5. The method of claim 2, wherein the training process of the tack comprises:
performing maximum pooling on the characteristics of the sample through an SPP module to obtain a multi-scale characteristic map;
detecting the multi-scale characteristic diagram through an FPN module to obtain a detection target;
and aggregating the semantic features and the position features of the detection target through a PAN module.
6. The method of claim 1 wherein after performing non-motor vehicle violation detection to obtain the non-motor vehicle normal driving score and the non-motor vehicle violation driving score for each detection target in the monitoring image, the method further comprises:
adopting a non-maximum value inhibition method to inhibit non-maximum value elements in the non-motor vehicle normal driving score and non-maximum value elements in the non-motor vehicle illegal driving score;
and performing weighted calculation on the inhibition result based on the dynamic weight to obtain a post-processing result of the normal driving score of the non-motor vehicle and a post-processing result of the illegal driving score of the non-motor vehicle.
7. The method of claim 1 wherein the optimization function of the violation detection model employs a stochastic gradient descent method incorporating momentum;
the learning rate is controlled by a multi-step learning rate function.
8. A violation identification device for a non-motorized vehicle, the device comprising:
the monitoring image acquisition module is used for acquiring a monitoring image to be identified and determining whether the definition of the monitoring image meets the standard;
the violation detection module is used for determining the non-motor vehicles and the personnel in the monitoring image as detection targets through a pre-trained violation detection model if the non-motor vehicles and the personnel in the monitoring image are detected, and carrying out non-motor vehicle violation detection to obtain the normal driving scores of the non-motor vehicles and the violation driving scores of the non-motor vehicles of all the detection targets in the monitoring image;
and the violation identification module is used for determining the violation identification result of the non-motor vehicle according to the normal running score of the non-motor vehicle and the violation driving score of the non-motor vehicle.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of violation identification of a non-motorized vehicle of any one of claims 1-7.
10. A computer readable storage medium characterized by computer instructions stored thereon for causing a processor to implement the method of non-motor vehicle violation identification of any of claims 1-7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210251039.5A CN115205794A (en) | 2022-03-15 | 2022-03-15 | Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210251039.5A CN115205794A (en) | 2022-03-15 | 2022-03-15 | Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115205794A true CN115205794A (en) | 2022-10-18 |
Family
ID=83574643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210251039.5A Pending CN115205794A (en) | 2022-03-15 | 2022-03-15 | Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115205794A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116883952A (en) * | 2023-09-07 | 2023-10-13 | 吉林同益光电科技有限公司 | Electric power construction site violation identification method and system based on artificial intelligence algorithm |
| CN117671972A (en) * | 2024-02-01 | 2024-03-08 | 北京交通发展研究院 | Vehicle speed detection method and device for slow traffic system |
-
2022
- 2022-03-15 CN CN202210251039.5A patent/CN115205794A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116883952A (en) * | 2023-09-07 | 2023-10-13 | 吉林同益光电科技有限公司 | Electric power construction site violation identification method and system based on artificial intelligence algorithm |
| CN116883952B (en) * | 2023-09-07 | 2023-11-17 | 吉林同益光电科技有限公司 | Electric power construction site violation identification method and system based on artificial intelligence algorithm |
| CN117671972A (en) * | 2024-02-01 | 2024-03-08 | 北京交通发展研究院 | Vehicle speed detection method and device for slow traffic system |
| CN117671972B (en) * | 2024-02-01 | 2024-05-14 | 北京交通发展研究院 | Vehicle speed detection method and device for slow traffic system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108725440B (en) | Forward collision control method and apparatus, electronic device, program, and medium | |
| EP3786835A1 (en) | Traffic image recognition method and apparatus, and computer device and medium | |
| CN115205794A (en) | Method, device, equipment and medium for identifying violation of regulations of non-motor vehicle | |
| CN110287778B (en) | Image processing method and device, terminal and storage medium | |
| US20150279036A1 (en) | Side window detection in near-infrared images utilizing machine learning | |
| CN101727672A (en) | Method for detecting, tracking and identifying object abandoning/stealing event | |
| CN111047874B (en) | Intelligent traffic violation management method and related product | |
| CN112287875A (en) | Abnormal license plate recognition method, device, equipment and readable storage medium | |
| CN113724286A (en) | Method and device for detecting saliency target and computer-readable storage medium | |
| CN111325171A (en) | Abnormal parking monitoring method and related product | |
| CN110782485A (en) | Vehicle lane change detection method and device | |
| CN116310993A (en) | Target detection method, device, equipment and storage medium | |
| CN113139488B (en) | Method and device for training segmented neural network | |
| CN115439692A (en) | Image processing method and device, electronic equipment and medium | |
| CN104112144A (en) | Person and vehicle identification method and device | |
| CN114120056A (en) | Small target identification method, small target identification device, electronic equipment, medium and product | |
| CN113806361B (en) | Method, device and storage medium for associating electronic monitoring equipment with road | |
| Sankaranarayanan et al. | Improved Vehicle Detection Accuracy and Processing Time for Video Based ITS Applications | |
| CN117975315B (en) | Warehouse geological disaster differential identification method based on unmanned aerial vehicle images | |
| Chithra et al. | Feature based multiple vehicle license plate detection and video based traffic counting | |
| CN114445711B (en) | Image detection method, image detection device, electronic equipment and storage medium | |
| CN116486324B (en) | Subway seat trampling behavior detection method, device, equipment and storage medium | |
| CN113012439B (en) | Vehicle detection method, device, equipment and storage medium | |
| CN116189422A (en) | Road traffic jam prediction method and device, electronic equipment and storage medium | |
| CN116665140A (en) | Method, device, equipment and storage medium for detecting shared single vehicle-mounted human behavior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |