CN112381859B - System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data - Google Patents
System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data Download PDFInfo
- Publication number
- CN112381859B CN112381859B CN202011313065.3A CN202011313065A CN112381859B CN 112381859 B CN112381859 B CN 112381859B CN 202011313065 A CN202011313065 A CN 202011313065A CN 112381859 B CN112381859 B CN 112381859B
- Authority
- CN
- China
- Prior art keywords
- target
- information
- video image
- video
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 144
- 238000012545 processing Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000012544 monitoring process Methods 0.000 claims abstract description 57
- 238000000605 extraction Methods 0.000 claims abstract description 43
- 238000001514 detection method Methods 0.000 claims description 29
- 230000008569 process Effects 0.000 claims description 24
- 230000006399 behavior Effects 0.000 claims description 23
- 239000013598 vector Substances 0.000 claims description 18
- 238000004364 calculation method Methods 0.000 claims description 16
- 238000013135 deep learning Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 239000003086 colorant Substances 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 5
- 238000010224 classification analysis Methods 0.000 claims description 4
- 230000037237 body shape Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 10
- 238000007726 management method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000002060 circadian Effects 0.000 description 3
- 238000012916 structural analysis Methods 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 239000000383 hazardous chemical Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000004382 visual function Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30236—Traffic on road, railway or crossing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a system for realizing intelligent analysis and identification processing aiming at video image data based on extraction of traffic security target elements, which comprises a traffic security video analysis module, a monitoring module and a control module, wherein the traffic security video analysis module is used for acquiring image data of a monitoring target to finish identification tracking; the video image supporting service module is used for carrying out attribute classification, analysis processing and feature extraction on the image data; and the video image service application module is used for performing behavior judgment and identification processing on the acquired parameters. The invention also relates to a method, a device, a processor and a computer readable storage medium for realizing intelligent analysis and identification processing for video image data based on traffic security target element extraction. The system, the method, the device, the processor and the computer readable storage medium for realizing intelligent analysis and identification processing aiming at video image data based on traffic security target element extraction are adopted, intelligent identification and law enforcement capability is enhanced through an intelligent analysis technology, and the intelligent level of urban supervision is effectively improved.
Description
Technical Field
The invention relates to the technical field of intelligent recognition and analysis of traffic security, in particular to the technical field of deep learning, and specifically relates to a system, a method, a device, a processor and a computer readable storage medium for realizing intelligent recognition and processing of video image data based on extraction of traffic security target elements.
Background
In recent years, with the development of deep learning and cloud computing technology, the analysis of fixed monitoring videos has been developed to a great extent, including video resources mainly including bayonet snap shots, electric police images and public security monitoring, and some of the video resources have been described in a structural way, so that support is provided for various police services of public security. In the advanced region of informatization construction, foundation platforms such as a video analysis center and video big data are also built, and the method plays an important role in public security business.
However, with the acceleration of urban evolution process in China, urban resident population is continuously increased, floating population is rapidly changed, various conflict of interests is caused, unstable factors still exist in a large quantity, and under the condition that police force is limited and illegal criminals have anti-reconnaissance consciousness public security, stable and orderly running of the city is better maintained, and tasks are becoming heavy. In order to adapt to the needs of new situations, social management is innovated, urban security and control capability under dynamic environment is improved, intelligent new police service is created in an effort, the specialization and intellectualization level of public security work is improved, and public security view application guided by the intellectualization of video images is enhanced.
In the traffic security video image, targets focused by different business demands are different, and any target (people, vehicles and behaviors thereof) in a scene can become a factor focused by public security police. The method is characterized by comprising the steps of realizing the fine management of social population and vehicles, carrying out comprehensive data management on ultra-large-scale video image resources by utilizing advanced technologies such as structuring, big data, deep learning and the like, realizing the convergence and archiving from massive video data, establishing a plurality of data models by combining various big data resources with actual services, and realizing the fine management and control of targets, thereby improving the roles of traffic public security video images in public security, social management, urban supervision and other services.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a system, a method, a device, a processor and a computer readable storage medium, wherein the system, the method, the device, the processor and the computer readable storage medium have high identification accuracy and can be used for realizing intelligent analysis and identification processing on video image data based on extraction of traffic security target elements.
In order to achieve the above object, the system for implementing intelligent analysis and identification processing for video image data based on extraction of traffic security target elements according to the present invention comprises:
The system for realizing intelligent analysis and identification processing of video image data based on extraction of traffic security target elements is mainly characterized by comprising the following components:
The traffic security video analysis module is used for acquiring video image data of a monitoring target from each video monitoring unit, analyzing video scenes, realizing monitoring and tracking of the target image and identification of road elements, and completing access processing of the video image data;
The video image supporting service module is connected with the traffic security video analysis module and is used for receiving the video image data, constructing a depth network model for the video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation on feature values on the data generated after the analysis processing, and outputting a target sequence similar to the target image in a target library;
The video image service application module is connected with the video image supporting service module and is used for receiving all attribute parameters of the monitored targets and performing behavior judgment and identification processing on the attribute parameters, and the monitoring, tracking and the distribution of the monitored targets are realized by detecting and tracking multiple targets, positioning the positions and track information of the monitored targets in the video image and drawing the running track of the monitored targets.
The method for realizing intelligent analysis and identification processing of video image data based on traffic security target element extraction is mainly characterized by comprising the following steps:
(1) Collecting video image data of the monitoring targets from each video monitoring unit, analyzing and judging video scenes, realizing monitoring and tracking of the concerned targets and identification of road elements, and completing access processing of the video image data;
(2) Constructing a depth network model from the acquired video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation of feature values on the data generated after the analysis processing, and outputting a target feature comparison sequence similar to the target image in a target library;
(3) And (3) performing behavior judgment and identification processing on the data generated in the step (2), and performing feature comparison on the target of interest by combining the target feature comparison sequence to realize monitoring tracking and control on the target of interest.
Preferably, the step (1) specifically includes the following steps:
The system of (1.1) collects the video image data from an electric police, a bayonet, a public security video, a vehicle-mounted device and an unmanned aerial vehicle video monitoring unit;
(1.2) calculating local brightness and average brightness of each frame of image from video frame information of each video monitoring unit, and analyzing and identifying a day and night mode of the target image by counting global brightness I G_avg and local brightness I Lb_avg of continuous frames;
(1.3) if the target image is detected to be in a daytime mode, entering a step (1.4); otherwise, if the target image is detected to be in a night mode, ending the whole process of intelligent analysis and identification processing on the video image data;
(1.4) carrying out multi-target tracking algorithm processing on the collected video image data, detecting a target image appearing in each frame of image, correlating the target image with a target detected in the previous frame of image, positioning the position of the target of interest appearing in the video image, and drawing a running track of the target, so as to realize multi-target detection tracking;
And (1.5) extracting lane line information and signal lamp information in the video image by combining brightness and saturation information of the video image data by using a traffic road element detection model, and taking the lane line information and the signal lamp information as different types of violation punishment reference bases.
Preferably, the step (2) specifically includes the following steps:
(2.1) constructing a depth network model for the target of interest detected and tracked by the video monitoring unit, detecting and identifying the target of interest, and extracting related information of the target of interest;
(2.2) performing coarse classification and fine classification attribute analysis on the detected and tracked target of interest by using a deep learning recognition model;
(2.3) extracting characteristic value information and characteristic attribute information of the target of interest after the classification analysis;
and (2.4) carrying out vector distance calculation of characteristic values on the extracted information data of the target of interest, and outputting a target sequence similar to the target of interest in the target library by taking the similarity score as an output result of characteristic value comparison.
More preferably, the attention target is a person, the related information of the person comprises a person object picture, person object feature value information and person object feature attribute information, the person object feature value information is sex, age and clothing, the person object feature attribute information is body feature and an accessory, the fine classification attribute of the person object is person attribute analysis information, and the person attribute analysis information comprises analysis information of illegal behaviors that the number of non-motor vehicle riders exceeds the standard or the number of the non-motor vehicle riders does not wear a helmet, and motor vehicle drivers and copiers do not tie a safety belt and make a call;
More preferably, the target of interest is a motor vehicle, the related information of the motor vehicle includes a motor vehicle object picture, motor vehicle object feature value information and motor vehicle object feature attribute information, the motor vehicle object feature value information is a license plate, a vehicle type, a vehicle brand, a vehicle color and a driving direction, the motor vehicle object feature attribute information is no-load, cargo or overload of the motor vehicle, the sub-classification attribute of the motor vehicle object is motor vehicle attribute analysis information, and the motor vehicle attribute analysis information includes overload, retrograde, red light running or illegal lane changing of the motor vehicle;
More preferably, the target of interest is a non-motor vehicle, the related information of the non-motor vehicle includes a non-motor vehicle object picture, non-motor vehicle object feature value information and non-motor vehicle object feature attribute information, the non-motor vehicle object feature value information is a non-motor vehicle type, color and running direction, the non-motor vehicle object feature attribute information is the number, body shape, clothing, body appearance feature and attachment of the non-motor vehicle riders, the sub-classification attribute of the non-motor vehicle object is non-motor vehicle attribute analysis information, and the non-motor vehicle attribute analysis information is non-motor vehicle lane change running, excessive number of riders or no helmet wear of the riders.
Preferably, the step (3) specifically includes the following steps:
(3.1) identifying the illegal behaviors of the non-motor vehicles in lane changing running, exceeding the number of pedestrians and not wearing helmets by combining the non-motor vehicle information and the personnel information on the non-motor vehicles analyzed in the step (2.3) with the traffic road element detection model, storing the illegal information, pictures and videos of the illegal processes, and providing accurate and effective illegal punishment evidence for public security traffic;
(3.2) judging, identifying and processing the motor vehicle information and the personnel information on the motor vehicle analyzed in the step (2.3) in combination with the traffic road element detection model, and judging and processing the motor vehicle overload, retrograde, red light running, illegal lane changing, and the illegal behaviors that the drivers and copiers on the motor vehicle are not belted and make calls, storing the information, pictures and videos of the illegal motor vehicle process, and providing more accurate and effective illegal punishment evidence for public security traffic;
(3.4) comparing and sequencing the personnel attribute analysis information, the non-motor vehicle attribute analysis information and the motor vehicle attribute analysis information with the target sequence in characteristic value information, and inquiring and retrieving the historical data of the concerned target;
(3.5) if the characteristic value information is successfully matched, adding corresponding information of the concerned target to realize detection tracking of multiple targets; otherwise, if the characteristic value matching fails, a new target tracker is created, meanwhile, unmatched concerned targets are deleted according to the limiting conditions of the target tracker, and the whole process of intelligent analysis and identification processing of the concerned targets is finished.
Preferably, the query search in the step (3.4) is a human information re-identification search, a non-vehicle information re-identification search or a vehicle information re-identification search.
The device for realizing intelligent analysis and identification processing of video image data based on traffic security target element extraction is mainly characterized by comprising the following components:
a processor configured to execute computer-executable instructions;
And a memory storing one or more computer-executable instructions which, when executed by the processor, perform the steps of the method for intelligent parsing recognition processing described above.
The processor for realizing intelligent analysis and identification processing of video image data based on extraction of traffic security target elements is mainly characterized in that the processor is configured to execute computer executable instructions, and the computer executable instructions are used for realizing the steps of the intelligent analysis and identification processing method when being executed by the processor.
The computer readable storage medium is characterized in that the computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method for intelligent analysis and identification processing.
The system, the method, the device, the processor and the computer readable storage medium for realizing intelligent analysis and identification processing aiming at video image data based on extraction of traffic security target elements are adopted, the collection of traffic security monitoring is taken as data support, the intelligent analysis and application capacity of a traffic security video monitoring system is improved through an intelligent analysis technology, the intelligent level of urban supervision is improved, data and service are provided for deep application of traffic security video, and the effect of the traffic security video image in public security, social management, urban supervision and other businesses is improved on the basis of the improvement.
Drawings
Fig. 1 is a schematic diagram of a functional module of a system for implementing intelligent analysis and recognition processing for video image data based on extraction of traffic security target elements.
Fig. 2 is a flow chart of a method for intelligent analysis and recognition processing of video image data based on traffic security target element extraction according to the present invention.
Fig. 3 is a schematic diagram of the overall framework and the working principle of the method for intelligent analysis and recognition processing of video image data based on the extraction of the traffic security target elements.
Fig. 4 is a schematic diagram of a fine classification attribute analysis flow for detecting and identifying an object of interest in the present invention.
Fig. 5 is a schematic diagram of a target violation determination flow of interest of a video image service application module according to the present invention.
Detailed Description
In order to more clearly describe the technical contents of the present invention, a further description will be made below in connection with specific embodiments.
Referring to fig. 1, the system for implementing intelligent analysis and identification processing for video image data based on extraction of traffic security target elements according to the present invention includes:
The traffic security video analysis module is used for acquiring video image data of a monitoring target from each video monitoring unit, analyzing video scenes, realizing monitoring and tracking of the target image and identification of road elements, and completing access processing of the video image data;
The video image supporting service module is connected with the traffic security video analysis module and is used for receiving the video image data, constructing a depth network model for the video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation on feature values on the data generated after the analysis processing, and outputting a target sequence similar to the target image in a target library;
The video image service application module is connected with the video image supporting service module and is used for receiving all attribute parameters of the monitored targets and performing behavior judgment and identification processing on the attribute parameters, and the monitoring, tracking and the distribution of the monitored targets are realized by detecting and tracking multiple targets, positioning the positions and track information of the monitored targets in the video image and drawing the running track of the monitored targets.
Referring to fig. 2, the method for implementing intelligent analysis and recognition processing for video image data based on traffic security target element extraction according to the present invention includes the following steps:
(1) Collecting video image data of the monitoring targets from each video monitoring unit, analyzing and judging video scenes, realizing monitoring and tracking of the concerned targets and identification of road elements, and completing access processing of the video image data;
(2) Constructing a depth network model from the acquired video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation of feature values on the data generated after the analysis processing, and outputting a target feature comparison sequence similar to the target image in a target library;
(3) And (3) performing behavior judgment and identification processing on the data generated in the step (2), and performing feature comparison on the target of interest by combining the target feature comparison sequence to realize monitoring tracking and control on the target of interest.
As a preferred embodiment of the present invention, the step (1) specifically includes the steps of:
The system of (1.1) collects the video image data from an electric police, a bayonet, a public security video, a vehicle-mounted device and an unmanned aerial vehicle video monitoring unit;
(1.2) calculating local brightness and average brightness of each frame of image from video frame information of each video monitoring unit, and analyzing and identifying a day and night mode of the target image by counting global brightness I G_avg and local brightness I Lb_avg of continuous frames;
(1.3) if the target image is detected to be in a daytime mode, entering a step (1.4); otherwise, if the target image is detected to be in a night mode, ending the whole process of intelligent analysis and identification processing on the video image data;
(1.4) carrying out multi-target tracking algorithm processing on the collected video image data, detecting a target image appearing in each frame of image, correlating the target image with a target detected in the previous frame of image, positioning the position of the target of interest appearing in the video image, and drawing a running track of the target, so as to realize multi-target detection tracking;
And (1.5) extracting lane line information and signal lamp information in the video image by combining brightness and saturation information of the video image data by using a traffic road element detection model, and taking the lane line information and the signal lamp information as different types of violation punishment reference bases.
As a preferred embodiment of the present invention, the step (2) specifically includes the following steps:
(2.1) constructing a depth network model for the target of interest detected and tracked by the video monitoring unit, detecting and identifying the target of interest, and extracting related information of the target of interest;
(2.2) performing coarse classification and fine classification attribute analysis on the detected and tracked target of interest by using a deep learning recognition model;
(2.3) extracting characteristic value information and characteristic attribute information of the target of interest after the classification analysis;
and (2.4) carrying out vector distance calculation of characteristic values on the extracted information data of the target of interest, and outputting a target sequence similar to the target of interest in the target library by taking the similarity score as an output result of characteristic value comparison.
As a preferred embodiment of the present invention, the attention target is a person, the relevant information of the person includes a person object picture, person object feature value information and person object feature attribute information, the person object feature value information is sex, age and clothing, the person object feature attribute information is a body feature and an accessory, the fine classification attribute of the person object is person attribute analysis information, the person attribute analysis information includes analysis information of illegal actions that the number of pedestrians on a non-motor vehicle exceeds a standard or the pedestrians do not wear a helmet, and the motor vehicle driver and co-driver do not tie a safety belt and make a call;
As a preferred embodiment of the present invention, the target of attention is a vehicle, the related information of the vehicle includes a vehicle object picture, vehicle object feature value information and vehicle object feature attribute information, the vehicle object feature value information is a license plate, a vehicle type, a vehicle brand, a vehicle color and a driving direction, the vehicle object feature attribute information is no-load, cargo or overload of the vehicle, the sub-classification attribute of the vehicle object is vehicle attribute analysis information, and the vehicle attribute analysis information includes overload, reverse running, red light running or lane change of the vehicle;
As a preferred embodiment of the present invention, the target of interest is a non-motor vehicle, the related information of the non-motor vehicle includes a non-motor vehicle object picture, non-motor vehicle object feature value information and non-motor vehicle object feature attribute information, the non-motor vehicle object feature value information is a non-motor vehicle type, color and driving direction, the non-motor vehicle object feature attribute information is the number, body shape, clothing, body appearance feature and attachment of the non-motor vehicle riders, the fine classification attribute of the non-motor vehicle object is non-motor vehicle attribute analysis information, and the non-motor vehicle attribute analysis information is non-motor vehicle lane change driving, the number of riders exceeds the standard or the riders do not wear a helmet.
As a preferred embodiment of the present invention, the step (3) specifically includes the following steps:
(3.1) identifying the illegal behaviors of the non-motor vehicles in lane changing running, exceeding the number of pedestrians and not wearing helmets by combining the non-motor vehicle information and the personnel information on the non-motor vehicles analyzed in the step (2.3) with the traffic road element detection model, storing the illegal information, pictures and videos of the illegal processes, and providing accurate and effective illegal punishment evidence for public security traffic;
(3.2) judging, identifying and processing the motor vehicle information and the personnel information on the motor vehicle analyzed in the step (2.3) in combination with the traffic road element detection model, and judging and processing the motor vehicle overload, retrograde, red light running, illegal lane changing, and the illegal behaviors that the drivers and copiers on the motor vehicle are not belted and make calls, storing the information, pictures and videos of the illegal motor vehicle process, and providing more accurate and effective illegal punishment evidence for public security traffic;
(3.4) comparing and sequencing the personnel attribute analysis information, the non-motor vehicle attribute analysis information and the motor vehicle attribute analysis information with the target sequence in characteristic value information, and inquiring and retrieving the historical data of the concerned target;
(3.5) if the characteristic value information is successfully matched, adding corresponding information of the concerned target to realize detection tracking of multiple targets; otherwise, if the characteristic value matching fails, a new target tracker is created, meanwhile, unmatched concerned targets are deleted according to the limiting conditions of the target tracker, and the whole process of intelligent analysis and identification processing of the concerned targets is finished.
As a preferred embodiment of the present invention, the query search in the step (3.4) is a human information re-recognition search, a non-vehicle information re-recognition search or a vehicle information re-recognition search.
The device for realizing intelligent analysis and identification processing of video image data based on traffic security target element extraction comprises:
a processor configured to execute computer-executable instructions;
And a memory storing one or more computer-executable instructions which, when executed by the processor, perform the steps of the method of intelligent parsing recognition processing described above.
The implementation of the invention is based on the processor for intelligent analysis and recognition processing of the traffic security target element extraction with respect to the video image data, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the intelligent analysis and recognition processing method are implemented.
The computer readable storage medium of the present invention has a computer program stored thereon, the computer program being executable by a processor to perform the steps of a method of intelligent parsing recognition processing.
As a preferred embodiment of the present invention, the specific processing procedure of the traffic police video analysis module of the system for performing intelligent analysis and identification processing on video image data based on the traffic police target element extraction is as follows:
In the step 1-1, the traffic security video data comprises video image data of a monitoring camera, an electric police and a bayonet. The data access of the intelligent analysis system for the traffic security video image element information is to collect the related data of the source access video image from the traffic security video such as electric police, bayonet, security video, vehicle-mounted equipment, unmanned aerial vehicle and the like according to the service requirement, and complete the access processing of the video image data.
Then, in the step 1-2, analyzing whether the video scene is daytime or night;
From the video frames shown, the local and average luminance of each frame is calculated, and by counting the global luminance i_ (g_avg) and the local luminance i_ (lb_avg) of consecutive frames, i_ (lt_avg) are used as important indicators for distinguishing video circadian from each other:
By judging the day and night mode of the video, the intelligent analysis of the video is finished when the video is a night scene, and the target false recognition caused by low target definition in the night video can be reduced. And when the video is in the daytime mode, entering a step 1-3 to detect and track the target of interest, and if the video is in the nighttime mode, ending the intelligent analysis process.
Then, entering into step 1-3, detecting and tracking a target of interest in the video scene;
And detecting a target of interest appearing in each frame of the video from the accessed video data by utilizing a multi-target detection tracking algorithm, and associating the target of interest with the target detected in the previous frame to realize multi-target detection tracking. And positioning the position of the concerned object appearing in the video and the track information thereof through multi-object detection tracking.
In the step 1-4, detecting and identifying traffic road elements in the video scene;
And extracting lane lines (such as solid lines, broken lines, yellow lines and white lines) and signal lamps (such as signal lamp shapes: arrows and circles, and signal lamp colors: yellow, red and green) in the video from the accessed video data by utilizing a traffic road element detection model and combining brightness and saturation information, and taking the lane lines (such as signal lamp shapes: arrows and circles, and signal lamp colors: yellow, red and green) as reference bases for different types of violation punishments.
The specific processing procedure of the video image supporting service module is as follows:
In the step 2-1, the detection and tracking of the traffic security video data are carried out through the step 1-3 to obtain attention targets such as people, vehicles and non-motor vehicles, a depth network model is constructed, the attention targets are detected and identified, and information such as pictures, feature values and feature attributes of the objects such as people, vehicles and non-motor vehicles is extracted.
In the step 2-2, the detection and recognition of the traffic security video attention target are carried out through the step 2-1, the coarse classification of personnel, vehicles and non-motor vehicles is carried out on the tracked target by utilizing a deep learning recognition model, and the fine classification attribute analysis of the attention target is carried out, wherein the fine classification attribute analysis comprises personnel attribute analysis, non-motor vehicle attribute analysis and vehicle attribute analysis.
In the step 2-3, the characteristic value of the vehicle function and the related characteristic attribute information, such as a muck truck, a dangerous chemical truck type, an engineering truck, a concrete mixer truck, a garbage truck and the like, are extracted through the recognition of the vehicle function after the analysis of the vehicle target in the step 2-2.
In step 2-4, the analysis of the cargo carrying of the truck is performed on the truck target image after the analysis of the truck target in step 2-2, and the characteristic value of the cargo carrying of the truck and the related characteristic attribute information, such as empty load, cargo carrying, overload and the like, of the truck are extracted.
In step 2-5, feature vectors are extracted from the targets appearing in the video in step 2-2, distance calculation is performed, and similarity scores are used as output of feature comparison. And carrying out distance calculation of characteristic values on the images in the target library on the specific target images, and outputting a target sequence similar to the target images in the target library by taking the similarity score as an index.
The specific processing procedure of the video image service application module is as follows:
and 3-1, identifying the illegal behaviors that the non-motor vehicle does not run in the specified road by combining the analysis information of the non-motor vehicle in the step 2-2 and the traffic road element information detected in the step 1-4, such as exceeding the number of riders, not wearing helmets by the riders and the like, storing the illegal information, pictures and videos of the illegal processes, and providing more accurate and effective illegal punishment evidence for public security traffic.
And 3-2, identifying the violations of the vehicles, such as lane changing, red light running and the like, by combining the analysis information of the vehicles in the step 2-2 with the traffic road element information detected in the step 1-4. The method is used for judging the unlawful behaviors of a vehicle driver and a copilot, such as unbuckling of a safety belt, calling and the like, storing information and pictures of the unlawful vehicles and videos of the unlawful processes, and providing more accurate and effective unlawful punishment evidence for public security traffic.
And 3-3, managing illegal actions such as overload, retrograde driving, red light running, driving without a specified lane and the like of engineering vehicles such as a muck truck, a skip car, a concrete mixer truck and the like by combining the vehicle function attribute in the step 2-3/2-4 and the urban road traffic control requirement, and simultaneously counting the access time and the quantity of the engineering vehicles in a certain area, wherein the information provides decision basis for government supervision.
And 3-4, combining the analysis information of the personnel, the non-motor vehicles and the vehicles in the step 2-2 with the target feature vector comparison and sequencing in the step 2-5, searching the historical data of the target of interest, and drawing the historical track of the target of interest, wherein the historical track comprises personnel searching, non-motor vehicle searching and vehicle searching.
And 3-5, performing control on the target of interest by combining the analysis information of the personnel, the non-motor vehicles and the vehicles in the step 2-2 and the target feature vector comparison sequencing in the step 2-5, and performing analysis and comparison on the targets appearing in the real-time video to realize the control on the target of interest (such as the personnel and the vehicles).
Referring to fig. 3, as an embodiment of the present invention, step s1 is a traffic security video analysis module, and the processing steps are as follows:
In step 101, the traffic security video data includes video image data of a monitoring camera, an electric police, and a gate. The data of the intelligent analysis system for the information of the elements of the traffic security video image is that the related data of the video image is accessed from the traffic security video such as electric police, bayonet, security video, vehicle-mounted equipment, unmanned aerial vehicle and the like through GB28181 according to the service requirement, and the access processing of the video image data is completed.
Thereafter, in step 102, the video scene is analyzed as day or night;
From the video frames shown, the local and average luminance of each frame is calculated, and by counting the global luminance i_ (g_avg) and the local luminance i_ (lb_avg) of consecutive frames, i_ (lt_avg) are used as important indicators for distinguishing video circadian from each other:
If i_ (g_avg) > i_ (day_th), i_ (lt_avg) > i_ (lb_avg), then the video scene is considered to be daytime video, labeled p_ (day_right) =1;
If i_ (g_avg) < i_ (right_th), i_ (lt_avg) < i_ (lb_avg), then the video scene is considered to be a night video, labeled p_ (day_right) =0;
Where I_ (day_th) represents the lower limit of the daytime luminance feature, I_ (right_th) represents the upper limit of the nighttime luminance feature, I_ (lt_avg) represents the average luminance of the upper portion of the height_img/2 in the single frame image, I_ (lb_avg) represents the average luminance of the lower portion of the height_img/2 in the single frame image, and P_ (day_right) represents the circadian pattern of the video.
By judging the day and night mode of the video, the intelligent analysis of the video is finished when the video is a night scene, and the target false recognition caused by low target definition in the night video can be reduced. If the video is in the daytime mode, the method enters step 103 to pay attention to detection and tracking of the target, and if the video is in the nighttime mode, the intelligent analysis process is ended.
Thereafter, in step 103, detecting and tracking the target of interest in the video scene;
detecting and extracting the position and characteristic information of a target of interest in each frame of the video from the accessed video data by utilizing a deep learning detection and segmentation model such as ssd, yolov and the like, matching the extracted target with the existing target tracker by utilizing a multi-target tracking algorithm, and adding the target information if the characteristic matching is successful; if the matching fails, a new target tracker is created; meanwhile, unmatched targets are deleted according to limiting conditions of the target tracker, and detection and tracking of multiple targets are circularly achieved. And positioning the position of the concerned object appearing in the video and the track information thereof through multi-object detection tracking.
In step 104, detecting and identifying traffic road elements in the video scene;
And extracting traffic element information such as lane lines (such as solid lines, broken lines, yellow lines and white lines) and signal lamps (such as signal lamp shapes: arrows and circles, and signal lamp colors: yellow, red and green) in the video by using a deep learning detection model and combining the brightness and saturation information of the video from the accessed video data, and taking the traffic element information as a reference basis for different types of violation punishments.
Step s2 is a video image support service module, further comprising the sub-steps of:
In step 201, a depth network model is constructed for the targets of interest such as personnel, vehicles and non-motor vehicles obtained by detecting and tracking the traffic security video data in step 103, and the targets of interest are subjected to structural attribute identification, and information such as pictures, characteristic values and structural attributes of the targets such as personnel, vehicles and non-motor vehicles is extracted.
Referring to fig. 4, in step 202, the detection and identification of the traffic police video attention target is performed through step 201, the tracked target is classified, and the classified target is subjected to fine classification attribute analysis, including personnel attribute analysis, non-motor vehicle attribute analysis and vehicle attribute analysis.
And analyzing the detected and tracked personnel target image in a human body-oriented way, and extracting human body characteristic values and related characteristic attribute information such as gender, age, clothes, body appearance characteristics, accessories and the like. The method is mainly used for text retrieval according to personnel attributes, and locating personnel images similar to description target information, such as male, middle-aged and black long-sleeve jackets and blue trousers.
Analyzing the detected and tracked target image of the non-motor vehicle facing the non-motor vehicle, and extracting characteristic values and related characteristic attribute information of the non-motor vehicle and personnel on the non-motor vehicle, such as the type, the color and the like of the non-motor vehicle, and riding personnel attributes: the number of riders, body type, clothing, body appearance characteristics, accessories, etc.
Analyzing the detected and tracked vehicle target image in a vehicle-oriented mode, extracting characteristic values of vehicles, vehicle driving and copilot personnel and related characteristic attribute information such as license plates, vehicle types, vehicle brands, vehicle colors, driving directions and the like, and attributes of drivers and copilot: gender, age, appurtenance, behavior, etc.
In step 203, through the recognition of the vehicle function after the analysis of the vehicle target in step 202, the characteristic value of the vehicle function and the related characteristic attribute information, such as the muck truck, the dangerous chemical truck type, the engineering truck, the concrete mixer truck, the garbage truck and the like, are extracted.
In step 204, the analysis for the cargo of the truck is performed on the truck target image after the analysis of the truck target in step 202, so as to extract the characteristic value of the cargo of the truck and the related characteristic attribute information, such as empty load, cargo load, overload, etc. of the truck.
In step 205, a technique of determining whether a specific target exists in the video or the image by using a deep learning technique, training a target re-recognition model, performing distance calculation on a target extraction feature vector appearing in the video in step 202, and using a similarity score as an output of feature comparison. And carrying out distance calculation of characteristic values on the images in the target library on the specific target images, and outputting a target sequence similar to the target images in the target library by taking the similarity score as an index.
Meanwhile, step s3 is a video image service application module, and further includes the following sub-steps:
Step 301, by combining the analysis information of the non-motor vehicles in step 202 with the traffic road element information detected in step 104, the non-motor vehicles are not driven in the specified road, the number of riders exceeds the standard, the riders do not wear helmets and other illegal behaviors are identified, the illegal information, pictures and videos of the illegal processes are stored, and more accurate and effective illegal punishment evidence is provided for public security traffic.
Step 302, by combining the analysis information of the vehicle in step 202 with the traffic road element information detected in step 104, the violations of the vehicle, such as lane changing, red light running, etc., are identified. The method is used for judging the unlawful behaviors of a vehicle driver and a copilot, such as unbuckling of a safety belt, calling and the like, storing information and pictures of the unlawful vehicles and videos of the unlawful processes, and providing more accurate and effective unlawful punishment evidence for public security traffic.
Step 303, by combining the vehicle function attribute in step 203/204 with the urban road traffic control requirement, the illegal actions such as overload, retrograde, red light running, no-specified lane running and the like of engineering vehicles such as a muck truck, a stone truck, a concrete mixer truck and the like are managed, meanwhile, the access time and the number of engineering vehicles in a certain area are counted, and the information provides decision basis for government supervision.
Step 304, through the analysis information of the personnel, the non-motor vehicles and the vehicles in step 202, the query search of the historical data of the target of interest is conducted by combining the comparison and sequencing of the target feature vectors in step 205, and the historical track of the target of interest is drawn, wherein the historical track comprises personnel re-identification search, non-motor vehicle re-identification search and vehicle re-identification search.
Step 305, by combining the structural description information of the personnel, the non-motor vehicles and the vehicles in step 202 with the target feature vector comparison and sorting in step 205, the specific targets are controlled, and by analyzing and comparing the targets appearing in the real-time video, the tracking and controlling of the specific targets (such as the personnel and the vehicles) under the cross-camera are realized.
The system, the method, the device, the processor and the computer readable storage medium for realizing intelligent analysis and identification processing for video image data based on extraction of traffic security target elements, wherein the verification process of the attention target comprises the following steps: analyzing the traffic security video, and positioning and tracking the moving target in the traffic security video according to the scene; constructing a multi-class deep learning network model through depth features of targets such as pedestrians, non-motor vehicles and vehicles in the video image, and carrying out structural feature description on moving targets in the road junction traffic by utilizing the pre-trained model; aiming at a target vehicle, according to the running direction of the target vehicle, the detected lane line information and red light signal information are utilized, and the traffic rule of the road is combined to judge the lane change, red light running and other violations of the target vehicle on the road, aiming at a target vehicle driver, according to structural description information after personnel analysis, the violations of unbelted safety belts, calling and the like are judged, and the information, pictures and videos of the violating processes of the violating vehicle are stored, so that more accurate and effective violating punishment evidence is provided for public security traffic; the vehicle target structural analysis information is utilized, and special vehicles such as a muck vehicle, a hazardous chemical vehicle and an engineering vehicle are controlled by combining data service and a visual function, so that a decision basis is provided for government supervision; judging illegal behaviors such as riding belters, reverse running and the like of the nonstandard vehicles by utilizing the nonstandard vehicle structural analysis information; and simultaneously storing structural analysis information and depth characteristics of pedestrians, nonstandard vehicles and vehicle targets on the road, and carrying out quick retrieval of high-order characteristics so as to carry out target control, historical information searching and post analysis in a system platform. The invention has the advantages of improving the functions of the traffic security video in public security, social management, urban supervision and other businesses, taking the collection of the traffic security video as data support, enhancing intelligent recognition and law enforcement capability through intelligent analysis technology, improving the intelligent level of urban supervision, improving the intelligent analysis and application capability of the traffic security video monitoring system, and providing data and service for the deep application of the traffic security video.
The specific implementation manner of this embodiment may be referred to the related description in the foregoing embodiment, which is not repeated herein.
It is to be understood that the same or similar parts in the above embodiments may be referred to each other, and that in some embodiments, the same or similar parts in other embodiments may be referred to.
It should be noted that in the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present invention, unless otherwise indicated, the meaning of "plurality" means at least two.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, where the program when executed includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented as software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The system, the method, the device, the processor and the computer readable storage medium for realizing intelligent analysis and identification processing aiming at video image data based on extraction of traffic security target elements are adopted, the collection of traffic security monitoring is taken as data support, the intelligent analysis and application capacity of a traffic security video monitoring system is improved through an intelligent analysis technology, the intelligent level of urban supervision is improved, data and service are provided for deep application of traffic security video, and the effect of the traffic security video image in public security, social management, urban supervision and other businesses is improved on the basis of the improvement.
In this specification, the invention has been described with reference to specific embodiments thereof. It will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Claims (8)
1. The system for realizing intelligent analysis and identification processing of video image data based on extraction of traffic security target elements is characterized by comprising the following components:
The traffic security video analysis module is used for acquiring video image data of a monitoring target from each video monitoring unit, analyzing video scenes, realizing monitoring and tracking of the target image and identification of road elements, and completing access processing of the video image data;
The video image supporting service module is connected with the traffic security video analysis module and is used for receiving the video image data, constructing a depth network model for the video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation on feature values on the data generated after the analysis processing, and outputting a target sequence similar to the target image in a target library;
The video image service application module is connected with the video image supporting service module and is used for receiving all attribute parameters of the monitoring target and performing behavior judgment and identification processing on the attribute parameters, positioning the position and track information of the concerned target in the video image and drawing the running track of the concerned target through detecting and tracking the multiple targets, so as to realize the monitoring, tracking and distribution control of the concerned target;
The intelligent analysis and recognition processing for the video image data based on the extraction of the traffic security target elements is realized through the system, and the method specifically comprises the following steps:
(1) Collecting video image data of the monitoring targets from each video monitoring unit, analyzing and judging video scenes, realizing monitoring and tracking of the concerned targets and identification of road elements, and completing access processing of the video image data;
(2) Constructing a depth network model from the acquired video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation of feature values on the data generated after the analysis processing, and outputting a target feature comparison sequence similar to the target image in a target library;
(3) Performing behavior judgment and identification processing on the data generated in the step (2), and performing feature comparison on the target of interest by combining the target feature comparison sequence to realize monitoring tracking and control on the target of interest;
the step (2) specifically comprises the following steps:
(2.1) constructing a depth network model for the target of interest detected and tracked by the video monitoring unit, detecting and identifying the target of interest, and extracting related information of the target of interest;
(2.2) performing coarse classification and fine classification attribute analysis on the detected and tracked target of interest by using a deep learning recognition model;
(2.3) extracting characteristic value information and characteristic attribute information of the target of interest after the classification analysis;
(2.4) carrying out vector distance calculation of characteristic values on the extracted information data of the concerned targets, and taking similarity scores as output results of characteristic value comparison, and outputting target sequences similar to the concerned targets in the target library;
the step (1) specifically comprises the following steps:
The system of (1.1) collects the video image data from an electric police, a bayonet, a public security video, a vehicle-mounted device and an unmanned aerial vehicle video monitoring unit;
(1.2) calculating local brightness and average brightness of each frame of image from video frame information of each video monitoring unit, and analyzing and identifying a day and night mode of the target image by counting global brightness I G_avg and local brightness I Lb_avg of continuous frames;
(1.3) if the target image is detected to be in a daytime mode, entering a step (1.4); otherwise, if the target image is detected to be in a night mode, ending the whole process of intelligent analysis and identification processing on the video image data;
(1.4) carrying out multi-target tracking algorithm processing on the collected video image data, detecting a target image appearing in each frame of image, correlating the target image with a target detected in the previous frame of image, positioning the position of the target of interest appearing in the video image, and drawing a running track of the target, so as to realize multi-target detection tracking;
And (1.5) extracting lane line information and signal lamp information in the video image by combining brightness and saturation information of the video image data by using a traffic road element detection model, and taking the lane line information and the signal lamp information as different types of violation punishment reference bases.
2. A method for implementing intelligent analysis and recognition processing for video image data based on traffic security target element extraction by using a system for implementing intelligent analysis and recognition processing for video image data based on traffic security target element extraction, the system comprising:
The traffic security video analysis module is used for acquiring video image data of a monitoring target from each video monitoring unit, analyzing video scenes, realizing monitoring and tracking of the target image and identification of road elements, and completing access processing of the video image data;
The video image supporting service module is connected with the traffic security video analysis module and is used for receiving the video image data, constructing a depth network model for the video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation on feature values on the data generated after the analysis processing, and outputting a target sequence similar to the target image in a target library;
The video image service application module is connected with the video image supporting service module and is used for receiving all attribute parameters of the monitoring target and performing behavior judgment and identification processing on the attribute parameters, positioning the position and track information of the concerned target in the video image and drawing the running track of the concerned target through detecting and tracking the multiple targets, so as to realize the monitoring, tracking and distribution control of the concerned target;
the method is characterized by comprising the following steps:
(1) Collecting video image data of the monitoring targets from each video monitoring unit, analyzing and judging video scenes, realizing monitoring and tracking of the concerned targets and identification of road elements, and completing access processing of the video image data;
(2) Constructing a depth network model from the acquired video image data, carrying out attribute classification, analysis processing and feature extraction on the video image data, carrying out vector distance calculation of feature values on the data generated after the analysis processing, and outputting a target feature comparison sequence similar to the target image in a target library;
(3) Performing behavior judgment and identification processing on the data generated in the step (2), and performing feature comparison on the target of interest by combining the target feature comparison sequence to realize monitoring tracking and control on the target of interest;
the step (2) specifically comprises the following steps:
(2.1) constructing a depth network model for the target of interest detected and tracked by the video monitoring unit, detecting and identifying the target of interest, and extracting related information of the target of interest;
(2.2) performing coarse classification and fine classification attribute analysis on the detected and tracked target of interest by using a deep learning recognition model;
(2.3) extracting characteristic value information and characteristic attribute information of the target of interest after the classification analysis;
(2.4) carrying out vector distance calculation of characteristic values on the extracted information data of the concerned targets, and taking similarity scores as output results of characteristic value comparison, and outputting target sequences similar to the concerned targets in the target library;
the step (1) specifically comprises the following steps:
The system of (1.1) collects the video image data from an electric police, a bayonet, a public security video, a vehicle-mounted device and an unmanned aerial vehicle video monitoring unit;
(1.2) calculating local brightness and average brightness of each frame of image from video frame information of each video monitoring unit, and analyzing and identifying a day and night mode of the target image by counting global brightness I G_avg and local brightness I Lb_avg of continuous frames;
(1.3) if the target image is detected to be in a daytime mode, entering a step (1.4); otherwise, if the target image is detected to be in a night mode, ending the whole process of intelligent analysis and identification processing on the video image data;
(1.4) carrying out multi-target tracking algorithm processing on the collected video image data, detecting a target image appearing in each frame of image, correlating the target image with a target detected in the previous frame of image, positioning the position of the target of interest appearing in the video image, and drawing a running track of the target, so as to realize multi-target detection tracking;
And (1.5) extracting lane line information and signal lamp information in the video image by combining brightness and saturation information of the video image data by using a traffic road element detection model, and taking the lane line information and the signal lamp information as different types of violation punishment reference bases.
3. The method for realizing intelligent analysis and recognition processing for video image data based on traffic security target element extraction according to claim 2, wherein,
The concerned targets are personnel, the related information of the personnel comprises personnel object pictures, personnel object characteristic value information and personnel object characteristic attribute information, the personnel object characteristic value information is sex, age and clothing, the personnel object characteristic attribute information is body appearance characteristics and accessories, the fine classification attribute of the personnel object is personnel attribute analysis information, and the personnel attribute analysis information comprises non-motor vehicle riding number exceeding or riding without helmet, and motor vehicle drivers and copiers are not tied with safety belts and illegal behavior analysis information of making calls; or alternatively
The concerned targets are motor vehicles, the related information of the motor vehicles comprises motor vehicle object pictures, motor vehicle object characteristic value information and motor vehicle object characteristic attribute information, the motor vehicle object characteristic value information is license plates, vehicle types, vehicle brands, vehicle colors and running directions, the motor vehicle object characteristic attribute information is no-load, cargo or overload of the motor vehicles, the fine classification attribute of the motor vehicle objects is motor vehicle attribute analysis information, and the motor vehicle attribute analysis information comprises motor vehicle overload, retrograde, red light running or illegal lane changing; or alternatively
The non-motor vehicle object feature value information is the type, the color and the running direction of the non-motor vehicle, the non-motor vehicle object feature information is the number, the body shape, the clothes, the body appearance features and the accessories of the non-motor vehicle riders, the fine classification attribute of the non-motor vehicle object is non-motor vehicle attribute analysis information, and the non-motor vehicle attribute analysis information is non-motor vehicle lane change running, the number of riders exceeds the standard or the riders do not wear helmets.
4. The method for implementing intelligent analysis and recognition based on traffic security target element extraction according to claim 3, wherein the step (3) specifically comprises the following steps:
(3.1) identifying the illegal behaviors of the non-motor vehicles in lane changing running, exceeding the number of pedestrians and not wearing helmets by combining the non-motor vehicle information and the personnel information on the non-motor vehicles analyzed in the step (2.3) with the traffic road element detection model, storing the illegal information, pictures and videos of the illegal processes, and providing accurate and effective illegal punishment evidence for public security traffic;
(3.2) judging, identifying and processing the motor vehicle information and the personnel information on the motor vehicle analyzed in the step (2.3) in combination with the traffic road element detection model, and judging and processing the motor vehicle overload, retrograde, red light running, illegal lane changing, and the illegal behaviors that the drivers and copiers on the motor vehicle are not belted and make calls, storing the information, pictures and videos of the illegal motor vehicle process, and providing more accurate and effective illegal punishment evidence for public security traffic;
(3.4) comparing and sequencing the personnel attribute analysis information, the non-motor vehicle attribute analysis information and the motor vehicle attribute analysis information with the target sequence in characteristic value information, and inquiring and retrieving the historical data of the concerned target;
(3.5) if the characteristic value information is successfully matched, adding corresponding information of the concerned target to realize detection tracking of multiple targets; otherwise, if the characteristic value matching fails, a new target tracker is created, meanwhile, unmatched concerned targets are deleted according to the limiting conditions of the target tracker, and the whole process of intelligent analysis and identification processing of the concerned targets is finished.
5. The method for implementing intelligent analysis and recognition based on traffic security target element extraction according to claim 4, wherein the query search in the step (3.4) is a human information re-recognition search, a non-vehicle information re-recognition search or a vehicle information re-recognition search.
6. An apparatus for implementing intelligent analysis and recognition processing for video image data based on traffic security target element extraction, characterized in that the apparatus comprises:
a processor configured to execute computer-executable instructions;
A memory storing one or more computer-executable instructions which, when executed by the processor, perform the steps of the method of any one of claims 2 to 5 for implementing intelligent parsing recognition processing for video image data based on traffic police target element extraction.
7. A processor for implementing intelligent parsing and identifying processing for video image data based on traffic police target element extraction, characterized in that the processor is configured to execute computer executable instructions, which when executed by the processor, implement the steps of the method for implementing intelligent parsing and identifying processing for video image data based on traffic police target element extraction according to any one of claims 2 to 5.
8. A computer-readable storage medium having stored thereon a computer program executable by a processor to perform the steps of the method of any one of claims 2 to 5 for implementing intelligent parsing recognition processing for video image data based on traffic control objective element extraction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011313065.3A CN112381859B (en) | 2020-11-20 | 2020-11-20 | System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011313065.3A CN112381859B (en) | 2020-11-20 | 2020-11-20 | System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112381859A CN112381859A (en) | 2021-02-19 |
| CN112381859B true CN112381859B (en) | 2024-06-04 |
Family
ID=74584565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011313065.3A Active CN112381859B (en) | 2020-11-20 | 2020-11-20 | System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112381859B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113160575A (en) * | 2021-03-15 | 2021-07-23 | 超级视线科技有限公司 | Traffic violation detection method and system for non-motor vehicles and drivers |
| CN113422938A (en) * | 2021-08-23 | 2021-09-21 | 深圳市旗扬特种装备技术工程有限公司 | Artificial intelligence road event monitoring method, device, system and storage medium |
| CN114333296A (en) * | 2021-11-30 | 2022-04-12 | 西南石油大学 | Traffic volume statistics and analysis system based on machine vision |
| CN114882597B (en) * | 2022-07-11 | 2022-10-28 | 浙江大华技术股份有限公司 | Target behavior identification method and device and electronic equipment |
| CN115861905B (en) * | 2023-03-01 | 2023-06-16 | 鹿马智能科技（上海）有限公司 | Hotel management system based on internet of things |
| CN117671572A (en) * | 2024-02-02 | 2024-03-08 | 深邦智能科技集团(青岛)有限公司 | Multi-platform linkage road image model processing system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015089867A1 (en) * | 2013-12-17 | 2015-06-25 | 东莞中国科学院云计算产业技术创新与育成中心 | Traffic violation detection method |
| CN107832672A (en) * | 2017-10-12 | 2018-03-23 | 北京航空航天大学 | A kind of pedestrian's recognition methods again that more loss functions are designed using attitude information |
| CN109858459A (en) * | 2019-02-20 | 2019-06-07 | 公安部第三研究所 | System and method based on police vehicle-mounted video element information realization intelligently parsing processing |
-
2020
- 2020-11-20 CN CN202011313065.3A patent/CN112381859B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015089867A1 (en) * | 2013-12-17 | 2015-06-25 | 东莞中国科学院云计算产业技术创新与育成中心 | Traffic violation detection method |
| CN107832672A (en) * | 2017-10-12 | 2018-03-23 | 北京航空航天大学 | A kind of pedestrian's recognition methods again that more loss functions are designed using attitude information |
| CN109858459A (en) * | 2019-02-20 | 2019-06-07 | 公安部第三研究所 | System and method based on police vehicle-mounted video element information realization intelligently parsing processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112381859A (en) | 2021-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112381859B (en) | System, method, device, processor and storage medium for implementing intelligent analysis and identification processing for video image data | |
| Tian et al. | An automatic car accident detection method based on cooperative vehicle infrastructure systems | |
| CN105702048B (en) | Highway front truck illegal road occupation identifying system based on automobile data recorder and method | |
| CN111800507A (en) | Traffic monitoring method and traffic monitoring system | |
| CN101739809A (en) | Automatic alarm and monitoring system for pedestrian running red light | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| Razalli et al. | Emergency vehicle recognition and classification method using HSV color segmentation | |
| CN113012436B (en) | Road monitoring method and device and electronic equipment | |
| Ketcham et al. | Recognizing the Illegal Parking Patterns of Cars on the Road in Front of the Bus Stop Using the Support Vector Machine | |
| CN112651293A (en) | Video detection method for road illegal stall setting event | |
| CN111292530A (en) | Method, device, server and storage medium for processing violation pictures | |
| CN114724122B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN108427718A (en) | A kind of vehicle traveling information temporal index and big data analysis method | |
| Ravish et al. | Intelligent traffic violation detection | |
| Zhou et al. | Monitoring-based traffic participant detection in urban mixed traffic: A novel dataset and a tailored detector | |
| Medina et al. | Automotive embedded image classification systems | |
| CN115019263A (en) | Traffic supervision model establishing method, traffic supervision system and traffic supervision method | |
| Ojala et al. | Motion detection and classification: ultra-fast road user detection | |
| CN112597924B (en) | Electric bicycle track tracking method, camera device and server | |
| CN114283361A (en) | Method and apparatus for determining status information, storage medium, and electronic apparatus | |
| Avupati et al. | Traffic Rules Violation Detection using YOLO and HAAR Cascade | |
| CN113515665A (en) | Video processing and information query method, device, system and storage medium | |
| CN113850112A (en) | Road condition identification method and system based on twin neural network | |
| Aminian et al. | Cost-efficient traffic sign detection relying on smart mobile devices | |
| Peng et al. | Traffic violation detection via depth and gradient angle change |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |