CN111931650A

CN111931650A - Target detection model construction and red light running responsibility tracing method, system, terminal and medium

Info

Publication number: CN111931650A
Application number: CN202010798307.6A
Authority: CN
Inventors: 曹学军
Original assignee: Hangzhou Zhixing Technology Co ltd
Current assignee: Hangzhou Zhixing Technology Co ltd
Priority date: 2020-08-11
Filing date: 2020-08-11
Publication date: 2020-11-13

Abstract

The invention provides a method, a system, a terminal and a medium for constructing a target detection model and tracing responsibility for red light running.

Description

Target detection model construction and red light running responsibility tracing method, system, terminal and medium

Technical Field

The invention relates to the technical field of artificial intelligence and deep learning, in particular to a method, a system, a terminal and a medium for constructing a target detection model and pursuing responsibility for running a red light.

Background

As urban public transport becomes saturated, more and more travelers choose non-motor vehicles as vehicles for medium and short distance travel, and therefore, the problem is that the traffic management department cannot effectively monitor and manage the non-motor vehicles.

Research shows that 20% of death accidents of the electric vehicle are caused by red light running, non-motor vehicle drivers are relatively low in safety consciousness due to lack of training in road traffic safety, illegal red light running behaviors easily exist in the road running process, more traffic accidents are caused, and great potential safety hazards are brought. For such illegal behaviors, the traffic management department mostly adopts the way crossing to set a checkpoint to grab the red light running agent, and to carry out road safety education and illegal punishment on the red light running agent. However, the inspection mode consumes a great amount of manpower, and people who break away have lucky psychology, and no measures for restricting the safe driving behavior of the non-motor vehicle driver are available in the unsupervised environment, so that the inspection mode of checking the red light running of the non-motor vehicle under the traffic control gate line is temporary and permanent. Therefore, a system method capable of performing closed-loop tracing on the behavior of the non-motor vehicle running the red light needs to be provided.

Disclosure of Invention

In view of the above defects in the prior art, the invention provides a method, a system, a terminal and a medium for constructing a target detection model and tracing when a vehicle runs a red light, which are used for solving the problems of low efficiency, high cost, lack of effectiveness and the like when a non-motor vehicle runs a red light in the prior art.

In order to achieve the above object, a first aspect of the present invention provides a method for constructing a deep learning-based target detection model, including: acquiring a traffic video stream in a designated area; marking the traffic video stream to obtain a vehicle target frame and a red light display area target frame, and forming first marking data; performing secondary marking on the vehicle target frame to obtain a license plate target frame and forming second marking data; inputting the first marking data into a first deep learning target detection model for training, testing the first deep learning target detection model after the training is finished, and determining a vehicle and a red light detection model for identifying the target vehicle and the red light state according to the test result; inputting the second labeling data into a second deep learning target detection model for training, testing the second deep learning target detection model after the training is finished, and determining a license plate detection model for recognizing a license plate according to a test result.

In a preferred embodiment of the present invention, the method further comprises: carrying out target frame clustering on the vehicle target frame and the red light display area target frame to obtain a target frame reference size which is close to that used in an actual scene; and performing error square sum judgment on the clustered vehicle target frames and the red light display area target frames.

In another preferred embodiment of the present invention, the first labeling data includes center point coordinate data, width and height data of the target frame of the red light display area, and further includes center point coordinate data, width and height data of the target frame of the vehicle; the second labeling data comprise coordinate data of the center point of the license plate target frame, width data and height data.

In order to achieve the above object, a second aspect of the present invention provides a deep learning-based red light running accountability method, including: acquiring a current traffic video stream, and determining coordinates of a red light target frame of an actual scene; inputting the current traffic video stream into a vehicle and red light detection model obtained by training a deep learning target detection model, and acquiring vehicle target frame coordinates and red light display area target frame coordinates output by the vehicle and red light detection model; calculating the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model, and judging the red light state of the current video stream according to the overlapping degree calculation result; inputting a vehicle target frame corresponding to the current traffic video stream judged as the red light into a license plate detection model obtained by training a deep learning target detection model, and acquiring the coordinates of the license plate target frame output by the license plate detection model; performing character recognition on the license plate according to the license plate target frame coordinates output by the license plate detection model to obtain a license plate number; and synchronously outputting the license plate number and the vehicle image for tracing the responsibility of running the red light.

In a preferred embodiment of the present invention, the determining the red light status of the current video stream according to the overlapping degree calculation result includes: judging whether the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model exceeds an overlapping degree threshold value or not; if the overlapping degree threshold value is exceeded, judging that the traffic indicator light in the current traffic video stream is in a red light state; otherwise, judging that the traffic indicator light in the current traffic video stream is not in the red light state.

In a preferred embodiment of the present invention, the overlap threshold is determined by testing the red light detection accuracy of the same traffic video stream at different overlap thresholds respectively.

In a preferred embodiment of the present invention, the method further comprises: and adding a timestamp to the license plate number and the vehicle image, and uploading the license plate number and the vehicle image to a pre-specified violation database.

In order to achieve the above object, a third aspect of the present invention provides a deep learning-based red light running accountability system, including: the image acquisition module is used for acquiring a traffic video stream in a specified area; the marking module is used for marking the traffic video stream to obtain a vehicle target frame and a red light display area target frame and forming first marking data; the license plate target frame is obtained after the vehicle target frame is subjected to secondary marking, and second marking data are formed; the vehicle and red light detection module is used for inputting the first marking data into a first deep learning target detection model for training, testing the first deep learning target detection model after the training is finished, and determining a vehicle and a red light detection model for identifying the states of the target vehicle and the red light according to a test result; the license plate recognition module is used for inputting the second labeled data into a second deep learning target detection model for training, testing the second deep learning target detection model after the training is finished, and determining a license plate detection model for recognizing a license plate according to a test result; the red light running judgment module is used for acquiring the current traffic video stream and determining the coordinates of a red light target frame of an actual scene; inputting the current traffic video stream into the vehicle and red light detection model, and acquiring coordinates of a vehicle target frame and coordinates of a red light display area target frame; calculating the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model, and judging the red light state of the current video stream according to the overlapping degree calculation result; inputting a vehicle target frame corresponding to the current traffic video stream judged as the red light into the license plate detection model, and acquiring the coordinates of the license plate target frame output by the license plate detection model; performing character recognition on the license plate according to the license plate target frame coordinates output by the license plate detection model to obtain a license plate number; and synchronously outputting the license plate number and the vehicle image for tracing the responsibility of running the red light.

In order to achieve the above object, a fourth aspect of the present invention provides a deep learning-based target detection model building terminal, including: a first storage unit for storing at least one computer program; the first processing unit is used for running the at least one computer program to execute the deep learning-based target detection model construction method.

In order to achieve the above object, a fifth aspect of the present invention provides a deep learning-based red light running accountability terminal, including: a second storage unit for storing at least one computer program; and the second processing unit is used for running the at least one computer program so as to execute the deep learning-based red light running tracing method.

To achieve the above object, a sixth aspect of the present invention provides a computer-readable storage medium storing at least one computer program, which when executed, executes the deep learning-based object detection model construction method; or, executing the deep learning-based red light running tracing method.

The invention provides a method, a system, a terminal and a medium for constructing a target detection model and pursuing responsibility by running red light, which have the following technical effects: when the non-motor vehicle runs the red light violation, the technical scheme of the invention can use the computer vision and the deep learning system to carry out automatic violation overtaking under the condition of non-human intervention, and carry out violation punishment on the non-motor vehicle driver with the red light running behavior, thereby standardizing the driving behavior of the non-motor vehicle driver and ensuring the driving safety.

The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.

Drawings

Fig. 1 is a schematic structural diagram of a deep learning-based red light violation accountability system according to an embodiment of the present invention.

Fig. 2 is a schematic flow chart of a target detection model construction method based on deep learning in an embodiment of the present invention.

Fig. 3 is a flowchart illustrating a red light violation tracing method based on deep learning according to an embodiment of the present invention.

Fig. 4 is a schematic structural diagram of a target detection model building terminal based on deep learning in an embodiment of the present invention.

Fig. 5 is a schematic structural diagram of a deep learning-based red light running accountability terminal according to an embodiment of the present invention.

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," and/or "comprising," when used in this specification, specify the presence of stated features, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, operations, elements, components, items, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions or operations are inherently mutually exclusive in some way.

In the prior art, the traffic department sets a checkpoint at a road junction to grab an offending actor to trace the red light running behavior of the non-motor vehicle, but the tracing mode needs a large amount of manpower, cannot cover all road junctions and has the problem of incomplete supervision.

In view of the above, the invention provides a red light running responsibility tracing scheme based on deep learning, which aims to process a monitoring video stream of a road crossing, firstly, a red light running detection and identification system returns position information of a target vehicle running red light in a specified area, then, a license plate identification system identifies license plate position information and character information in the specified area, the target vehicle running red light is compared with the vehicle identification information by the red light running time of the traffic light, license plate information of the target vehicle running red light is determined, and a license plate number and a violation vehicle target image are returned, so that the purposes of all-weather monitoring and violation necessity of red light running of the vehicle are achieved, and the problem that the vehicle running red light is difficult to trace in the prior art is solved.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention are further described in detail by the following embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The first embodiment is as follows:

fig. 1 is a schematic structural diagram illustrating a deep learning-based red light running responsibility-following system according to an embodiment of the present invention. The license plate recognition and red light running tracing system 100 in this embodiment mainly includes the following modules: the system comprises an image acquisition module 101, a labeling module 102, a vehicle and red light detection module 103, a license plate detection module 104 and a red light running judgment module 105.

The image acquisition module 101 is disposed in a designated area and is configured to acquire a traffic video stream in the designated area. For example, a camera may be installed at a road junction, a video stream of the road junction is captured by the camera, and the captured video stream is labeled.

In some examples, the image acquisition module 101 may employ a camera module comprising a camera device, a storage device, and a processing device; the image capturing device includes but is not limited to: a camera, a video camera, an image pickup module integrated with an optical system or a CCD chip, an image pickup module integrated with an optical system and a CMOS chip, and the like, but the present embodiment is not limited thereto.

The labeling module 102 is configured to label the acquired traffic video stream to obtain first labeling data; the annotation content of the first annotation data includes, but is not limited to, the following categories: the method comprises the following steps of 1) marking a target frame in a red light display area of a traffic indicator light, wherein the marking content comprises the center point coordinate of the target frame, the width and the height of the target frame; preferably, the marking target frame comprises a complete red light display area, and the target frame is square; and 2) marking a target frame on the vehicle, wherein the marking content comprises the center point coordinate of the target frame, the width and the height of the target frame.

The labeling module 102 is further configured to perform secondary labeling on the vehicle target frame to obtain second labeling data, specifically, intercept the labeled vehicle target frame, perform secondary labeling on the license plate portion of each vehicle to obtain a license plate target frame, where the labeling content is a center point coordinate of the license plate target frame, a width and a height of the target frame.

The vehicle and red light detection module 103 is configured to input the first labeled data into the deep learning target detection model for training, test the deep learning target detection model after the training is completed, test the accuracy of the model for detecting the target, and select the target detection model with the smallest model loss and the highest accuracy as the vehicle and red light detection model for identifying the target vehicle and the traffic light according to the test result.

It should be noted that the target detection model is a deep convolutional neural network model, a picture is input, the picture is converted into an input of a convolutional neural network, the input is extracted into different dimensions through convolutional layers, information of the different dimensions is respectively input into a full convolutional feature extractor, each feature layer outputs a prediction result, and the results are regressed according to the degree of confidence to obtain a final prediction result. Specifically, the target detection model can be a candidate region-based target detection model (e.g., R-CNN model, Fast R-CNN model, FPN model, etc.), a single target detection model (e.g., SSD model, YOLO model, etc.), or the like.

In some examples, the vehicle and red light detection module 103 of the present embodiment is modified based on the existing object detection model as follows:

the improvement point 1) utilizes a clustering algorithm to perform target frame clustering on target vehicles marked in an actual scene and a red light display area of a traffic indicator lamp, so as to obtain the reference size of a target frame used closest to the scene. The clustering algorithm used in this embodiment includes, but is not limited to, K-Means clustering algorithm, mean shift clustering algorithm, density-based clustering algorithm, maximum expected clustering algorithm using gaussian mixture model, agglomerative hierarchical clustering algorithm, or graph community detection clustering algorithm, etc.

And 2) carrying out error square sum judgment on the basis of carrying out target frame clustering on target vehicles marked in the actual scene and red light display areas of traffic indicator lamps by utilizing a clustering algorithm so as to solve the problem that a clustering result possibly causes local optimization rather than global optimization.

It is worth mentioning that the invention adopts a fully intelligent recognition system, the traffic light signal state is judged by computer vision, manual intervention is not needed, signal access of a traffic light display is not needed, and the invention is different from the prior method that the time sequence module of the traffic light and the signal light is needed to be connected to judge the state of the traffic light.

The license plate detection module 104 is configured to input the second labeled data into the deep learning target detection model for training, test the deep learning target detection model after the training is completed, test the accuracy of the model for detecting the target, and select the model with the minimum model loss and the highest accuracy as the license plate detection model for identifying the license plate number according to the test result.

The red light running judgment module 105 is used for inputting the current traffic video stream into the vehicle, the red light detection model and the license plate detection model, and judging whether a red light running behavior occurs according to the output result of the model so as to carry out rule-breaking pursuit on the red light running behavior, and the specific execution process is as follows.

Step 1) obtaining a current traffic video stream, and determining coordinate values of a red light target frame of an actual scene in the current traffic video stream, for example, determining actual coordinates of a square area where a red light of a traffic indicator light of a current crossing is: upper left point coordinates (s1, q1), lower right point coordinates (s2, q 2).

And 2) inputting the current traffic video stream into the vehicle and red light detection model, and outputting the coordinate values of the vehicle target frame and the coordinate values of the red light detection target frame by the model. For example, the video stream of the non-motor vehicle road camera is input into the vehicle and red light detection model, and the model outputs the coordinates [ x _ min, y _ min, x _ max, y _ max ] of the non-motor vehicle target frame; and outputs coordinates [ s _ min, q _ min, s _ max, q _ max ] of the red light target frame.

And 3) calculating the overlapping degree of the red light detection target frame and the red light target frame of the actual scene, and judging whether the traffic signal lamp in the current traffic video stream is in a red light state or not according to the overlapping degree calculation result. Specifically, if the overlapping degree exceeds the overlapping degree threshold value, the current traffic signal lamp is judged to be in a red light state; and if the overlap degree threshold value is not exceeded, judging that the current traffic signal lamp is not a red lamp.

In some examples, the overlap threshold is determined by separately testing red light state detection accuracy for the same video stream at different overlap thresholds. For example, in this embodiment, the overlap threshold is set to be 0.3, that is, if the calculated overlap IOU between the red light detection target frame and the red light target frame of the actual scene is greater than or equal to 0.3, it can be determined that the current traffic signal lamp is in the red light state; if the degree of overlap IOU is less than 0.3, the current traffic signal lamp is judged to be not red; the method is a conclusion obtained by a large number of practical scene application test experiments, and when the test result shows that the IOU is more than or equal to 0.3, the influence of the offset distance of the red light target frame influenced by noise on the judgment of the on and off states of the red light in the practical scene can be ensured to be minimum. It should be understood that the overlapping degree threshold is not limited in this embodiment, and the overlapping degree threshold may be set to 0.1,0.2,0.3, …,0.8,0.9, etc. in a pace of 0.1.

And 4) inputting the vehicle target frame of the current traffic video stream detected as the red light into a license plate detection model, wherein the license plate detection model carries out license plate detection on the input vehicle target frame and outputs the position information of the detected license plate target frame.

And 5) extracting a license plate target frame output by the license plate detection model, performing character recognition after character segmentation on the license plate target frame to recognize the license plate number of the license plate, and finally, synchronously outputting the recognized license plate number and a vehicle picture as a basis for red light running violation punishment, adding a timestamp and uploading the timestamp to a violation database of a server so as to perform responsibility pursuing punishment on an actor running the red light.

Therefore, when the non-motor vehicle breaks the red light violation, the license plate recognition and red light violation accountability system provided by the invention can use a computer vision and deep learning system to automatically carry out violation accountability under the condition of non-human intervention, and punishment on the violation of the non-motor vehicle driver with the red light violation is carried out, so that the driving behavior of the non-motor vehicle driver is regulated, and the driving safety is guaranteed.

It should be understood that the division of the modules of the above system is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the red light running determination module may be a processing element separately installed, or may be implemented by being integrated in a chip of the system, or may be stored in a memory of the system in the form of program codes, and the processing element of the system calls and executes the functions of the red light running determination module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

Example two:

fig. 2 is a schematic flow chart illustrating a method for constructing a deep learning-based target detection model according to an embodiment of the present invention. The method flow of the present embodiment mainly includes the following steps.

Step S201: and acquiring the traffic video stream in the designated area.

Step S202: and labeling the traffic video stream to obtain a vehicle target frame and a red light display area target frame, and forming first labeling data.

In some examples, the first annotation data includes center point coordinate data, width and height data of the red light display area target frame, and further includes center point coordinate data, width and height data of the vehicle target frame.

Step S203: and performing secondary marking on the vehicle target frame to obtain a license plate target frame and forming second marking data.

In some examples, the second labeling data includes center point coordinate data, width, and height data of the license plate target frame.

Step S204: inputting the first labeling data into a first deep learning target detection model for training, testing the first deep learning target detection model after the training is finished, and determining a vehicle and a red light detection model for identifying the target vehicle and the red light state according to the test result.

In some examples, the target detection model is a deep convolutional neural network model, the input is a picture, the picture is converted into the input of a convolutional neural network, the input is extracted into different dimensions through convolutional layers, information of the different dimensions is respectively input into a full convolutional feature extractor, each feature layer outputs a prediction result, and the results are regressed according to the confidence degree to obtain the final prediction result. Specifically, the target detection model can be a candidate region-based target detection model (e.g., R-CNN model, Fast R-CNN model, FPN model, etc.), a single target detection model (e.g., SSD model, YOLO model, etc.), or the like.

Step S205: inputting the second labeling data into a second deep learning target detection model for training, testing the second deep learning target detection model after the training is finished, and determining a license plate detection model for recognizing a license plate according to a test result.

It should be noted that the method for constructing the target detection model based on deep learning in this embodiment is similar to the implementation of the red light running responsibility-following system based on deep learning in the first embodiment, and thus is not described again.

It should be noted that the target detection model construction method based on deep learning in this embodiment can be applied to various types of hardware devices. The hardware device may be a controller, such as an arm (advanced RISC machines) controller, an fpga (field Programmable Gate array) controller, a soc (system on chip) controller, a dsp (digital Signal processing) controller, or an mcu (micro controller unit) controller; the hardware device may also be a Personal computer, such as a desktop computer, a notebook computer, a tablet computer, a smart phone, a smart television, a Personal Digital Assistant (PDA for short), and the like; the hardware device may also be a server, and the server may be arranged on one or more entity servers according to various factors such as functions, loads, and the like, or may be formed by a distributed or centralized server cluster, which is not limited in this embodiment.

Example three:

fig. 3 is a flowchart illustrating a red light running tracing method based on deep learning according to an embodiment of the present invention. The method flow of the present embodiment mainly includes the following steps.

Step S301: and acquiring the current traffic video stream, and determining the coordinates of the red light target frame of the actual scene. For example, the actual coordinates of the square area where the red light of the traffic light at the current crossing is located are determined as follows: upper left point coordinates (s1, q1), lower right point coordinates (s2, q 2).

Step S302: and inputting the current traffic video stream into a vehicle and red light detection model obtained by training a deep learning target detection model, and acquiring vehicle target frame coordinates and red light display area target frame coordinates output by the vehicle and red light detection model.

For example, the video stream of the non-motor vehicle road camera is input into the vehicle and red light detection model, and the model outputs the coordinates [ x _ min, y _ min, x _ max, y _ max ] of the non-motor vehicle target frame; and outputs coordinates [ s _ min, q _ min, s _ max, q _ max ] of the red light target frame.

Step S303: and calculating the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model, and judging the red light state of the current video stream according to the overlapping degree calculation result. Specifically, if the overlapping degree exceeds the overlapping degree threshold value, the current traffic signal lamp is judged to be in a red light state; and if the overlap degree threshold value is not exceeded, judging that the current traffic signal lamp is not a red lamp.

Step S304: and inputting the vehicle target frame corresponding to the current traffic video stream judged as the red light into a license plate detection model obtained by training a deep learning target detection model, and acquiring the coordinates of the license plate target frame output by the license plate detection model.

Step S305: and performing character recognition on the license plate target frame output by the license plate detection model to obtain the license plate number.

Step S306: and synchronously outputting the license plate number and the vehicle image for tracing the responsibility of running the red light.

In some examples, the method further comprises the step of synchronously outputting the identified license plate number and the vehicle picture as the basis for punishing the red light running violation, and adding a timestamp to upload the timestamp to a violation database of the server so as to punish the red light running agent for responsibility pursuit.

It should be noted that the red light running responsibility-following method based on deep learning in this embodiment is similar to the red light running responsibility-following system based on deep learning in the first embodiment, and thus, the description is omitted.

It should be noted that the deep learning-based red light violation tracing method in this embodiment can be applied to various hardware devices. The hardware device may be a controller, such as an arm (advanced RISC machines) controller, an fpga (field Programmable Gate array) controller, a soc (system on chip) controller, a dsp (digital Signal processing) controller, or an mcu (micro controller unit) controller; the hardware device may also be a Personal computer, such as a desktop computer, a notebook computer, a tablet computer, a smart phone, a smart television, a Personal Digital Assistant (PDA for short), and the like; the hardware device may also be a server, and the server may be arranged on one or more entity servers according to various factors such as functions, loads, and the like, or may be formed by a distributed or centralized server cluster, which is not limited in this embodiment.

Example four:

fig. 4 is a schematic structural diagram illustrating a target detection model building terminal based on deep learning according to an embodiment of the present invention.

The target detection model building terminal 400 in this embodiment includes:

a first storage unit 401 for storing at least one computer program. Illustratively, the first storage unit 401 may include one or more memories. The memory may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In certain embodiments, the memory may also include memory that is remote from the one or more processors, such as network attached memory that is accessed via RF circuitry or external ports and a communications network, which may be the internet, one or more intranets, local area networks, wide area networks, storage area networks, and the like, or suitable combinations thereof. The memory controller may control access to the memory by other components of the device, such as the CPU and peripheral interfaces.

A first processing unit 402 for executing the at least one computer program for performing, for example, the method of fig. 2. Illustratively, the first processing unit 402 may include one or more processors, which may be one or more general purpose microprocessors, one or more special purpose processors, one or more field programmable logic arrays, or any combination thereof.

For example, the terminal 400 for constructing the parking specification detection model may be implemented in various processing terminals, such as a server, a desktop computer, a notebook computer, a smart phone, a tablet computer, smart glasses, a smart band, a smart watch, and the like, which is not limited in this embodiment.

Example five:

fig. 5 is a schematic structural diagram illustrating a deep learning-based red light running responsibility-following terminal according to an embodiment of the present invention.

The terminal 500 for tracing the responsibility of running red light in this embodiment includes:

a second storage unit 501 for storing at least one computer program. Illustratively, the second storage unit 501 may include one or more memories. The memory may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In certain embodiments, the memory may also include memory that is remote from the one or more processors, such as network attached memory that is accessed via RF circuitry or external ports and a communications network, which may be the internet, one or more intranets, local area networks, wide area networks, storage area networks, and the like, or suitable combinations thereof. The memory controller may control access to the memory by other components of the device, such as the CPU and peripheral interfaces.

A second processing unit 502 for executing the at least one computer program for performing, for example, the method of fig. 3. Illustratively, the second processing unit 502 may include one or more processors, which may be one or more general purpose microprocessors, one or more special purpose processors, one or more field programmable logic arrays, or any combination thereof.

For example, the terminal 500 for tracing responsibility when running red light may be implemented in various processing terminals, such as a server, a desktop computer, a notebook computer, a smart phone, a tablet computer, smart glasses, a smart bracelet, and a smart watch, which is not limited in this embodiment.

Example six:

the present embodiment provides a computer-readable storage medium storing at least one computer program, which when executed, executes the deep learning-based object detection model construction method; or, executing the deep learning-based red light running tracing method.

It will be appreciated that the various functions performed in the foregoing embodiments relate to computer software products; the computer software product is stored in a storage medium, and is used for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention, such as the steps of the flowcharts in the embodiments of the methods in fig. 2 and 3, when the computer software product is executed.

In embodiments provided herein, the computer-readable and writable storage medium may comprise read-only memory, random-access memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, a USB flash drive, a removable hard disk, or any other medium which can be used to store desired program code in the form of instructions or data structures and which can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable-writable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are intended to be non-transitory, tangible storage media. Disk and disc, as used in this application, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

In one or more exemplary aspects, the functions described by the computer program referred to in the method flow of the invention may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The steps of a disclosed method or algorithm may be embodied in processor-executable software modules, which may be located on a tangible, non-transitory computer-readable and/or writable storage medium. Tangible, non-transitory computer readable and writable storage media may be any available media that can be accessed by a computer.

The flowcharts and block diagrams in the above-described figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In conclusion, the invention provides a method, a system, a terminal and a medium for constructing a target detection model and tracing the responsibility of running a red light, when the violation phenomenon of running the red light of a non-motor vehicle occurs, the technical scheme of the invention can use a computer vision and deep learning system to automatically trace the responsibility of running the rules and regulations under the condition of non-human intervention, and carry out violation punishment on the non-motor vehicle driver with the behavior of running the red light, thereby standardizing the driving behavior of the non-motor vehicle driver and ensuring the driving safety.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. A target detection model construction method based on deep learning is characterized by comprising the following steps:

acquiring a traffic video stream in a designated area;

marking the traffic video stream to obtain a vehicle target frame and a red light display area target frame, and forming first marking data;

performing secondary marking on the vehicle target frame to obtain a license plate target frame and forming second marking data;

inputting the first marking data into a first deep learning target detection model for training, testing the first deep learning target detection model after the training is finished, and determining a vehicle and a red light detection model for identifying the target vehicle and the red light state according to the test result;

inputting the second labeling data into a second deep learning target detection model for training, testing the second deep learning target detection model after the training is finished, and determining a license plate detection model for recognizing a license plate according to a test result.

2. The method of constructing an object detection model of claim 1, further comprising:

carrying out target frame clustering on the vehicle target frame and the red light display area target frame to obtain a target frame reference size which is close to that used in an actual scene;

and performing error square sum judgment on the clustered vehicle target frames and the red light display area target frames.

3. The method for constructing a target detection model according to claim 1, wherein the first labeling data includes center point coordinate data, width and height data of the target frame of the red light display area, and further includes center point coordinate data, width and height data of the target frame of the vehicle; the second labeling data comprise coordinate data of the center point of the license plate target frame, width data and height data.

4. A red light running tracing method based on deep learning is characterized by comprising the following steps:

acquiring a current traffic video stream, and determining coordinates of a red light target frame of an actual scene;

inputting the current traffic video stream into a vehicle and red light detection model obtained by training a deep learning target detection model, and acquiring vehicle target frame coordinates and red light display area target frame coordinates output by the vehicle and red light detection model;

calculating the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model, and judging the red light state of the current video stream according to the overlapping degree calculation result;

inputting a vehicle target frame corresponding to the current traffic video stream judged as the red light into a license plate detection model obtained by training a deep learning target detection model, and acquiring the coordinates of the license plate target frame output by the license plate detection model;

performing character recognition on the license plate according to the license plate target frame coordinates output by the license plate detection model to obtain a license plate number;

and synchronously outputting the license plate number and the vehicle image for pursuing responsibility when running the red light.

5. The method according to claim 4, wherein the determining the red light status of the current video stream according to the overlapping degree calculation result comprises:

judging whether the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model exceeds an overlapping degree threshold value or not;

if the overlapping degree threshold value is exceeded, judging that the traffic indicator light in the current traffic video stream is in a red light state;

otherwise, judging that the traffic indicator light in the current traffic video stream is not in the red light state.

6. The method according to claim 5, wherein the overlap threshold is determined by testing the red light detection accuracy of the same traffic video stream at different overlap thresholds.

7. The red light running accountability method according to claim 4, further comprising:

and adding timestamps to the license plate number and the vehicle image and uploading the license plate number and the vehicle image to a pre-specified violation database.

8. A deep learning-based red light running accountability system is characterized by comprising:

the image acquisition module is used for acquiring a traffic video stream in a specified area;

the marking module is used for marking the traffic video stream to obtain a vehicle target frame and a red light display area target frame and forming first marking data; the license plate target frame is obtained after the vehicle target frame is subjected to secondary marking, and second marking data are formed;

the vehicle and red light detection module is used for inputting the first marking data into a first deep learning target detection model for training, testing the first deep learning target detection model after the training is finished, and determining a vehicle and a red light detection model for identifying the states of the target vehicle and the red light according to a test result;

the license plate recognition module is used for inputting the second labeled data into a second deep learning target detection model for training, testing the second deep learning target detection model after the training is finished, and determining a license plate detection model for recognizing a license plate according to a test result;

the red light running judgment module is used for acquiring the current traffic video stream and determining the coordinates of a red light target frame of an actual scene; inputting the current traffic video stream into the vehicle and red light detection model, and acquiring coordinates of a vehicle target frame and coordinates of a red light display area target frame; calculating the overlapping degree between the coordinates of the red light target frame of the actual scene and the coordinates of the red light display area target frame output by the vehicle and red light detection model, and judging the red light state of the current video stream according to the overlapping degree calculation result; inputting a vehicle target frame corresponding to the current traffic video stream judged as the red light into the license plate detection model, and acquiring the coordinates of the license plate target frame output by the license plate detection model; performing character recognition on the license plate according to the license plate target frame coordinates output by the license plate detection model to obtain a license plate number; and synchronously outputting the license plate number and the vehicle image for pursuing responsibility when running the red light.

9. A target detection model building terminal based on deep learning is characterized by comprising the following steps:

a first storage unit for storing at least one computer program;

a first processing unit for executing the at least one computer program to perform the method for constructing a deep learning based object detection model according to any one of claims 1 to 3.

10. The utility model provides a terminal of following responsibility of making a dash across red light based on deep learning which characterized in that includes:

a second storage unit for storing at least one computer program;

a second processing unit, configured to run the at least one computer program to execute the deep learning-based red light violation tracing method according to any one of claims 4 to 7.

11. A computer-readable storage medium, in which at least one computer program is stored, the at least one computer program being executed to perform the deep learning-based object detection model construction method according to any one of claims 1 to 3; or, the deep learning-based red light running accountability method according to any one of claims 4 to 7 is performed.