CN110197589A - A kind of illegal detection method of making a dash across the red light based on deep learning - Google Patents

A kind of illegal detection method of making a dash across the red light based on deep learning Download PDF

Info

Publication number
CN110197589A
CN110197589A CN201910459288.1A CN201910459288A CN110197589A CN 110197589 A CN110197589 A CN 110197589A CN 201910459288 A CN201910459288 A CN 201910459288A CN 110197589 A CN110197589 A CN 110197589A
Authority
CN
China
Prior art keywords
vehicle
lane
traffic
red light
line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910459288.1A
Other languages
Chinese (zh)
Other versions
CN110197589B (en
Inventor
胡程
陈教
李万清
钱逯
胡明中
唐建斌
洪青阳
袁友伟
彭瀚
夏嘉璐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Cheng Road Polytron Technologies Inc
Original Assignee
Hangzhou Cheng Road Polytron Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Cheng Road Polytron Technologies Inc filed Critical Hangzhou Cheng Road Polytron Technologies Inc
Priority to CN201910459288.1A priority Critical patent/CN110197589B/en
Publication of CN110197589A publication Critical patent/CN110197589A/en
Application granted granted Critical
Publication of CN110197589B publication Critical patent/CN110197589B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The illegal detection method of making a dash across the red light based on deep learning that the invention discloses a kind of, is related to traffic offence detection field.The present invention predefines the lane of traffic light intersection and the position of signal lamp, and signal lamp is matched with corresponding lane.The case where traffic light intersection, is photographed to record by camera, the photograph collection that the later period carrys out camera transmission focuses on, and judges the vehicle with the presence or absence of making a dash across the red light behavior in the position of different picture frames on same vehicle under same signal lamp.In addition for vehicle outside line unrecognized due to hypertelorism, identify whether it is known vehicle by the method that vehicle similarity compares.The present invention combines many algorithms and model, and this novel traffic offence detection method is made to meet urban traffic management for the accuracy rate of illegal detection and the demand of flexibility.

Description

A kind of illegal detection method of making a dash across the red light based on deep learning
Technical field
The present invention relates to traffic offence detection fields, and in particular to one kind is known based on deep learning convolutional neural networks image The illegal detection method of making a dash across the red light for the traffic light intersection that other and image similarity compares.
Background technique
With society and economic increasingly developed, automobile plays an increasingly important role in our life.? The problems such as a series of traffic violations, blocking are brought while facilitating the routine life of people.The overwhelming majority therein is again It is caused by breaking rules and regulations or make a dash across the red light as automobile driver.
Mode that there are mainly two types of the principles captured in the traffic control system in China city to violation vehicle at present: one Kind is embedded underground induction coil, and candid photograph of the digital camera to making a dash across the red light is set up on cross bar.This method there are it is at high cost, be packed into and The not high several significant drawbacks of inconvenient maintenance, not flexible, capture rate.Another kind is to set up video camera, for red to exceeding the speed limit, rushing Lamp, violation vehicle etc. are used videotape to record in real time.That there are accuracys is low for this processing method, occupy a large amount of police strength, law enfrocement official is easy Fatigue is unable to the disadvantages of all weather operations.
In view of the foregoing, it is necessary to propose a kind of novel traffic light intersection Vehicle Detection management method.
Summary of the invention
It is an object of the invention to overcome above two traditional the at high cost of illegal vehicle monitor mode, low efficiency, no Flexibly, the shortcomings that low accuracy, and a kind of illegal detection method of making a dash across the red light based on deep learning is provided, meets urban road Traffic administration is for the accuracy rate of illegal detection and the demand of flexibility.
Specific technical solution of the present invention is as follows:
A kind of illegal detection method of making a dash across the red light based on deep learning, for detecting whether there are vehicles running red light, step It is rapid as follows:
S1: by the camera being installed on above traffic light intersection to be monitored, the video for the traffic light intersection that takes a crane shot is obtained Image；It takes a frame to pre-process to appointing in video image, positions signal lamp region in the camera scope of sight and several Lane；
The signal lamp region be position of the signal lamp in video image, every lane be all separated out line inner region and Line exterior domain；
The line inner region in every lane is the lane not yet across the holding areas of stop line, by two lane lines of two sides As boundary, determine that the direction of traffic in every lane, direction of traffic are in turning left, keep straight on, turning right according to the mark for being oriented to arrow At least one；
The line exterior domain in every lane is across the crossing interior zone after stop line；Direction of traffic single for every Lane, at it across being equipped with a line exterior domain in the planning driving path after stop line；Have for every and keeps straight on and turn to two kinds The lane of direction of traffic, is equipped with inside and outside two line exterior domains in its planning driving path, and inside cord exterior domain is located on the outside of stop line Straight trip and turn to planning driving path overlapping region on, out conductor exterior domain be located at straight trip planning driving path on but be located at turn to roadway Outside diameter；
S2: in the detection process, to the video image that same camera takes, signal lamp is carried out to continuous picture frame Color identification, remains with the picture frame of red eye in order, the picture frame collection that obtains that treated；
S3: the every picture frame concentrated for picture frame detects the red light direction in its signal lamp region, then according to red Lamp direction and the corresponding lane that no through traffic are associated, wherein the association corresponding with left turn lane of left-hand rotation arrow, straight trip arrow with directly The corresponding association of runway, the association corresponding with right-turn lane of right-hand rotation arrow, there are two types of the lane of direction of traffic and two kinds of driving sides for tool It is associated with simultaneously to corresponding arrow；
S4: dividing picture frame collection, and successive image frame all has unidirectional red light if it exists, then by these figures As frame is divided into a pictures, several pictures are thus obtained, wherein i-th of pictures is denoted as Pi=[p1, p2 ..., ps, Type], type indicates an identical red light direction in signal lamp, and p1, p2 ..., ps is respectively the 1st, 2 ..., and s is opened with type The picture frame of direction red eye arrow；
S5: for each pictures Pi, vehicle identification is carried out to each frame image respectively by vehicle detection model, often One frame image only needs to detect red light direction type in the image and corresponds in associated no through traffic lane whether vehicle occur；For The vehicle identified continues that its license plate is identified and recorded, and is carried out license plate if it can not identify license plate in present image Unidentified label；
S6: for each pictures Pi, the vehicle identified for each judges that it is located at locating lane, with And judge that it is in the line inner region in the lane or line exterior domain；I.e. not online inner region also not online exterior domain vehicle into Row screens out；
S7: according to the recognition result of S6, to each pictures PiIn the illegal vehicle being accused of making a dash across the red light determined:
S71: if in the pictures red light direction type correspond to it is associated no through traffic that lane only has single direction of traffic, The vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames is obtained, the first vehicle emptied in advance is put into In set, then the vehicle in the line exterior domain in the lane that appears in that no through traffic in all picture frames is obtained, is put into advance In the second vehicle set emptied；A vehicle first appears in the first vehicle set if it exists, then appears in the second vehicle again In set, then judge the vehicle for illegal vehicle.
S72: if red light direction type corresponds to associated no through traffic lane and has and keeps straight on and turn to two kinds in the pictures Direction of traffic then obtains the vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames, is put into advance In the first vehicle set emptied；The vehicle in the inside cord exterior domain in the lane that appears in that no through traffic in all picture frames is obtained again , it is put into the second vehicle set emptied in advance；The outer of the lane that appears in that no through traffic in all picture frames is obtained again Vehicle in side line exterior domain is put into the third vehicle set emptied in advance；Red light direction in pictures is judged, if red Lamp direction type is straight trip direction and there are a vehicles to first appear in the first vehicle set, then appears in third vehicle set In, then judge the vehicle for illegal vehicle；If red light direction type is turn direction and there are a vehicles to first appear at the first vehicle In set, then appears in the second vehicle set but do not appear in third vehicle set, then judge the vehicle to be illegal Vehicle.
Based on above scheme, each step can also be realized using following preferred embodiment.
Preferably, in the step S1, the camera is fixing camera, is owned in the video image obtained The visual angle of picture frame and scope of sight immobilize.
Preferably, in the step S2, signal lamp color is carried out to picture frame and knows method for distinguishing are as follows: first in picture frame It is middle to set area-of-interest for signal lamp region, the rgb color space of area-of-interest is then transformed into HSV space, then Signal lamp color is judged from HSV space.
Preferably, the signal lamp is arrowhead-shaped traffic lights, there is advance, left-hand rotation, three kinds of directions of turning right；Described The detection method in red light direction in step S3 are as follows: establish the template library of different directions arrow, calculating color in signal lamp region is The similitude in red region and template, so that it is determined that arrow direction
Preferably, in the step S5, using convolutional neural networks model inspection vehicle, when detection, is divided an image into The grid of s × s, if the center of a target falls into one of grid, which is responsible for detecting the vehicle in the region；Often A window area output bounding box, bounding box detected is containing there are five information [x1,y1,x2,y2, Confidence] wherein (x1,y1),(x2,y2) refer to vehicle window left upper apex and bottom right vertex that current grid is predicted Coordinate.Confidence indicates current window confidence score；Window to confidence level lower than threshold value excludes, coincidence Minimum outsourcing rectangle of the highest window of window selection confidence level as vehicle.
Further, the convolutional neural networks model uses YOLO model.
Preferably, in the step S5, the recognition methods of license plate are as follows: License Plate is carried out to vehicle window, uses mind Vehicle pictures are detected through network model, obtain license plate window position information；Then using in the convolutional neural networks identification of training The characters on license plate in license plate window is stated, license plate number is obtained.
Preferably, in the step 6, the recognition methods in lane locating for vehicle are as follows: calculate the minimum outsourcing rectangle of vehicle With the overlapping area of line inner region, line exterior domain, the ratio of vehicle minimum outsourcing rectangular area is accounted for, according to overlapping area to determine Lane locating for vehicle and the line inner region in the lane or line exterior domain.
Preferably, in the S7, judge a vehicle whether and meanwhile appear in the method in two vehicle set are as follows:
If a vehicle has license board information in the second vehicle set or third vehicle set, directly by the vehicle License board information is compared with the license board information of vehicle in the first vehicle set, shows the vehicle when identical while appearing in two In set；
If there is the vehicle with the unidentified label of license plate in the second vehicle set or third vehicle set, by itself and the Vehicle in one vehicle set carries out vehicle similarity comparison one by one, judges this two cars to be same when similarity is more than threshold value Vehicle, the license board information for obtaining the similar vehicle being located in the first vehicle set are assigned to the vehicle of unidentified license plate.
Preferably, in the step S7, vehicle similarity comparison method are as follows:
The two image F compared will be participated in first and image G zooms to identical size；
Then, image F and image the G grid for being respectively divided into S × S are obtained into subgraph F1,F2...Fs×sAnd G1, G2...Gs×s, the grey level histogram of all subgraphs of two field pictures is calculated, corresponding function f is obtained1,f2...fs×sAnd g1, g2...gs×s, for the correspondence subgraph of image F and image G, calculate separately the grey level histogram registration m of each subgraph1, m2...ms×s, the grey level histogram Total contact ratio M of two images is finally obtained as its similarity, and calculation method is as follows:
Note: fn(i) and gn(i) subgraph F is respectively representednWith subgraph GnGrey level histogram the i-th rank gray scale number of pixels, max(fn(i),gn(i)) f is indicatedn(i) and gn(i) biggish one of numerical value in.
The present invention in terms of existing technologies, has the advantages that the present invention overcomes traditional illegal vehicles The shortcomings that the at high cost of monitor mode, low efficiency, not flexible, low accuracy, provide it is a kind of it is intelligent, accuracy is high, low Cost monitors the flexibly illegal detection method of making a dash across the red light based on deep learning.The present invention combines many algorithms and model, Make this novel traffic offence detection method meet urban traffic management for illegal detection accuracy rate and flexibly The demand of property.
Detailed description of the invention
The step of Fig. 1 is the illegal detection method of making a dash across the red light based on deep learning figure；
Fig. 2 is the overhaul flow chart in embodiment；
Fig. 3 is in embodiment to signal lamp color overhaul flow chart
Fig. 4 is the flow chart determined in embodiment signal lamp
Fig. 5 is the flow chart of vehicle similarity comparison process in embodiment；
Fig. 6 is the picture frame that the camera that device number is 33036xxxx in embodiment transmits；
Fig. 7 be embodiment in signal lamp region, line inner region, line exterior domain division schematic diagram；
Fig. 8 is the schematic diagram in embodiment after the visualization of information of vehicles sequence；
Fig. 9 is the second frame image in pictures.
Specific embodiment
The present invention is further elaborated and is illustrated with reference to the accompanying drawings and detailed description.Each implementation in the present invention The technical characteristic of mode can carry out the corresponding combination under the premise of not conflicting with each other.
As shown in Figure 1, being a kind of flow chart of illegal detection method of making a dash across the red light based on deep learning, this method is for examining Survey whether there is vehicles running red light, therefore its it is illegal refer to it is illegal make a dash across the red light, be detailed below the detection method realization step It is rapid:
S1: by the camera being installed on above traffic light intersection to be monitored, the video for the traffic light intersection that takes a crane shot is obtained Image.Wherein, it is contemplated that the needs that subsequent image compares, the camera need to be adopted as fixing camera, the video obtained The visual angle of all picture frames and scope of sight immobilize in image, thereby guarantee that the background elements such as signal lamp, lane line are constant.
In order to obtain the element positions such as signal lamp, lane line, needs to take a frame to pre-process in video image, determine Signal lamp region in the camera scope of sight of position and the lane to be monitored below several cameras.In order to make accurate positioning, Vehicle interference should be reduced in the picture frame of taking-up to the greatest extent, can more clearly show lane line position and signal location.
Wherein, signal lamp region is position of the signal lamp in video image, and signal lamp region is being regarded by signal lamp profile Angular coordinate in frequency image determines.And every lane is all separated out line inner region and line exterior domain, the line inner region in every lane Domain is the lane not yet across the holding areas of stop line, by two lane lines of two sides as boundary, according to guiding arrow The direction of traffic for determining every lane is identified, direction of traffic is at least one of to turn left, keep straight on, turn right.The line in every lane Inner region is determined by 4 extreme coordinates of two lane lines of two sides.The line exterior domain in every lane is across after stop line Crossing interior zone.
Since the shooting of camera has certain angle, it will lead to vehicle and be although not above stop line, but in image In its profile be really located at the crossing interior zone outside stop line.Therefore it in actual division line inner region and line exterior domain, needs Consider to translate the top boundary of holding areas towards crossing interior side, with state of the front part of vehicle just in road surface stop line Under, subject to the highest vertex position of the vehicle's contour shot by camera, the horizontal line across this vertex can be with As the practical stop line in image.And the bottom edge of line exterior domain should also be as maintaining a certain distance with the stop line on road surface, It can be subject to above-mentioned practical stop line, equally avoid because the angle of camera asks generation erroneous judgement.It, can when with pavement To select the boundary from the middle line on pavement initially as line inner region and line exterior domain.
It should be noted that the lane of direction of traffic single for every, at it across in the planning driving path after stop line Equipped with a line exterior domain, when vehicle enters the line exterior domain surface it crossed actual road surface stop line, work as letter Indicate that its is illegal when signal lamp is red light.But there is the lane kept straight on and turn to two kinds of direction of traffic for every, due to it Tool there are two types of driving direction, a kind of driving direction be red light it is alternatively possible be green light, single one line exterior domain is unable to satisfy Testing requirements need to be equipped with inside and outside two line exterior domains in its planning driving path, respectively outside inside cord exterior domain and out conductor Region.Wherein, inside cord exterior domain is located at the straight trip on the outside of stop line and turns on planning driving path overlapping region, and outside out conductor Region, which is located in straight trip planning driving path but is located at, to be turned to outside planning driving path, and out conductor exterior domain is closer to opposite road in general Mouthful.Each line exterior domain is quadrangle form, is determined by four angular coordinates in region.
S2: in the detection process, to the video image that same camera takes, signal lamp is carried out to continuous picture frame Color identification, remains with the picture frame of red eye in order, the picture frame collection that obtains that treated.
Fig. 3 is to carry out signal lamp color to signal lamp color overhaul flow chart to picture frame and know method for distinguishing are as follows: exist first Area-of-interest is set by signal lamp region in picture frame, is affected because RGB picture is illuminated by the light, by picture Rgb color space is transformed into HSV space, then judges signal lamp color, specific color judgement from the HSV space of area-of-interest Threshold value can refer to Fig. 3, can also be according to practical adjustment.
In picture image set K={ k1,k2,k3...kmIn by the picture for having red eye retain, without red eye Picture reject, arranged by the sequencing of picture frame, the picture frame collection K '={ k that obtains that treated1,k2,k3...kn}。
S3: the every picture frame concentrated for picture frame detects the red light direction in its signal lamp region, then according to red Lamp direction and the corresponding lane that no through traffic are associated, wherein the association corresponding with left turn lane of left-hand rotation arrow, straight trip arrow with directly The corresponding association of runway, the association corresponding with right-turn lane of right-hand rotation arrow, there are two types of the lane of direction of traffic and two kinds of driving sides for tool It is associated with simultaneously to corresponding arrow.When a direction is red light in signal lamp, the direction is no through traffic direction, and and the party It is no through traffic lane to associated lane.Based on this, subsequent illegal detection is carried out.
There are many signal lamp types, and different types needs different detection methods to judge its red light direction.It is most common Signal lamp be arrowhead-shaped traffic lights, have advance, left-hand rotation, three kinds of directions of turning right, also use in subsequent embodiment of the present invention The traffic lights that these three direction arrows are constituted, for this signal lamp, the detection method in red light direction can use template Matching method, as shown in figure 4, specific practice are as follows:
The template library of different directions arrow is established, calculating color in signal lamp region is that red region is similar to template Property, so that it is determined that arrow direction.
S4: it after getting the picture frame collection with red light, needs to divide picture frame collection: successive image frame if it exists All there is unidirectional red light, then these picture frames is all extracted, be divided into a pictures.Certainly, due to simultaneously can There can be the red light of multiple directions, therefore same picture frame may be respectively present in different pictures.Thus it obtains several Pictures, wherein i-th of pictures is denoted as Pi=[p1, p2 ..., ps, type], type indicate signal lamp in identical one it is red Lamp direction, p1, p2 ..., ps are respectively the 1st, 2 ..., and s opens the picture frames with the direction type red eye arrow.Due to one It is all the red light in the same direction in a pictures, therefore this series of timing diagrams picture can be red for judging whether vehicle is rushed Lamp.It is not red light before first frame image in pictures, therefore first frame image represents original state when red light lights, Last frame represents the final state at the end of red light.If in the first frame, line inner region of the vehicle in no through traffic lane Domain, and vehicle then shows that running red light for vehicle is illegal in the lane line exterior domain in pictures subsequent frame；Similarly, if vehicle exists Certain intermediate frame is in the line inner region in no through traffic lane, and vehicle is again showed that in the lane line exterior domain in pictures subsequent frame Running red light for vehicle is illegal.Thus, it is only required to detect whether vehicle first appears at line inner region, occurring online exterior domain afterwards can be real The existing illegal detection of running red light for vehicle.Specific implementation process is described below in detail, detection is based on each pictures PiIt carries out, These pictures can be generated constantly in real time, can also carry out processed offline generation to the monitor video of camera.
S5: for each pictures Pi, vehicle identification is carried out to each frame image respectively by vehicle detection model, often One frame image only needs to detect red light direction type in the image and corresponds in associated no through traffic lane whether vehicle occur, remaining Lane without identification.Certainly, if remaining lane is also what no through traffic, then should have, another should no through traffic The pictures in direction detect its violation of law, but are without detection in current pictures.For pictures Pi In the vehicle that identifies, need to continue that its license plate is identified and recorded, by it if it can not identify license plate in present image The unidentified label of license plate is carried out, remains subsequent further to be identified.
In the present invention, convolutional neural networks model inspection vehicle can be used, YOLO can be used in convolutional neural networks model Model realization.The grid of s × s is divided an image into when detection, it, should if the center of a target falls into one of grid Grid is responsible for detecting the vehicle in the region；The window area each detected exports bounding box, and bounding box contains There are five information [x1,y1,x2,y2, confidence] wherein (x1,y1),(x2,y2) refer to the vehicle that current grid is predicted The coordinate of window left upper apex and bottom right vertex.Confidence indicates current window confidence score；Threshold is lower than to confidence level The window of value is excluded, minimum outsourcing rectangle of the highest window of window selection confidence level of coincidence as vehicle.
When Car license recognition, License Plate first can be carried out to vehicle window, use convolutional neural networks model orientation license plate Position obtains license plate window position information；Then the license plate in above-mentioned license plate window is identified using the convolutional neural networks of training Character obtains license plate number.
S6: for each pictures Pi, the vehicle identified for each judges that it is located at locating lane, with And judge that it is in the line inner region in the lane or line exterior domain.
Since vehicle itself has certain altitude, and camera shooting direction is inclined, it is therefore desirable to consider these because Element identifies lane locating for vehicle.A kind of recognition methods are as follows: calculate outside the minimum outsourcing rectangle and line inner region, line of vehicle The overlapping area in region accounts for the ratio of vehicle minimum outsourcing rectangular area according to overlapping area, to determine lane locating for vehicle And line inner region or line exterior domain in the lane.Vehicle is physically located in, and when a certain lane, overlapping area will necessarily It is greater than a certain threshold value, therefore the location of vehicle can be screened by one threshold value of setting.The threshold value needs basis Actual conditions are configured, related with camera angle, and different lanes are because of far and near different, the best threshold apart from camera Value is also likely to be present difference.
S7: according to the recognition result of S6, to each pictures PiIn the illegal vehicle being accused of making a dash across the red light determined, and When being the lane of single direction of traffic or be the lane of non-single direction of traffic due to lane, determining method method is different, because This needs judges in two ways respectively.In the present invention, the lane of non-single direction of traffic only considers two kinds of driving sides To lane, including straight trip+steering, such as straight trip+right-hand rotation, straight trip+left-hand rotation, remaining increasingly complex situation need additional special Design.
S71: if red light direction type corresponds to that associated no through traffic that lane only has single direction of traffic (example in the pictures As single straight trip or single left-hand rotation, such lane only have a line exterior domain), then it obtains in the pictures in all picture frames Vehicle in the line inner region in the lane that appears in that no through traffic, is put into the first vehicle set emptied in advance, then obtain Vehicle in the line exterior domain in the lane that appears in that no through traffic in the pictures in all picture frames is put into and empties in advance In second vehicle set；A vehicle first appears in the first vehicle set if it exists, then appears in again in the second vehicle set, Then judge the vehicle for illegal vehicle.
S72: if red light direction type corresponds to associated no through traffic lane and has and keeps straight on and turn to two kinds in the pictures Direction of traffic then obtains the vehicle in the line inner region in the lane that appears in that no through traffic in the pictures in all picture frames, will It is placed in the first vehicle set emptied in advance；The lane that appears in that no through traffic in the pictures in all picture frames is obtained again Inside cord exterior domain in vehicle, be put into the second vehicle set emptied in advance；It obtains in the pictures and owns again Vehicle in the out conductor exterior domain in the lane that appears in that no through traffic in picture frame, is put into the third vehicle collection emptied in advance In conjunction；Red light direction in pictures is judged, if red light direction type is straight trip direction and there are a vehicles to first appear at the first vehicle Set in, then appear in third vehicle set, then judge the vehicle for illegal vehicle；If red light direction type is turning Direction and there are a vehicles to first appear in the first vehicle set, then appears in the second vehicle set but does not appear in third In vehicle set, then judge the vehicle for illegal vehicle.
It should be noted that it is above-mentioned judge a vehicle whether and meanwhile appear in the method in two vehicle set and can pass through Confirmation is compared in the license board information detected in advance, but Some vehicles are in line exterior domain farther away from camera, it is understood that there may be Target is too far and leads to that vehicle pictures are fuzzy, cannot smoothly extract license board information situation, therefore that present invention adds vehicles is similar Degree control methods is assisted.When similarity compares, due to necessarily also going through line inner region, therefore in vehicular motion The image in the closer line inner region of camera can be called, license board information can be recognized substantially in this parts of images, such as Fruit vehicle is that same vehicle then can assign its license board information to similar vehicle.
It is of the invention it is a kind of judge a vehicle whether and meanwhile the method that appears in two vehicle set be implemented as follows:
If a vehicle has license board information in the second vehicle set or third vehicle set, directly by the vehicle License board information is compared with the license board information of vehicle in the first vehicle set, shows the vehicle when identical while appearing in two In set.
If there is the vehicle with the unidentified label of license plate in the second vehicle set or third vehicle set, by itself and the Vehicle in one vehicle set carries out vehicle similarity comparison one by one, judges this two cars to be same when similarity is more than threshold value Vehicle, the license board information for obtaining the similar vehicle being located in the first vehicle set are assigned to the vehicle of unidentified license plate.
Further, a kind of specific implementation of vehicle similarity comparison method are as follows:
The two image F compared will be participated in first and image G zooms to identical size；
Then, image F and image the G grid for being respectively divided into S × S are obtained into subgraph F1,F2...Fs×sAnd G1, G2...Gs×s, the grey level histogram of all subgraphs of two field pictures is calculated, corresponding function f is obtained1,f2...fs×sAnd g1, g2...gs×s, for the correspondence subgraph of image F and image G, calculate separately the grey level histogram registration m of each subgraph1, m2...ms×s, the grey level histogram Total contact ratio M of two images is finally obtained as its similarity, and calculation method is as follows:
Note: fn(i) and gn(i) subgraph F is respectively representednWith subgraph GnGrey level histogram the i-th rank gray scale (ash of certain single order Degree can be obtained from histogram functions) number of pixels, max (fn(i),gn(i)) f is indicatedn(i) and gn(i) numerical value is larger in One.
After obtaining illegal vehicle, the illegal image of illegal vehicle can be extracted, then be gone forward side by side with license board information one Row storage, as its punishment on contravention of regulation evidence.Certainly, these evidences can be further provided to secondary auditor and assist really Recognize, reduces False Rate.
The present invention combines many algorithms and model, and this novel traffic offence detection method can satisfy city road Road traffic administration is for the accuracy rate of illegal detection and the demand of flexibility.
Below based on this method, explanation is shown to it in conjunction with specific embodiments.
Embodiment
In the present embodiment, for basic testing process as described in above-mentioned S1~S7, lower mask body shows its practical application Effect.
Step S1. framing signal lamp region and line interior lines exterior domain.
Signal lamp region, that is, position of the signal lamp in figure, signal lamp region is regular figure, by four ends counterclockwise Point indicates that line inner region is made of inside lane, and number of track-lines measurer body is determined by crossing, and line inside lane is divided into Through Lane, a left side It changes trains or buses and right-turn lane, lane is made of lane line, lane line is determined by two endpoints in figure.
Since camera position is fixed, signal lamp region and the lane line position in every picture in the picture frame collection K that transmits Set it is constant, so only need to once be positioned to signal lamp region, line inner region and line exterior domain.Signal lamp is obtained from picture Foundation of four endpoints in region as division signals lamp region, every lane line determine by two endpoints in picture, lane The figure that line is surrounded is lane region, carries out label to lane region.Lane line crosses stop line and extends to form virtual vehicle Road can also enter in the virtual lane when vehicle advances, and the virtual lane and line inner region lane of line exterior domain correspond.Cause This is divided into line inner region and line exterior domain to lane, and lane is divided into left-hand rotation, straight trip, right-hand rotation in the present embodiment, respectively corresponds left-hand rotation Signal lamp, straight trip signal lamp, right turn signal lamp.The picture that the camera of distinct device number transmits divides different picture frame collection K, The picture frame collection of same device number is operated every time.The first picture is chosen from a picture frame collection, is positioned manually from picture The position of camera and lane line, obtain
Signal lamp region S1=[(x1,y1),(x2,y2),(x3,y3),(x4,y4)]、
Line inner regionWith
Line exterior domain
Wherein, left indicates, and top indicates straight trip, and right indicates to turn right.
The case where for allowing straight trip and right-hand rotation or straight and turning left on a lane simultaneously, which exists Two.As shown in Fig. 2, right-lane line exterior domain includes two parts, line exterior domain are as follows:
Step S2. by the identification to traffic light color, realizes the category filter of present image collection first.
Fig. 3 is to set area-of-interest for the signal lamp region navigated to first to signal lamp color overhaul flow chart. It is affected because RGB picture is illuminated by the light, the rgb color space of picture is transformed into HSV space, then sentence from HSV space Break signal lamp color.
In the present embodiment, the color of traffic lights is judged according to the following conditions:
It is red: H ∈ [0,0.05] ∪ [0.9,1], S ∈ [0.05,1]
It is green: H ∈ [0.48,0.6], S ∈ [0.2,1]
It is yellow: H ∈ [0.05,0.2], S ∈ [0.05,1]
In picture frame collection K={ k1, k2, k3...kmIn will have the picture frame of red eye by shooting time sequencing protect It stays, the picture frame collection K '={ k that obtains that treated1, k2, k3...kn}。
Step S3. is to the judgement in signal lamp region and sets and includes:
Fig. 4 is the flow chart determined signal lamp.For the area of signal lamp: can set according to the actual situation, mistake Filtering surface product is excessive or crosses zonule.For arrowhead-shaped traffic lights, it is also necessary to further confirm that arrow direction.Arrowhead-shaped traffic Signal lamp has advance, left-hand rotation, three kinds of directions of turning right.Template library is established, the similitude of region to be matched and template is calculated, determines arrow Head direction.
If original image F size is M × N, template image p size is m × n, then original image is cut to be matched The matching degree of region q and its are as follows:
P (m, n), q (m, n) indicate that coordinate is the pixel value of (m, n) in template image and region to be matched.
Associated with corresponding lane according to the arrow direction of signal lamp: left-hand rotation arrow is corresponding with left turn lane；Straight trip arrow Head is corresponding with Through Lane；Right-hand rotation arrow is corresponding with right-turn lane.It is only corresponding with interception and analysis red eye after association Lane traveling state of vehicle, can largely reduce examination scope, improve inspection efficiency.
Step S4. divides picture frame collection.
By each frame image k in K 'iAfter being associated with lane, every corresponding image signals lamp is obtained no through traffic lane Number information.K ' is divided, successive image frame is all that no through traffic in a certain lane number if it exists, then is divided into a picture Collection, obtains new multiple groups pictures P={ P1,P2…}.Wherein i-th group of pictures Pi=[p1, p2 ..., ps, type], type table Show an identical red light direction, p1, p2 ... in signal lamp, ps is respectively the 1st, 2 ..., and s is opened with the direction type danger signal The picture frame of lamp arrow.To same signal lamp, there are two or more no through traffic signals, separately divide.Such as: original image Collection extract red eye after obtain K '=[k1, top, left], [k2, top, left], [k3, top, right], [k4, top,right],[k5,right]}；Left, top, right indicate no through traffic direction.3 pictures are obtained after k is divided {k1,k2,left},{k1,k2,k3,k4,top},{k3,k4,k5,right}；
Step S5: vehicle detection and Car license recognition in no through traffic lane.
Step S51. detection flow for the automobile uses convolutional neural networks model inspection vehicle；
Early period trains vehicle detection model and character recognition model: the convolutional Neural using YOLO model as vehicle detection Network model uses the data training convolutional neural networks marked.Character machining model is trained using the data of mark.When two A model training to result within an acceptable range when, by two models be applied to it is actually detected in.
One by one by pictures PiAs input, information of vehicles is successively detected.
Whether each frame image only needs to detect red light direction type in the image and corresponds to go out in associated no through traffic lane Existing vehicle.Camera acquired image is divided into the grid of s × s, if the center of a target falls into one of lattice Son, then the grid is responsible for detecting the vehicle in the region.Each region exports bounding box, there are five bounding box contains Information [x1,y1,x2,y2, confidence] wherein (x1,y1),(x2,y2) refer to the vehicle window left side that current grid is predicted The coordinate on upper vertex and bottom right vertex.Confidence indicates current window confidence score；It is lower than the window of threshold value to confidence level Mouth is excluded, minimum outsourcing rectangle of the highest window of window selection confidence level of coincidence as vehicle.
When Car license recognition, License Plate first can be carried out to vehicle window, use convolutional neural networks model orientation license plate Position obtains license plate window position information；Then the license plate in above-mentioned license plate window is identified using the character machining model of training Character obtains license plate number.
Step S52: car plate detection process: carrying out License Plate to vehicle window obtained in step S51, uses convolution mind Through network model positioning licence plate position, license plate window position information is obtained.Above-mentioned vehicle is identified using trained convolutional neural networks Board window characters on license plate, obtains license plate number.
Step S53: vehicle location obtained in step S51 and step S52 integrates license plate number information, obtains new Data information of vehicles:
C=[number, x1,y1,x2,y2]
Number indicates the license plate number recognized, x1,y1,x2,y2Indicate the vehicle window upper left corner, bottom right angular coordinate.
Due to pixel and vehicle window size, it is understood that there may be the window that some license plates cannot identify needs exist for complete Portion retains, and information of vehicles is denoted as:
C=[None, x1,y1,x2,y2]
Note: None expression fails correctly to identify license plate number.
Information of vehicles sequence: C={ c is obtained as a result,1,c2,c3...cn, ciFor the information of vehicles of i-th vehicle.
Step S6: to information of vehicles sequence C obtained in step S52 sequentially judge be in no through traffic line inner region or In no through traffic line exterior domain, more new data, data item: lane number is added.
It is to the method for calculating lane locating for vehicle below:
The ratio of vehicle body and line inner region, the area of line exterior domain and vehicle body area is calculated to determine vehicle in which vehicle Road, and belong to the line inner region or line exterior domain in lane；
Calculate overlap coefficient l:
L=coincide (sci,S2)/Area(sci)
sciIndicate that i-th vehicle vehicle body region is minimum outsourcing rectangular area, S in information of vehicles sequence2Indicate a certain The line inner region or line exterior domain in lane, coincide (a, b) indicate the area that zoning a and region b is overlapped；Area (sci) indicate i-th vehicle vehicle body minimum outsourcing rectangular area area；
It needs to set a threshold θ to be judged, when l is greater than given threshold value θ, determines vehicle in the line in corresponding lane Region, or determine vehicle in the line exterior domain in corresponding lane.To being not belonging to the information of vehicles of line inner region and line exterior domain not Give reservation.
Updated vehicle sequence is obtained as a result:
C '={ c '1,c′2,c′3...c′m}
Wherein i-th of information c in sequencei'=[cn, number, x1,y1,x2,y2], cn indicates the vehicle in lane locating for vehicle Taoist monastic name needs to distinguish line inner region lane or line exterior domain lane.
Step S7. is according to S6's as a result, to each pictures PiIn the illegal vehicle being accused of making a dash across the red light determined, respectively Two kinds of situations of S71 and S72.
S71: if in the pictures red light direction type correspond to it is associated no through traffic that lane only has single direction of traffic, The vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames is obtained, the first vehicle emptied in advance is put into In set, then the vehicle in the line exterior domain in the lane that appears in that no through traffic in all picture frames is obtained, is put into advance In the second vehicle set emptied；A vehicle first appears in the first vehicle set if it exists, then appears in the second vehicle again In set, then judge the vehicle for illegal vehicle.
Such as type is left in Fig. 2, that is, when forbidding vehicle to turn left:
1. first according to pictures no through traffic direction obtains no through traffic lane number and its line inner region, line outskirt Domain, set p, q, r are emptied.
2. pair first frame image, sequence p is added in vehicle of the detection in line inner region no through traffic lane；
3. detecting subsequent image frames:
3.1. vehicle of the detection in line inner region no through traffic lane saves new information of vehicles or updates existing vehicle Information p；
3.2. vehicle of the detection in line exterior domain no through traffic lane, if there are vehicles in line exterior domain no through traffic lane , information of vehicles is saved to set q；For the vehicle in set q, need to select to sentence according to whether it recognizes license board information Determine method, as follows respectively:
3.2.1 if present vehicle information is not sky in set q, it is compared one by one with the information of vehicles in sequence p, such as There are same vehicles for fruit, then determining the vehicle, there are current lanes in the case of no through traffic, and wired inner region drives to line outskirt The behavior in domain determines the vehicle illegal.
3.2.2 if a certain information of vehicles is sky in set q, it is subjected to similarity comparison with the vehicle in sequence p, such as It is more than threshold value that fruit, which exists with a certain vehicle similarity, then determines the vehicle once online inner region.In current lane, no through traffic In the case of, which leaves for the behavior of line exterior domain, determines the vehicle illegal.
S72: if red light direction type corresponds to associated no through traffic lane and has and keeps straight on and turn to two kinds in the pictures Direction of traffic then obtains the vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames, is put into advance In the first vehicle set emptied；The vehicle in the inside cord exterior domain in the lane that appears in that no through traffic in all picture frames is obtained again , it is put into the second vehicle set emptied in advance；The outer of the lane that appears in that no through traffic in all picture frames is obtained again Vehicle in side line exterior domain is put into the third vehicle set emptied in advance；Red light direction in pictures is judged, if red Lamp direction type is straight trip direction and there are a vehicles to first appear in the first vehicle set, then appears in third vehicle set In, then judge the vehicle for illegal vehicle；If red light direction type is turn direction and there are a vehicles to first appear at the first vehicle In set, then appears in the second vehicle set but do not appear in third vehicle set, then judge the vehicle to be illegal Vehicle.
Such as type is top in Fig. 2, that is, when forbidding straight trip, but the case where right lane can not only keep straight on but also can turn right.It is assumed that should Lane fare inner region Taoist monastic name is t, because there are the divisions of the lane line exterior domain of two kinds of travel situations to have in two parts, respectively Ct1 and ct2, ct2 are by near line inner region part.Forbid the judgement of straight trip are as follows: vehicle appears in ct1 and once occurs online Inner region t then determines in the case of currently forbidding straight trip that vehicle drives to line exterior domain ct1 by line inner region t, if vehicle goes out Present line exterior domain ct2 wouldn't be judged to making a dash across the red light, and whether detection subsequent pictures vehicle appears in ct1.But such as type in Fig. 2 Judgement when for right, i.e. no right turn are as follows: check that entire no right turn pictures, vehicle online inner region t occur, then protect Information of vehicles is deposited to p.There is online exterior domain lane ct2 and then saves information of vehicles to set q, if vehicle occurs outside line in vehicle Region lane ct1.Information of vehicles is then saved to set r；At the end of current image collection detects, for the vehicle in q, if in q Vehicle appears in set p but does not appear in set r, then determines that vehicle is turned right in q.
For illegal vehicle registration device number, time, information of vehicles, illegal information is obtained:
Bcar_list={ divce_id, time, cn, number, x1,y1,x2,y2}
In addition, needing to do further similarity comparison to the vehicle of unidentified license plate number that may be present in above-mentioned steps Process, Fig. 5 are the flow charts of vehicle similarity comparison in the present embodiment.For leading to vehicle pictures because illegal vehicle is too far It is fuzzy, cannot smoothly extract license board information situation, determine whether just to have had been taken using vehicle similarity comparison Vehicle is determined as same vehicle if vehicle similarity is more than threshold value.
In the present embodiment, vehicle similarity comparison process is as follows:
Differentiated by the registration of grey level histogram to compare two picture similarity degrees.But grey level histogram is from picture The global distribution of color can not compare the local distribution of color to compare, it is therefore desirable to picture is divided into the subgraph of S × S, point Its similarity is not calculated, is finally comprehensively considered.
Then, image F and image the G grid for being respectively divided into S × S are obtained into subgraph F1,F2...Fs×sAnd G1, G2...Gs×s, the grey level histogram of all subgraphs of two field pictures is calculated, corresponding function f is obtained1,f2...fs×sAnd g1, g2...gs×s, for the correspondence subgraph of image F and image G, calculate separately the grey level histogram registration m of each subgraph1, m2...ms×s, the grey level histogram Total contact ratio M of two images is finally obtained as its similarity, and calculation method is as follows:
Note: fn(i) and gn(i) subgraph F is respectively representednWith subgraph GnGrey level histogram the i-th rank gray scale (ash of certain single order Degree can be obtained from histogram functions) number of pixels, max (fn(i),gn(i)) f is indicatedn(i) and gn(i) numerical value is larger in One.
It by compared with given threshold value, judging whether two width pictures are similar, i.e., whether is same vehicle.For example same vehicle , known license board information is updated into final output sequence.
Below by taking certain city's traffic intersection as an example, the specific implementation of the above process is shown:
Fig. 6 is the picture frame that the camera that device number is 33036xxxx transmits.
Firstly, signal lamp region, line inner region, line exterior domain is positioned manually, Fig. 7 is the figure after these area visualizations Piece.Smaller figure is signal lamp region S in the first row in figure1.The crossing has altogether four lanes, and lane number is respectively 1,2, 3,4.The closed figure of bottom four is line inner region, and top is the line exterior domain in every lane, is respectively labeled as c1, c2, c3, C41, c42, wherein c1 is the line exterior domain of No. 1 left turn lane, and c2 is the line exterior domain of No. 2 Through Lanes, and c3 is No. 3 straight traffics The line exterior domain in road, c41 are No. 4 straight trip+right-turn lane out conductor exterior domains, and c42 is No. 4 straight trip+right-turn lane insides Line exterior domain.
It is then detected that the signal lamp color in every frame image, picture signal lamp has red in region, then protects picture frame It stays.Signal lamp direction is detected again, to red area uses template matching in signal lamp region in picture, obtains signal in exemplary diagram Lamp information is to forbid straight trip and no right turn.Subsequent continuous picture frame is divided, obtain one of pictures [p, Top], direction is to forbid keeping straight on.
Based on the pictures, the vehicle in image scene is identified by vehicle detection model, obtains in p first Picture information of vehicles sequence:
[[None,(1549,233,1699,316)],[None,(755,55,776,71)],[None,(726,51,745, ], 67) [None, (745,153,800,200)], [' Zhejiang XXXXXX', (629,696,877,946)], [None, (1168,260, 1259,325)]]
Fig. 8 is by the picture after the visualization of information of vehicles sequence.
After again, no through traffic direction (pictures are to forbid keeping straight on) is divided according to pictures, obtains what no through traffic Line inner region lane number is 2,3,4, and the line exterior domain lane that no through traffic is c2, c3, c41.Vehicle is judged according to overlapping area Locating region.The vehicle sequence information of Fig. 8 is determined, obtain set p=[2, ' Zhejiang CS0Q55', (629,696,877,946)], Q=[].
No. 2 lanes of Fig. 8 middle line inner region are there are vehicle ' Zhejiang XXXXXX', and no through traffic that lane does not have vehicle for line exterior domain. It then returns and detects subsequent image frames again, Fig. 9 is the second frame image in pictures.Vehicle sequence is detected in this picture:
P=[[c2, None, (651,491,819,653)]]
License board information fails to identify for some reason, into vehicle similarity comparison step:
The vehicle and vehicle in set q are compared one by one, obtain the vehicle and license plate number be ' Zhejiang XXXXXX' vehicle Similarity is greater than given threshold value, is judged as same vehicle.I.e. the vehicle is in forbidding straight trip pictures by online inner region, traveling To line exterior domain, [' Zhejiang XXXXXX', (651,491,819,653)] it is recorded as illegal vehicle, obtain illegal information [33036xxxx, c2,2019xxxx075807, ' Zhejiang XXXXXX', 629,696,877,946].
It then proceedes to detect subsequent pictures collection.
This method has the characteristics that high efficiency and high accuracy, overcomes traditional traffic lights road based on shooting and video recording The problem of the accuracy deficiency of the existing complexity and detection detected in mass data of detection mode is led in oral sex.We The color that the high efficiency of method embodies the signal lamp in the image formerly come to video camera transmission identifies, is green by signal lamp Picture exclude after, then based on vehicle detection model the vehicle in image is accurately positioned, and by it according to different Lane and vehicle are classified in line or outside line, logically determine whether the behavior of vehicle is illegal.Therefore, This method reduces the range of vehicle illegal detection, improves the efficiency of detection.The main body of the Accuracy and high efficiency of this method A large amount of people are reduced instead of original foundation eye recognition with the image-recognizing method of deep learning convolutional neural networks now Power loss.In addition for some line exterior domain vehicles that cannot effectively identify license board information due to hypertelorism, can pass through Similarity compares to identify its license board information, improves the accuracy of the illegal detection of traffic light intersection.
Above-mentioned embodiment is only a preferred solution of the present invention, so it is not intended to limiting the invention.Have The those of ordinary skill for closing technical field can also make various changes without departing from the spirit and scope of the present invention Change and modification.Therefore all mode technical solutions obtained for taking equivalent substitution or equivalent transformation, all fall within guarantor of the invention It protects in range.

Claims (10)

1. a kind of illegal detection method of making a dash across the red light based on deep learning, for detecting whether there are vehicles running red light, feature It is, steps are as follows:
S1: by the camera being installed on above traffic light intersection to be monitored, the video image for the traffic light intersection that takes a crane shot is obtained； It takes a frame to pre-process to appointing in video image, positions signal lamp region and several lanes in the camera scope of sight；
The signal lamp region is position of the signal lamp in video image, and every lane is all separated out outside line inner region and line Region；
The line inner region in every lane is the lane not yet across the holding areas of stop line, by two lane line conducts of two sides Boundary, according to guiding arrow mark determine every lane direction of traffic, direction of traffic be turn left, straight trip, turn right in extremely Few one kind；
The line exterior domain in every lane is across the crossing interior zone after stop line；The vehicle of direction of traffic single for every Road, at it across being equipped with a line exterior domain in the planning driving path after stop line；Have for every and keeps straight on and turn to two kinds of rows The lane in vehicle direction, is equipped with inside and outside two line exterior domains in its planning driving path, and inside cord exterior domain is located on the outside of stop line In straight trip and steering planning driving path overlapping region, out conductor exterior domain, which is located in straight trip planning driving path but is located at, turns to planning driving path It is external；
S2: in the detection process, to the video image that same camera takes, signal lamp color is carried out to continuous picture frame Identification, remains with the picture frame of red eye in order, the picture frame collection that obtains that treated；
S3: the every picture frame concentrated for picture frame detects the red light direction in its signal lamp region, then according to red light side To, wherein left-hand rotation arrow with left turn lane corresponding association associated with the corresponding lane that no through traffic, keep straight on arrow and straight traffic The corresponding association in road, the association corresponding with right-turn lane of right-hand rotation arrow, there are two types of the lane of direction of traffic and two kinds of direction of traffic pair for tool It the arrow answered while being associated with；
S4: dividing picture frame collection, and successive image frame all has unidirectional red light if it exists, then by these picture frames A pictures are divided into, several pictures are thus obtained, wherein i-th of pictures is denoted as Pi=[p1, p2 ..., ps, Type], type indicates an identical red light direction in signal lamp, and p1, p2 ..., ps is respectively the 1st, 2 ..., and s is opened with type The picture frame of direction red eye arrow；
S5: for each pictures Pi, vehicle identification, each frame are carried out to each frame image respectively by vehicle detection model Image only needs to detect red light direction type in the image and corresponds in associated no through traffic lane whether vehicle occur；For identification Vehicle out continues that its license plate is identified and recorded, is carried out license plate if it can not identify license plate in present image and do not known It does not mark；
S6: for each pictures Pi, the vehicle identified for each judges that it is located at locating lane, and judgement Its line inner region for being in the lane or line exterior domain；The vehicle of i.e. not online inner region also not online exterior domain is sieved It removes；
S7: according to the recognition result of S6, to each pictures PiIn the illegal vehicle being accused of making a dash across the red light determined:
S71: if in the pictures red light direction type correspond to it is associated no through traffic that lane only has single direction of traffic, obtain Vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames, is put into the first vehicle collection emptied in advance In conjunction, then the vehicle in the line exterior domain in the lane that appears in that no through traffic in all picture frames is obtained, is put into and empties in advance The second vehicle set in；A vehicle first appears in the first vehicle set if it exists, then appears in the second vehicle set again In, then judge the vehicle for illegal vehicle.
S72: if red light direction type corresponds to associated no through traffic lane and has and keeps straight on and turn to two kinds of drivings in the pictures Direction then obtains the vehicle in the line inner region in the lane that appears in that no through traffic in all picture frames, is put into and empties in advance The first vehicle set in；The vehicle in the inside cord exterior domain in the lane that appears in that no through traffic in all picture frames is obtained again, It is put into the second vehicle set emptied in advance；The out conductor in the lane that appears in that no through traffic in all picture frames is obtained again Vehicle in exterior domain is put into the third vehicle set emptied in advance；Red light direction in pictures is judged, if red light side To type be straight trip direction and there are a vehicles to first appear in the first vehicle set, is then appeared in third vehicle set, Then judge the vehicle for illegal vehicle；If red light direction type is turn direction and there are a vehicles to first appear at the first vehicle collection In conjunction, then appears in the second vehicle set but do not appear in third vehicle set, then judge the vehicle for illegal vehicle.
2. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In S1, the camera is fixing camera, and the visual angle of all picture frames and scope of sight are solid in the video image obtained It is fixed constant.
3. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In S2, signal lamp color is carried out to picture frame and knows method for distinguishing are as follows: sets signal lamp region to first in picture frame to feel emerging Then the rgb color space of area-of-interest is transformed into HSV space, then judges signal lamp color from HSV space by interesting region.
4. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the signal Lamp is arrowhead-shaped traffic lights, there is advance, left-hand rotation, three kinds of directions of turning right；The detection side in red light direction in the step S3 Method are as follows: the template library of different directions arrow is established, the similitude that color in signal lamp region is red region and template is calculated, So that it is determined that arrow direction.
5. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In S5, using convolutional neural networks model inspection vehicle, when detection, divides an image into the grid of s × s, if target Center falls into one of grid, then the grid is responsible for detecting the vehicle in the region；The window area output each detected Bounding box, bounding box is containing there are five information [x1,y1,x2,y2, confidence] wherein (x1,y1),(x2,y2) Refer to the coordinate for the vehicle window left upper apex and bottom right vertex that current grid is predicted.Confidence indicates current window Confidence score；Window to confidence level lower than threshold value excludes, the highest window conduct of the window selection confidence level of coincidence The minimum outsourcing rectangle of vehicle.
6. the illegal detection method of making a dash across the red light based on deep learning as claimed in claim 5, which is characterized in that the convolution Neural network model uses YOLO model.
7. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In S5, the recognition methods of license plate are as follows: License Plate is carried out to vehicle window, uses convolutional neural networks model inspection vehicle figure Piece obtains license plate window position information；Then the license plate word in above-mentioned license plate window is identified using the convolutional neural networks of training Symbol, obtains license plate number.
8. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In 6, the recognition methods in lane locating for vehicle are as follows: calculate the minimum outsourcing rectangle of vehicle and line inner region, line exterior domain is overlapped Area accounts for the ratio of vehicle minimum outsourcing rectangular area according to overlapping area, to determine lane locating for vehicle and be in be somebody's turn to do The line inner region or line exterior domain in lane.
9. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the S7 In, judge a vehicle whether and meanwhile appear in the method in two vehicle set are as follows:
If a vehicle has license board information in the second vehicle set or third vehicle set, directly by the license plate of the vehicle Information is compared with the license board information of vehicle in the first vehicle set, shows the vehicle when identical while appearing in two set In；
If there is the vehicle with the unidentified label of license plate in the second vehicle set or third vehicle set, by itself and the first vehicle Set in vehicle carry out vehicle similarity comparison one by one, when similarity be more than threshold value when judge this two cars for same vehicle , the license board information for obtaining the similar vehicle being located in the first vehicle set is assigned to the vehicle of unidentified license plate.
10. the illegal detection method of making a dash across the red light based on deep learning as described in claim 1, which is characterized in that the step In rapid S7, vehicle similarity comparison method are as follows:
The two image F compared will be participated in first and image G zooms to identical size；
Then, image F and image the G grid for being respectively divided into S × S are obtained into subgraph F1,F2...Fs×sAnd G1,G2...Gs×s, meter The grey level histogram for calculating all subgraphs of two field pictures, obtains corresponding function f1,f2...fs×sAnd g1,g2...gs×s, for image The correspondence subgraph of F and image G calculates separately the grey level histogram registration m of each subgraph1,m2...ms×s, finally obtain two For the grey level histogram Total contact ratio M of image as its similarity, calculation method is as follows:
Note: fn(i) and gn(i) subgraph F is respectively representednWith subgraph GnGrey level histogram the i-th rank gray scale number of pixels, max (fn (i),gn(i)) f is indicatedn(i) and gn(i) biggish one of numerical value in.
CN201910459288.1A 2019-05-29 2019-05-29 Deep learning-based red light violation detection method Active CN110197589B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910459288.1A CN110197589B (en) 2019-05-29 2019-05-29 Deep learning-based red light violation detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910459288.1A CN110197589B (en) 2019-05-29 2019-05-29 Deep learning-based red light violation detection method

Publications (2)

Publication Number Publication Date
CN110197589A true CN110197589A (en) 2019-09-03
CN110197589B CN110197589B (en) 2020-09-01

Family

ID=67753284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910459288.1A Active CN110197589B (en) 2019-05-29 2019-05-29 Deep learning-based red light violation detection method

Country Status (1)

Country Link
CN (1) CN110197589B (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110459063A (en) * 2019-09-11 2019-11-15 驭势（上海）汽车科技有限公司 A kind of determination method of running red light for vehicle, electronic equipment and storage medium
CN110991221A (en) * 2019-10-16 2020-04-10 合肥湛达智能科技有限公司 Dynamic traffic red light running identification method based on deep learning
CN111008609A (en) * 2019-12-16 2020-04-14 北京迈格威科技有限公司 Traffic light and lane matching method and device and electronic equipment
CN111160183A (en) * 2019-12-20 2020-05-15 华瑞新智科技（北京）有限公司 Method and device for detecting red light running of vehicle
CN111311601A (en) * 2020-03-26 2020-06-19 深圳极视角科技有限公司 Segmentation method and device for spliced image
CN111325988A (en) * 2020-03-10 2020-06-23 北京以萨技术股份有限公司 Real-time red light running detection method, device and system based on video and storage medium
CN111368774A (en) * 2020-03-12 2020-07-03 北京以萨技术股份有限公司 Waste film rollback method, system, terminal and medium based on traffic violation image
CN111462499A (en) * 2020-03-26 2020-07-28 深圳极视角科技有限公司 Method and device for detecting traffic violation
CN111583661A (en) * 2020-05-27 2020-08-25 魏保安 Vehicle red light running detection method based on gun-ball linkage and DNN
CN112016735A (en) * 2020-07-17 2020-12-01 厦门大学 Patrol route planning method and system based on traffic violation hotspot prediction and readable storage medium
CN112200148A (en) * 2020-11-06 2021-01-08 中国计量大学 Decentralized traffic integrity evaluation system
CN112767710A (en) * 2021-01-20 2021-05-07 青岛以萨数据技术有限公司 Vehicle illegal behavior detection method and device and storage medium
CN112863194A (en) * 2021-01-20 2021-05-28 青岛以萨数据技术有限公司 Image processing method, device, terminal and medium
CN113034587A (en) * 2019-12-25 2021-06-25 沈阳美行科技有限公司 Vehicle positioning method and device, computer equipment and storage medium
CN113392762A (en) * 2021-06-15 2021-09-14 北京纵目安驰智能科技有限公司 Intersection detection method, system, terminal and computer readable storage medium
CN113761967A (en) * 2020-06-01 2021-12-07 中移(苏州)软件技术有限公司 Identification method and device
CN115752441A (en) * 2022-11-25 2023-03-07 重庆长安汽车股份有限公司 Traffic light construction method of high-precision map
US12112553B2 (en) 2020-11-09 2024-10-08 Hayden Ai Technologies, Inc. Lane violation detection using convolutional neural networks

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003091965A1 (en) * 2002-04-15 2003-11-06 Gatsometer B.V. Method and device for controlling a red light camera
KR100785757B1 (en) * 2007-01-17 2007-12-18 주식회사 진성아이엔티 System and method for automatically monitoring illegally-stopping/parking vehicle using fixed camera
CN105975941A (en) * 2016-05-31 2016-09-28 电子科技大学 Multidirectional vehicle model detection recognition system based on deep learning
CN106886755A (en) * 2017-01-19 2017-06-23 北京航空航天大学 A kind of intersection vehicles system for detecting regulation violation based on Traffic Sign Recognition
US9759812B2 (en) * 2014-10-02 2017-09-12 Trimble Inc. System and methods for intersection positioning
CN207020820U (en) * 2017-07-14 2018-02-16 深圳云天励飞技术有限公司 Red-running Monitor System
CN108847032A (en) * 2018-08-21 2018-11-20 北京深瞐科技有限公司 A kind of traffic violation recognition methods and device
CN109409421A (en) * 2018-10-09 2019-03-01 杭州诚道科技股份有限公司 Motor vehicle, driver's archival image recognition methods based on convolutional neural networks

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003091965A1 (en) * 2002-04-15 2003-11-06 Gatsometer B.V. Method and device for controlling a red light camera
KR100785757B1 (en) * 2007-01-17 2007-12-18 주식회사 진성아이엔티 System and method for automatically monitoring illegally-stopping/parking vehicle using fixed camera
US9759812B2 (en) * 2014-10-02 2017-09-12 Trimble Inc. System and methods for intersection positioning
CN105975941A (en) * 2016-05-31 2016-09-28 电子科技大学 Multidirectional vehicle model detection recognition system based on deep learning
CN106886755A (en) * 2017-01-19 2017-06-23 北京航空航天大学 A kind of intersection vehicles system for detecting regulation violation based on Traffic Sign Recognition
CN207020820U (en) * 2017-07-14 2018-02-16 深圳云天励飞技术有限公司 Red-running Monitor System
CN108847032A (en) * 2018-08-21 2018-11-20 北京深瞐科技有限公司 A kind of traffic violation recognition methods and device
CN109409421A (en) * 2018-10-09 2019-03-01 杭州诚道科技股份有限公司 Motor vehicle, driver's archival image recognition methods based on convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
吴玉枝 等: "基于卷积神经网络的违章停车事件检测", 《现代计算机》 *
钟海波: "基于深度学习的人脸识别技术在智慧安防建设中的应用", 《广东公安科技》 *

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110459063A (en) * 2019-09-11 2019-11-15 驭势（上海）汽车科技有限公司 A kind of determination method of running red light for vehicle, electronic equipment and storage medium
CN110991221B (en) * 2019-10-16 2024-02-27 合肥湛达智能科技有限公司 Dynamic traffic red light running recognition method based on deep learning
CN110991221A (en) * 2019-10-16 2020-04-10 合肥湛达智能科技有限公司 Dynamic traffic red light running identification method based on deep learning
CN111008609A (en) * 2019-12-16 2020-04-14 北京迈格威科技有限公司 Traffic light and lane matching method and device and electronic equipment
CN111008609B (en) * 2019-12-16 2023-05-19 北京迈格威科技有限公司 Traffic light and lane matching method and device and electronic equipment
CN111160183A (en) * 2019-12-20 2020-05-15 华瑞新智科技（北京）有限公司 Method and device for detecting red light running of vehicle
CN113034587B (en) * 2019-12-25 2023-06-16 沈阳美行科技股份有限公司 Vehicle positioning method, device, computer equipment and storage medium
CN113034587A (en) * 2019-12-25 2021-06-25 沈阳美行科技有限公司 Vehicle positioning method and device, computer equipment and storage medium
CN111325988A (en) * 2020-03-10 2020-06-23 北京以萨技术股份有限公司 Real-time red light running detection method, device and system based on video and storage medium
CN111368774A (en) * 2020-03-12 2020-07-03 北京以萨技术股份有限公司 Waste film rollback method, system, terminal and medium based on traffic violation image
CN111462499A (en) * 2020-03-26 2020-07-28 深圳极视角科技有限公司 Method and device for detecting traffic violation
CN111311601B (en) * 2020-03-26 2023-10-13 山东极视角科技股份有限公司 Segmentation method and device for spliced image
CN111311601A (en) * 2020-03-26 2020-06-19 深圳极视角科技有限公司 Segmentation method and device for spliced image
CN111583661A (en) * 2020-05-27 2020-08-25 魏保安 Vehicle red light running detection method based on gun-ball linkage and DNN
CN113761967A (en) * 2020-06-01 2021-12-07 中移(苏州)软件技术有限公司 Identification method and device
CN112016735B (en) * 2020-07-17 2023-03-28 厦门大学 Patrol route planning method and system based on traffic violation hotspot prediction and readable storage medium
CN112016735A (en) * 2020-07-17 2020-12-01 厦门大学 Patrol route planning method and system based on traffic violation hotspot prediction and readable storage medium
CN112200148A (en) * 2020-11-06 2021-01-08 中国计量大学 Decentralized traffic integrity evaluation system
CN112200148B (en) * 2020-11-06 2024-01-26 中国计量大学 Decentralizing traffic integrity evaluation system
US12112553B2 (en) 2020-11-09 2024-10-08 Hayden Ai Technologies, Inc. Lane violation detection using convolutional neural networks
CN112863194B (en) * 2021-01-20 2022-08-23 青岛以萨数据技术有限公司 Image processing method, device, terminal and medium
CN112863194A (en) * 2021-01-20 2021-05-28 青岛以萨数据技术有限公司 Image processing method, device, terminal and medium
CN112767710A (en) * 2021-01-20 2021-05-07 青岛以萨数据技术有限公司 Vehicle illegal behavior detection method and device and storage medium
CN113392762A (en) * 2021-06-15 2021-09-14 北京纵目安驰智能科技有限公司 Intersection detection method, system, terminal and computer readable storage medium
CN113392762B (en) * 2021-06-15 2024-04-26 北京纵目安驰智能科技有限公司 Intersection detection method, system, terminal and computer readable storage medium
CN115752441A (en) * 2022-11-25 2023-03-07 重庆长安汽车股份有限公司 Traffic light construction method of high-precision map

Also Published As

Publication number Publication date
CN110197589B (en) 2020-09-01

Similar Documents

Publication Publication Date Title
CN110197589A (en) A kind of illegal detection method of making a dash across the red light based on deep learning
CN106652468B (en) The detection and from vehicle violation early warning alarm set and method in violation of rules and regulations of road vehicle front truck
CN109670376B (en) Lane line identification method and system
WO2020000251A1 (en) Method for identifying video involving violation at intersection based on coordinated relay of video cameras
CN104751634B (en) The integrated application method of freeway tunnel driving image acquisition information
CN106599792B (en) Method for detecting hand driving violation behavior
KR102122859B1 (en) Method for tracking multi target in traffic image-monitoring-system
CN103824452B (en) A kind of peccancy parking detector based on panoramic vision of lightweight
CN104537841B (en) Unlicensed vehicle violation detection method and detection system thereof
CN110717433A (en) Deep learning-based traffic violation analysis method and device
CN103065470B (en) Detection device for behaviors of running red light of vehicle based on machine vision with single eye and multiple detection faces
CN101929867B (en) Clear path detection using road model
CN110689724B (en) Automatic motor vehicle zebra crossing present pedestrian auditing method based on deep learning
JP2917661B2 (en) Traffic flow measurement processing method and device
CN107978165A (en) Intersection identifier marking and signal lamp Intellisense method based on computer vision
CN109948416A (en) A kind of illegal occupancy bus zone automatic auditing method based on deep learning
CN109949579A (en) A kind of illegal automatic auditing method that makes a dash across the red light based on deep learning
CN105844257A (en) Early warning system based on machine vision driving-in-fog road denoter missing and early warning method
CN105654073B (en) A kind of speed automatic control method of view-based access control model detection
CN101944176A (en) Exist the more excellent clear path of means of transportation sign to detect
CN105825696A (en) Driving assistance system based on signal lamp information prompts
CN102521983A (en) Vehicle violation detection system based on high definition video technology and method thereof
CN202472943U (en) Vehicle violation detecting system based on high definition video technology
CN117809275B (en) Environment sensing method and system based on 360-degree circular system of civil aviation vehicle
CN110188606A (en) Lane recognition method, device and electronic equipment based on high light spectrum image-forming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant