CN100565555C - Peccancy parking detector based on computer vision - Google PatentsPeccancy parking detector based on computer vision Download PDF
- Publication number
- CN100565555C CN100565555C CNB2007101644805A CN200710164480A CN100565555C CN 100565555 C CN100565555 C CN 100565555C CN B2007101644805 A CNB2007101644805 A CN B2007101644805A CN 200710164480 A CN200710164480 A CN 200710164480A CN 100565555 C CN100565555 C CN 100565555C
- Prior art keywords
- vision sensor
- Prior art date
- 235000017899 Spathodea campanulata Nutrition 0.000 claims abstract description 36
- 240000004317 Spathodea campanulata Species 0.000 claims description 33
- 238000003384 imaging method Methods 0.000 claims description 29
- 238000000034 methods Methods 0.000 claims description 23
- 230000000875 corresponding Effects 0.000 claims description 19
- 239000011651 chromium Substances 0.000 claims description 16
- 239000010955 niobium Substances 0.000 claims description 16
- 239000000284 extracts Substances 0.000 claims description 13
- 238000009826 distribution Methods 0.000 claims description 12
- 238000006243 chemical reactions Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 10
- 238000003708 edge detection Methods 0.000 claims description 8
- 230000001537 neural Effects 0.000 claims description 7
- 230000003044 adaptive Effects 0.000 claims description 6
- 230000001808 coupling Effects 0.000 claims description 6
- 238000010168 coupling process Methods 0.000 claims description 6
- 238000005859 coupling reactions Methods 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 241001157067 Leucoagaricus meleagris Species 0.000 claims description 4
- 239000004744 fabrics Substances 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 238000002156 mixing Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 206010034719 Personality change Diseases 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000000295 complement Effects 0.000 claims description 3
- 239000000203 mixtures Substances 0.000 claims description 3
- 238000003892 spreading Methods 0.000 claims description 3
- 239000000725 suspensions Substances 0.000 claims description 3
- 238000003379 elimination reactions Methods 0.000 claims description 2
- 239000000463 materials Substances 0.000 claims description 2
- 238000003032 molecular docking Methods 0.000 claims description 2
- 240000003344 Silene chalcedonica Species 0.000 abstract 3
- 239000000306 components Substances 0.000 description 14
- 230000000007 visual effect Effects 0.000 description 13
- 238000005516 engineering processes Methods 0.000 description 10
- 239000003795 chemical substances by application Substances 0.000 description 7
- 238000010586 diagrams Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000004088 simulation Methods 0.000 description 4
- 238000000151 deposition Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 239000011521 glasses Substances 0.000 description 2
- 230000011514 reflex Effects 0.000 description 2
- 230000001755 vocal Effects 0.000 description 2
- 241000985665 Cecropia obtusifolia Species 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- 240000007594 Oryza sativa Species 0.000 description 1
- 235000007164 Oryza sativa Nutrition 0.000 description 1
- 208000008425 Protein Deficiency Diseases 0.000 description 1
- 210000002356 Skeleton Anatomy 0.000 description 1
- 238000004458 analytical methods Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 239000008358 core components Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002708 enhancing Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000010410 layers Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 101710062303 moon Proteins 0.000 description 1
- 230000000877 morphologic Effects 0.000 description 1
- 230000000414 obstructive Effects 0.000 description 1
- 230000003287 optical Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001105 regulatory Effects 0.000 description 1
- 239000011347 resins Substances 0.000 description 1
- 229920005989 resins Polymers 0.000 description 1
- 235000009566 rice Nutrition 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001131 transforming Effects 0.000 description 1
The invention belongs to omnibearing vision sensor technology, computer vision technique, image recognition technology and the network communications technology and relate to, especially a kind of peccancy parking detector in the application of parking offense context of detection.
Along with the fast development of Chinese national economy, the surge of motor vehicle quantity, caused transport need increase too fast and cause such as a series of problems such as traffic jams, wherein motor vehicle illegal parking phenomenon is a key factor that causes obstruction to traffic.
According to the investigation of traffic control department, the motor vehicle illegal parking mainly contains five harm greatly: the one, and illegal parking becomes traffic to gather around the resistance source.The 2nd, illegal parking becomes scratch traffic hazard chief culprit.According to statistics, the vehicle scratch accident that causes because of illegal parking accounts for 38% of this class accident.The 3rd, illegal parking becomes stolen target.The 4th, illegal parking becomes disorderly to blow a trumpet and the retrograde inducement of vehicle.The 5th, illegal parking becomes the walkway obstacle that passes through.
About illegal parking being punished the law that relates to mainly is " law on road traffic safety " implemented " administrative penalties law " and in May, 2004." law on road traffic safety " has three clauses that illegal parking is stipulated.It is as follows to make a summary: 56 " motor vehicle should be parked in the regulation place.Forbid on the walkway, parking motor vehicle; But, execute except the parking position of drawing according to the 33 regulation of this law.Temporary parking on road must not hinder other vehicles and pedestrians." 33 " in the urban road scope, under the situation that does not influence pedestrian, vehicle pass-through, relevant government department can execute draws a parking position." 93 " to violating law on road traffic safety rule, rules about motor vehicle parking, temporary parking regulation, can point out illegal activities, and give verbal warning, make it sail out of immediately.Though vehicle driver not at the scene or at the scene refusal sail out of immediately, hinder other vehicles, walk, locate fine below 200 yuan more than 20 yuan.
Present processing mode is, the illegal temporary parking of motor vehicle, and traffic police or the personnel of municipal administration will point out illegal in accordance with the law and give verbal warning, order the illegal parking people to be sailed out of immediately; Illegal parking people refusal sails out of immediately or refuses and sails out of, and the traffic police can punish at the scene; The illegal temporary parking driver of motor vehicle not at the scene, the traffic police will take to shoot with video-corder the evidence obtaining mode and punish, law enforcement is before the traffic police leaves the scene of shooting with video-corder, the illegal parking people returns and approves fact of malfeasance, is ready the acceptance of punishment, the traffic police can punish then and there; The traffic police does not take to shoot with video-corder the evidence obtaining mode when punishing at the scene to illegal temporary parking people, should fill in " illegal parking processing advice note " folds up under motor vehicle windshield glass wiper, select the suitable angle shot display machines motor-car trade mark, be placed " illegal parking processing advice note ", the illegal place of parking prohibits and stops sign or can the clear and definite illegal picture prints such as the significant object in place, shooting date and time of parking, and should be in time hand over Traffic Warden Subteam to preserve and implement punishment to the illegal parking involved party in view of the above taking the evidence obtaining data; Before the traffic police shot with video-corder evidence obtaining data friendship Traffic Warden Subteam, the illegal parking involved party had arrived Traffic Warden Subteam's acceptance of punishment and without demur to fact of malfeasance, can punish according to " illegal parking processing advice note "; The illegal parking involved party has objection to the traffic offence fact, and traffic control department should inform after the litigant waits to shoot with video-corder the data input system and accept the illegal activities processing again.The illegal parking punishment criteria is: according to " law on road traffic safety " regulation, the illegal motor vehicle of parking is suitable for non-at-scene punishment, and everyone locates 200 yuan of fine and remembers 2 fens to driver or motor vehicle; Illegal temporary parking, the driver refuses to sail out of, and applicable on-the-spot punishment is located 200 yuan of fine and is remembered 2 fens the driver; Everyone is warned or is located 20 yuan of fine to bicycle illegal parking involved party or vehicle.
But present parking offense is handled law enforcement and is existed some disadvantages at technical elements, has caused the opposition between government's law enforcement and the masses' illegal activities, has produced being discord of society, has caused showing great attention to of society." purpose of traffic administration is in order to guide citizen to observe traffic rules and regulations, and should be auxilliary to criticize and educate, to punish, and can not manage on behalf of another to penalize, and assigns fine as the extra earning means in some people's proposition." some people's proposition should be to criticize and educate and warning to first slight violator again; To for the second time slight violator, can attempt imposing a fine half; To slight violator for the third time, fine in full again." allowing the driver experience the law-executor is in order to safeguard traffic order really, rather than for extra earning." but to realize that this target needs the support of new and high technology means to a great extent.
At first law-executor and parking lawbreaker's main body all is the people, monitor that facing to the law-executor whether parking offense is arranged is not a nothing the matter 24 hours every days, to make to parking lawbreaker result genuinely convinced, oral during law-executor's illegal parking is handled in addition, processing mode according to this hommization needs the law-executor to make great efforts greatly to tend to surpass law-executor's working limit with patient; On the other hand, the pained money of parking lawbreaker is that main is this law enforcement mode that does not understand the traffic police on the one hand.Many people think that there is the many places breach of procedural law in law enfrocement official's law enforcement behavior: one, law is not forbidden, allows, and oneself place of parking cars does not significantly prohibit and stops indicating the no mistake of oneself stopping there; Two, disorderly to stop even oneself belong to, law enforcement agency also should handle by legal procedure in the punishment process.It is caused when the focus of the problems referred to above is a certain side omissions of law enfrocement official and the litigant that is punished; And each parking offense incident requires law-executor and parking lawbreaker all at the scene when taking place, and is difficult in the operation realize.Here just need a middle-agent, make law-executor and the information symmetry of attempting the parking offense between the person.The middle-agent can inform in time that this place of driver can not park cars, and makes the driver that right to know be arranged when the person that attempts the parking offense wants to park cars; The middle-agent continues to monitor, if detecting this is apprised of the driver and still ignores and advise and to carry out parking offense, the middle-agent just collects evidence, so that punish according to the fact back, simultaneously notify the law enfrocement official to handle as early as possible detecting in the very first time of parking offense the diverse network means, because the parking offense evidence has been arranged, even the law enfrocement official is not under at the scene the situation, also can safeguard traffic order by the middle-agent, can ensure to the driver who is punished simultaneously on technological means has statement and explaination chance.
Summary of the invention
Mainly rely on when overcoming present processing parking offense artificial, working strength is big, sensing range is less, inefficient deficiency, the invention provides a kind of working strength that reduces the parking offense law enfrocement official, realize the peccancy parking detector that real time automatic detection, detection orientation are wide, efficient is high of parking offense based on computer vision.
The technical solution adopted for the present invention to solve the technical problems is:
A kind of peccancy parking detector based on computer vision, the omnibearing vision sensor that comprises the video image that is used to obtain guarded region, be used to capture the fire ball video camera of the detailed local message of parking offense vehicle and be used for video image understood and analyze and carry out the microprocessor that parking offense detects, described omnibearing vision sensor is connected with microprocessor by video card with the fire ball video camera, described omnibearing vision sensor comprises the evagination catadioptric minute surface that is used for reflecting monitoring field object, evagination catadioptric minute surface down, be used to support and protect the transparent round body of evagination catadioptric minute surface, be used to take the camera of imaging body on the evagination mirror surface, camera faces toward the evagination mirror surface up, and described camera is positioned at the virtual focus of described evagination catadioptric minute surface; Described microprocessor comprises:
The vedio data read module is used to read the video image information of coming from the omnibearing vision sensor biography; Parking offense monitoring field customized module is used for the monitoring range that custom video is monitored vehicles peccancy;
The omnibearing vision sensor demarcating module is used to set up a kind of material picture of space and the corresponding relation of the video image that is obtained;
The video data Fusion Module, be used to control the rotation and the focusing of fire ball vision sensor, make the fire ball vision sensor can aim at the vehicle of following the tracks of and carry out the feature candid photograph, the panoramic picture center that obtains with omnibearing vision sensor is the center of circle, according to differentiating needs panoramic picture is divided into annulus, according to the parameters such as height, fire ball vision sensor locus and camera focal length of omnibearing vision sensor, determine regional required level, vertical angle and the focal length that rotates that the fire ball vision sensor will detect successively apart from monitoring scene ground;
Vehicle enters the monitoring range detection module, is used for after detecting the monitoring range that vehicle enters customization, and system produces an incident automatically, and system is every all can call corresponding processing module when producing an incident automatically; Vehicle ID number and deposit the automatically-generating module of following the tracks of the vehicle image file, the vehicle object that is used for just entering the monitoring range of customization is named, and generates a file with ID number name of this vehicle simultaneously, is used to deposit the close-up image of this vehicle;
The multiple target tracking module is used to follow the tracks of the vehicle object that enters in the monitoring field;
The parking offense detection module is used to detect the vehicle object that has been parked in the monitoring range, includes:
Edge detection unit, be used to adopt method of differential operator, image differentiated try to achieve gradient and carry out rim detection, from marginal point often corresponding to the big point of single order differential amplitude, while also sets out corresponding to the zero cross point of second-order differential, set single order or second-order differential operator and try to achieve its gradient or second derivative zero crossing, select threshold value to extract the border again;
The image pretreatment unit is used for image is carried out edge extraction, binaryzation, profile is linked to be closed curve and the curve inner region is carried out the white pixel filling;
The template matches unit is used for the model of the car body that will will detect and the object of all positions of monitoring scene and compares, and investigates the object that whether exists the model with car body to be complementary; Turn to the image of M*N for a spatial spreading, if i, j is each pixel coordinate in the monitoring scene of discretize, and (i is (i for coordinate j) to f, the gray-scale value of pixel j), according to the notion and the acquiring method of above-mentioned invariant moments, according to the disposal route of the digital picture of discretize, then " central moment " of object can be approached by following dual summation form in the image scene, represent with formula (23)
Carry out normalized, can get normalization permanent center square and be: η Pq=D Pq/ D 00, r=(p+q)/2+1;
If p+q＜2 are then derived two constant central moment functions of RST and are represented with formula (24),
The object of monitoring scene and the center invariant moments of template are compared, and get the phase difference, tentatively think the object that template is represented less than threshold value; The centre of form with template overlaps with the centre of form coordinate of object in the scene then;
Utilize the rotational transform of coordinate to determine the anglec of rotation of object correspondence in the scene, the algorithm that convergent-divergent and rotation are adopted is respectively:
The convergent-divergent algorithm:
x 2＝N(x 1-x 0)+x 0
y 2＝N(y 1-y 0)+y 0
The rotation algorithm:
x 2＝(x 1-x 0)cosθ+(y 1-y 0)sinθ+x 0
y 2＝(y 1-y 0)cosθ+(x 1-x 0)sinθ+y 0
More than in two groups of formulas the edge contour point to template carry out convergent-divergent and rotation processing, also need link afterwards and the closed curve interior pixel is filled (x in the formula 2, y 2) be the new coordinate of pixel; (x 1, y 1) be former coordinate; (x 0, y 0) be the centre of form coordinate of object; N is a zoom factor; θ is the angle of rotation;
The bianry image that above-mentioned two width of cloth have been aimed at carries out exclusive-OR operation, compares with residual pixel and preset threshold value, as less than preset threshold value, judges that object is the type of car body shown in the template and vehicle;
Module is captured in vehicle general image location, is used to locate the general image of capturing vehicle;
Module is captured in vehicle license plate location, be used for positioned vehicle this vehicle of location suspension car plate position and this position is captured the image of car plate;
The car plate identification module is used to discern the license plate number of vehicles peccancy;
Caveat generates and broadcast unit, is used to warn the parking offense driver and does not want parking offense;
Record generation unit violating the regulations is used for surpassing the time of setting after caution, and vehicle is in the parking offense position, generates a record violating the regulations automatically, and the image and the number-plate number of storage vehicles peccancy.
As preferred a kind of scheme: described microprocessor also comprises: testing result affirmation, change, completion module are used to confirm whether license plate number identification is correct, recognition result, the completion of change identification error not have the license plate number discerned.
As preferred another scheme: described microprocessor also comprises: network transmission module is used for the video image of detected vehicle peccancy docking process, the vehicles peccancy general image of candid photograph and relevant record violating the regulations are sent to law enforcement agency by network.
As preferred another kind of scheme: described microprocessor also comprises: the real-time play module is used for the associated video image of detected parking offense incident, captures image and be played to display device by this module.
Further, in described multiple target tracking module, will in the monitoring scene pixel of prospect vehicle be extracted, includes:
The adaptive background reduction unit, be used to adopt adaptive background elimination algorithm based on mixture gaussian modelling, detect at the brightness value Y component in the YCrCb color space of image, each picture point has been adopted the hybrid representation of a plurality of Gauss models, if be used for describing total K of each Gaussian distribution of putting color distribution, be labeled as (27) respectively:
η(Y t，μ t，i，∑ t，i)，i＝1，2，3…，k (27)
Subscript t express time in the formula (27);
Each Gaussian distribution has different weights and priority respectively, again with K background model according to priority order ordering from high to low, get suitable surely background model weights and threshold value, when detecting the foreground point, according to priority order with Y tMate one by one with each Gaussian distribution model, if coupling is judged that then this point may be the foreground point, otherwise is the foreground point; If certain Gaussian distribution and Y tCoupling is then upgraded by the turnover rate of setting weights and Gauss's parameter of this Gaussian distribution;
Shade suppresses the unit, be used for handling the shadow region of the resulting foreground target of adaptive background reduction unit, learn the color component CrCb and the luminance component Y on the ground of monitoring scene earlier, when running into the foreground point, whether the color component CrCb that judges this point is close with the ground of monitoring scene, whether luminance component Y is lower than the ground of monitoring scene, and evaluation algorithm is represented by formula (28):
The point of mark 0 belongs to shade in the formula (28), the point of mark 1 belongs to prospect, abs represents to ask its absolute value, Cr is the Cr color component of this point, Cb is the Cb color component of this point, RoadCr represents the Cr color component of guarded region, and RoadCb is the Cb color component of road, and threshold represents the threshold value that is provided with;
The connected region identify unit is used to adopt eight connected region extraction algorithms to obtain the size and the shape information of vehicle;
The vehicle subject tracking unit, be used for after monitoring scene extracts prospect vehicle object, adopt based target color characteristic track algorithm, utilize the color characteristic of vehicle target object in video image, to find the position and the size at vehicle target object place, in the next frame video image, with vehicle target current position and big or small initialization search window, repeat this process and realize Continuous Tracking vehicle target.
Further again, described car plate identification module comprises: the license plate image pretreatment unit, and be used for that original image is carried out various zones and handle, at first license plate image is being carried out grey level stretching, adopt the global threshold method that image is carried out binary conversion treatment, then it is adopted medium filtering;
The car plate positioning unit, be used for car plate being carried out level and vertical projection location car plate at the view picture license plate image, according to the licence plate feature rough detection is carried out in the place that might have licence plate in the vehicle general image, if discovery has similar licence plate then positions, otherwise requires system to capture vehicle image again;
The characters on license plate cutting unit is used for characters on license plate is divided into single character, adopts character vertical projection histogram to combine with characters on license plate priori width information and cuts apart;
The normalized unit is used for vehicle license plate characteristic and extracts and identification, is unified format size with character change, adopts the method for normalizing of neighbor interpolation;
Extract feature unit, be used for the character recognition in later stage, from image, extract the various mathematical features that can distinguish the kinds of characters kind, adopt the PCA method to carry out character feature and extract, and feature is carried out yojan with RS, the feature after the yojan is sent into neural network train;
Character recognition unit is used for the characters on license plate after cutting apart is discerned, and examines and adopts multistage multi-categorizer, and template matches is combined with neural network, adopts numeral, letter, digital alphabet mixing and Chinese Character Recognition sorter to discern.
Technical conceive of the present invention is: Flame Image Process and computer vision are constantly new technologies of development, adopt computer vision to observe four purposes in principle, i.e. the debating of the feature extraction of pre-service, the bottom, mid-level features known and by the explanation of image to senior sight.In general, computer vision comprises principal character, Flame Image Process and image understanding.Image is the extension of human vision.By machine vision, can hold the generation whether the parking offense incident is arranged immediately exactly.The basis of image detection rapidity is that the information that vision is accepted is communication media with light; And image information is abundant and directly perceived, is that other present various Detection Techniques all can not provide so abundant and information intuitively.In some flourishing cities, used camera head to carry out road monitoring and security monitoring at present in a large number, but these a large amount of video mass datas all also are not fully utilized, it only is the effect of having served as a telemonitoring eye, lack the support of new and high technologies such as artificial intelligence, do not served as intelligent middle-agent's role.
The omnibearing vision sensor ODVS that developed recently gets up (OmniDirectional Vision Sensors) provide a kind of new solution for the panoramic picture that obtains scene in real time.The characteristics of ODVS are looking away (360 degree), can become piece image to the Information Compression in the hemisphere visual field, and the quantity of information of piece image is bigger; When obtaining a scene image, the riding position of ODVS in scene is free more; ODVS is without run-home during monitoring environment; Algorithm is simpler during moving object in the detection and tracking monitoring range; Can obtain the realtime graphic of scene.Therefore the fully-directional visual system based on ODVS developed rapidly in recent years, just becoming the key areas in the computer vision research, IEEE held the special symposial (IEEE workshop on Omni-directional vision) of annual omni-directional visual since 2000.Because whether the Video Detection at road, street crossing and large scene has parking offense need cover broad as far as possible scope, therefore utilize omnibearing vision sensor can realize above-mentioned requirement, just collect the video information that includes vehicles peccancy easily as long as omnibearing vision sensor is installed in the centre of large scene, hold the vehicle peccancy situation by the understanding of dynamic image, realize middle-agent's role in conjunction with omnibearing vision sensor and dynamic image understanding technology.Also do not retrieve at present the paper and the patent that omnibearing vision sensor and dynamic image understanding technology are applied to the parking offense technical field.
Therefore, adopt omnibearing vision sensor ODVS and utilize digital image processing techniques, some features in conjunction with monitoring scene and vehicle, whether detection has parking behavior violating the regulations in monitoring scene, for the driver who desires parking offense in the monitoring field provides information warning, give digital urban management, the intelligentized insight of law enforcement agency's equipment a pair of.
Beneficial effect of the present invention mainly shows: 1) sensing range is wide, can detect with interior parking offense behavior at 200 rice diameters the orientation; 2) intellectuality of parking offense detection, the robotization that parking offense is handled, the hommization of processing procedure violating the regulations have been realized; 3) improve the dynamics that parking offense is enforced the law effectively, reduced parking offense law enfrocement official's working strength, realized the real time automatic detection of parking offense, transmission automatically in real time, real-time automatic issue and processing automatically in real time; 4) alleviate opposition between law-executor and the parking lawbreaker, made " administrative penalties law " and " law on road traffic safety " on technological layer, have more operability, increased social harmony; 5) the guiding citizen observe traffic rules and regulations, and really make the parking offense driver experience the seriousness of law, improve the civilization degree of entire society.
Description of drawings
Fig. 1 reflexes to omni-directional visual planar imaging synoptic diagram for three-dimensional space;
Fig. 2 is the structural representation of omnibearing vision sensor;
Fig. 3 is the synoptic diagram of omnibearing vision sensor and the integrated design of fire ball camera head;
Fig. 4 is the perspective projection imaging model synoptic diagram of omnibearing vision sensor and general perspective imaging model equivalence;
Fig. 5 is the omnibearing vision sensor undeformed simulation synoptic diagram of epigraph in the horizontal direction;
Fig. 6 is cut apart figure for the monitoring range and the scope of omnibearing vision sensor;
Whether Fig. 7 is for there being the Flame Image Process process flow diagram of parking in the Video Detection monitoring range;
Fig. 8 is the system construction drawing that omnibearing vision sensor cooperates with the fire ball camera head;
The example schematic that Fig. 9 uses for this device;
The process flow diagram that Figure 10 handles for this device software;
Figure 11 detects the figure as a result whether parking is arranged in the monitoring field for edge detection algorithm;
Figure 12 is parking offense vehicle license plate identification processing flow chart.
Below in conjunction with accompanying drawing the present invention is further described.
With reference to Fig. 1～Figure 12, a kind of peccancy parking detector based on computer vision, monitor parking offense behavior in the whole field by a large-range monitoring vision sensor, understand technology for detection to the parking offense incident being arranged a situation arises down by dynamic image, microprocessor is captured the vehicle space positional information indication fire ball camera head of this parking offense by the mapping relations between large-range monitoring vision sensor and the fire ball camera head to this vehicles peccancy, then the general image of being captured violating the regulations is carried out car plate identification, obtain the license plate number of this vehicles peccancy, then do not want parking offense by speech play unit caution parking offense driver, under the inoperative situation of caution advice, system generates a parking offense record automatically, the process video image that in the parking offense record, includes whole parking offense behavior, the close-up image of vehicles peccancy and relevant recognition result, the law enfrocement official can punish vehicles peccancy according to these records, and whole process as shown in Figure 9.
Designed peccancy parking detector based on computer vision is when coming into operation, at first to go out parking offense monitoring field according to the omni-directional visual scope division, specific practice: at first read 360 ° of omni-directional visual video images, as shown in Figure 6, customization monitoring field on these 360 ° of omnidirectional video images does not so just need to carry out this step work if all monitor the field as parking offense in the whole omnidirectional video image then;
Further, Video Detection for large space, in order to obtain more video information, large-range monitoring vision sensor 12 among the present invention adopts omnibearing vision sensor, because omnibearing vision sensor can capture on the horizontal direction 360 ° video image, as shown in Figure 3, and the angle range of the horizontal direction of fire ball vision sensor 1 is 360 °, the angle range of vertical direction is 90 °, omnibearing vision sensor and fire ball vision sensor are carried out information fusion, after omnibearing vision sensor finds that certain vehicle subject object enters in the monitoring range, system returns the coordinate information of this vehicle subject object, and utilize this information to control the fire ball vision sensor and take the vehicle subject object with the location, capture the detail image information of vehicle subject object and handle for follow-up car plate identification.It is worthy of note, need to set in advance a spatial correspondence calibration scale, the fire ball vision sensor is located fast.Scaling method is: with the panoramic picture center is the center of circle, according to differentiating needs panoramic picture is divided into some annulus, as shown in Figure 6, then each annulus is divided several equal portions, can divide according to actual needs.Like this, a width of cloth panoramic picture just has been divided into several zones regularly, and there are its specific angle, direction and size in each zone.Then, according to the parameters such as height, fire ball vision sensor locus and camera focal length of omnibearing vision sensor, determine regional required level, vertical angle and the focal length that rotates that the fire ball vision sensor will detect successively apart from monitoring scene ground.Exist if omnibearing vision sensor has detected the vehicle subject object, system just can regulate the fire ball vision sensor automatically according to this target region and calibration scale, and the vehicle subject object is captured feature; Can realize 360 ° real-time monitoring of horizontal direction in omnibearing vision sensor, its core component is the catadioptric minute surface, shown in 11 among Fig. 3; Its principle of work is: the manufacturing technology scheme of the opticator of ODVS camera head, ODVS camera head are mainly constituted by vertically downward catadioptric mirror with towards last camera.Concrete structure or be fixed on bottom by the round body of transparent resin or glass by the image unit that collector lens and CCD constitute, the top of round body is fixed with the catadioptric mirror of a downward deep camber.What Fig. 1 represented is the schematic diagram of the optical system of omnibearing vision sensor of the present invention.
Catadioptric omnidirectional imaging system can be carried out imaging analysis with the pin-hole imaging model, must must satisfy the requirement of real-time to the contrary projection of the real scene image of gathering but will obtain the perspective panorama picture.
The coordinate of the horizontal coordinate of object point and corresponding picture point is linear in the monitoring scene just can guarantee that horizontal scene is undistorted, omnibearing vision sensor as peccancy parking detector is installed in the monitoring scene center top, monitor the situation that the vehicle object on the horizontal direction in the whole monitoring scene occurs, therefore when the catadioptric minute surface of design omnibearing vision device, will guarantee in the horizontal direction indeformable.
At first select for use CCD (CMOS) device and imaging len to constitute camera in the design, preresearch estimates system physical dimension on the basis that the camera inner parameter is demarcated is determined the mirror surface shape parameter according to the visual field of short transverse then.
As shown in Figure 1, the projection centre C of camera is the horizontal scene h of distance place above the horizontal scene of road, and the summit of catoptron is above projection centre, apart from projection centre zo place.Be that true origin is set up coordinate system with the camera projection centre among the present invention, the face shape of catoptron is with z (X) function representation.The pixel q of distance images central point ρ has accepted from horizontal scene O point (apart from Z axle d), at the light of mirror M point reflection in as the plane.Therefore desirable detected status is undistorted on horizontal scene, requires the coordinate of the horizontal coordinate of scene object point and corresponding picture point linear;
ρ is and the distance of the face shape central point of catoptron in the formula (1), and α is the magnification of imaging system.
If the normal that catoptron is ordered at M and the angle of Z axle are γ, the angle of incident ray and Z axle is Φ, and the angle of reflection ray and Z axle is θ.Then
By reflection law
Obtain the differential equation (7) by formula (2), (4), (5) and (6)
In the formula;
Obtain the differential equation (9) by formula (7)
Obtain formula (10) by formula (1), (5)
By formula (8), (9), (10) and starting condition, separate the digital solution that the differential equation can obtain reflecting mirror surface shape.The main digital reflex mirror of system's physical dimension is from the distance H o and the aperture of a mirror D of camera.Select suitable camera according to application requirements during the refractive and reflective panorama system design, calibrate Rmin, the focal distance f of lens is determined the distance H o of catoptron from camera, calculates aperture of a mirror Do by (1) formula.
Determining of systematic parameter:
Determine systematic parameter af according to the visual field of using desired short transverse.Obtain formula (11) by formula (1), (2) and (5), done some simplification here, with z (x) ≈ z 0, main consideration is smaller with respect to the change in location of minute surface and camera for the height change of minute surface;
With the inconocenter point largest circumference place in the center of circle as the plane
Corresponding visual field is ф max.Then can obtain formula (12);
The imaging simulation adopts the direction opposite with actual light to carry out.If light source is in the camera projection centre, equally spaced selected pixels point in the picture plane by the light of these pixels, intersects with surface level after mirror reflects, if intersection point is equally spaced, illustrates that then catoptron has the distortionless character of horizontal scene.The imaging simulation can be estimated the imaging character of catoptron on the one hand, can calculate aperture of a mirror and thickness exactly on the other hand.
Image transformation relates to the conversion between the different coordinates.In the imaging system of video camera, what relate to has following 4 coordinate systems; (1) real-world coordinates is XYZ; (2) with the video camera be the coordinate system x^y^z^ that formulate at the center; (3) photo coordinate system, formed photo coordinate system x in video camera *y *o *(4) computer picture coordinate system, the coordinate system MN that the computer-internal digital picture is used is a unit with the pixel.
According to the different transformational relation of above several coordinate systems, just can obtain needed omnidirectional vision camera imaging model, converse the corresponding relation of two dimensional image to three-dimensional scenic.The approximate perspective imaging analytical approach that adopts catadioptric omnibearing imaging system among the present invention is with the formed corresponding relation that is converted to three-dimensional scenic as the planimetric coordinates two dimensional image in the video camera, Fig. 3 is general perspective imaging model, d is an object height, ρ is an image height, t is an object distance, and F is image distance (equivalent focal length).Can obtain formula (13)
When above-mentioned horizontal scene does not have the design of catadioptric omnibearing imaging system of distortion, require the coordinate of the horizontal coordinate of scene object point and corresponding picture point linear, represent suc as formula (1); Comparison expression (13), (1), horizontal as can be seen scene does not have the be imaged as perspective imaging of the catadioptric omnibearing imaging system of distortion to horizontal scene.Therefore with regard to horizontal scene imaging, the catadioptric omnibearing imaging system that horizontal scene can not had distortion is considered as having an X-rayed camera, and α is the magnification of imaging system.If the projection centre of this virtual perspective camera is C point (seeing accompanying drawing 3), its equivalent focal length is F.Comparison expression (13), (1) formula can obtain formula (14);
Obtain formula (15) by formula (12), (14)
Carry out the system imaging simulation according to above-mentioned omnidirectional vision camera imaging model, by the camera projection centre send through in the pixel planes equidistantly after the reflection of the light family of pixel, intersection point on the surface level of the monitoring scene of distance projection centre 5m is equally spaced basically, as shown in Figure 4.Therefore according in the above-mentioned design concept this patent relation between the coordinate of the coordinate of monitoring scene surface level and corresponding comprehensive picture point being reduced to linear relationship, that is to say that design by mirror surface be XYZ to the conversion of photo coordinate system with real-world coordinates can be the linear dependence of ratio with magnification α.Be conversion below from photo coordinate system to the used coordinate system of computer-internal digital picture, the image coordinate unit that uses in the computing machine is the number of discrete pixel in the storer, so also need round the imaging plane that conversion just can be mapped to computing machine to reality as the coordinate on plane, its conversion expression formula is for to be provided by formula (16);
In the formula: Om, On are respectively the line number and the columns at the some pixel place that the initial point of image plane shone upon on the computer picture plane; Sx, Sy are respectively scale factor in the x and y direction.Determining of Sx, Sy is by placing scaling board apart from the Z place between camera and mirror surface, video camera is demarcated the numerical value that obtains Sx, Sy, and unit is (pixel); Om, On.Determine it is that unit is (pixel) according to selected camera resolution pixel.
Further, whether car is arranged in the monitoring field, can adopt two kinds of detection modes: 1) based on the detection of color model for Video Detection based on omnibearing vision sensor; 2) based on the detection of image border profile (seeing that from depression angle the profile of vehicle all is rendered as the rectangle of rule).
The parking offense Video Detection of monitoring scene is the key of total system, be implemented in and detected car in the monitoring scene and park, on Flame Image Process, need to safeguard and step such as background subtracting confirms whether vehicle is arranged in the whole monitoring scene by the background image threshold value.In general, the ground color in the monitoring scene and the color of vehicle have tangible difference.Utilize color characteristic in the video image to judge the parking offense of monitoring scene in this patent.
Whether monitoring scene in have parking offense, the first step is to obtain a more stable reference image in image recognition, utilizes it to do the background subtracting algorithm then if judging.Adopted the threshold value of in monitoring scene, extracting an average chrominance in this patent, as color threshold.The color of considering monitoring scene ground all is approaching, therefore can be used as the global color threshold value of whole monitoring scene.In actual use, some variations also can take place in the color model on monitoring scene ground, therefore need constantly upgrade the color threshold data.
Further, adopt the YCrCb color model of image to discern whether the generation of parking incident is arranged in the monitoring scene.Because the Y component is to light sensitive in the YCrCb color space, and these two components of CrCb are just relevant to color, and the color of distinguishing vehicle by these two components of CrCb can reach the purpose whether generation of parking incident is arranged in the monitoring scene with the color on ground, parking stall.By the Cr in the color space, the detection of Cb component can reach and the irrelevant effect of monitoring scene illuminating ray basically.Therefore from the color space of the video image that omnibearing vision sensor obtained is RGB, needs in program video image from the RGB color space conversion to the YCrCb color space, and conversion formula (17) provides,
Further, adopt the background subtracting algorithm to obtain the vehicle object information, so-called background subtracting algorithm is also referred to as difference method, is a kind of image processing method that is usually used in detected image variation and moving object.
The computing formula of background subtracting as the formula (18),
f d(X，t 0，t i)＝f(X，t i)-f(X，t 0) (18)
In the formula: f d(X, t 0, t i) be to photograph the colourity result who carries out image subtraction between panoramic picture and reference panoramic picture in real time.F (X, t i) be to photograph image chroma value, f (X, t in real time 0) be reference image chroma reference value.Whether to have the parking incident to take place in the monitoring scene in order judging, therefore to make background subtracting and calculate in defined monitoring scene, judge Cr, whether these two components of Cb have surpassed the threshold range of defined.Exist parking offense if surpassed the words of the threshold range of defined with regard to the mark monitoring scene, otherwise, parking offense do not had.Judgment formula as the formula (19),
In the formula: Cr represents the Cr mean value in the detected current monitoring scene, Cb represents the Cb mean value in the detected current monitoring scene, ThresholdCr represents the Cr mean value on ground in the defined monitoring scene, ThresholdCb represents the Cb mean value on ground in the defined monitoring scene, threshold1 represents the threshold range of Cr, and threshold2 represents the threshold range of Cb.
The another kind of method whether car is arranged in the current monitoring scene that detects, out of doors since the background of monitoring scene can be because the variation of weather flies upward thing (leaf, the polybag etc.) influence of face color model over the ground in (rain, snow etc.), the monitoring scene, whether have car can bring bigger false recognition rate, therefore can detect at the parking offense of monitoring scene and adopt the edge of image detection method with color model if carrying out monitoring scene;
The border of car body is the very important descriptor of a class of describing the car body feature, and these borders may produce marginal information in imaging process.The edge is meant the combination that those pixels of significant change are arranged in its surrounding pixel gray scale.The edge is the vector with amplitude and direction, and it shows as the sudden change of gray scale in image.Rim detection is exactly the noncontinuity that will detect this gray scale in the image.
There is several method to select to rim detection at present, because what expectation obtained in the present invention is the edge of car body, and it is less demanding to the integrality and the slickness of edge wheel hub, therefore it is simple that we adopt calculating wherein, classical edge detection method-the method for differential operator of fast operation, this method relies on image differentiated and tries to achieve gradient and carry out rim detection, main from marginal point often corresponding to the big point of single order differential amplitude, while also sets out corresponding to the zero cross point of second-order differential, design some single orders or second-order differential operator, try to achieve its gradient or second derivative zero crossing, select certain threshold value to extract the border again.
Described edge detection method can be divided into following four steps haply:
1. filtering: edge detection algorithm mainly is based on the first order derivative and the second derivative of image intensity, but the calculating of derivative is very sensitive to noise, therefore must use wave filter to improve the performance of the edge detection method relevant with noise.It may be noted that most of wave filters have also caused the loss of edge strength when reducing noise.Therefore the edge strengthens and reduces between the picture noise needs to obtain a kind of balance.
2. strengthen: the basis that strengthens the edge is a changing value of determining each vertex neighborhood intensity in the image.Enhancement algorithms can be given prominence to the point that the neighborhood intensity level has significant change.The edge strengthens generally to be finished by the compute gradient amplitude.
3. detect: in image, have the gradient magnitude of many points bigger, and these might not all be the edges under specific situation, so should be with coming someway to determine that those points are marginal points.The simplest rim detection criterion is a gradient magnitude A value criterion.
4. locate: determine the pixel at place, edge, if more accurate definite marginal position also can come the estimated edge position on subpixel resolution, the direction at edge also can be estimated.
Adopt Suo Beier (Sobel) operator as edge detection algorithm in the present invention, the Sobel operator adopts the template of 3*3 size, has so just avoided compute gradient on the interpolated point between the pixel.The Sobel operator calculates partial derivative with following formula:
S x＝(a 2+ca 3+a 4)-(a 0+ca 7+a 6)
S y＝(a 0+ca 1+a 2)-(a 6+ca 5+a 4)
Constant c is 2 in the formula.The Sobel operator can be realized with following convolution template:
The method that can be used for detecting car body in monitoring scene can be a template matching method.This method is that the object of all positions in the model of the car body that will will detect and the monitoring scene compares, and investigates the object that whether exists the model with car body to be complementary.The matching algorithm that the present invention adopts is: the vehicle image template of establishing the known target object is T, and size be M*N, and vehicle image template T leaves in the computing machine, makes different templates according to different vehicle needs, so that use when comparing; The image of vehicle is I in the monitoring scene to be detected, and size is L*W (L＞M, W＞N).The process of coupling is to manage a template T to be superimposed upon on the image I, and compares the difference of the subimage of the I under T and its covering.If difference, thinks then that T has coupling preferably at the subimage of this place and I, has promptly found destination object less than certain prior preset threshold.Entire image to be matched is pressed individual element scanning and implemented aforesaid operations, then can determine whether exist the determined destination object of template T, i.e. certain type vehicle in the image I.The mathematical description of matching process can be represented by formula (22):
Before images match is handled, need the pre-service of image, comprise and carry out edge extraction, binaryzation, profile is linked to be closed curve and the curve inner region is carried out white pixel fill.Obtain the centre of form coordinate and the center invariant moments of the bianry image of the car body template that is stored in the computing machine and the object in the monitoring scene respectively and (suppose object zero lap in the monitoring scene here, omni-directional visual passed after device is installed in certain altitude the vehicle in the monitoring range is not overlapped), with the invariant moments of template respectively with monitoring scene in each object invariant moments relatively, difference can predicate the object shown in the template-be car body less than certain preset threshold.Then the centre of form of object in the scene of parking stall is aimed at the centre of form of template, utilize " convergent-divergent " and image processing methods such as " rotations " that template is alignd with object in the scene, to determine whether object really is the represented object of template in the scene, if just be judged to be in this monitoring range car arranged, then according to the result of template matches, obtain the type of parking vehicle in this monitoring range.
Turn to the image of M*N for a spatial spreading, if i, j is each pixel coordinate in the monitoring scene of discretize, and (i is (i for coordinate j) to f, the gray-scale value of pixel j), according to the notion and the acquiring method of above-mentioned invariant moments, according to the disposal route of the digital picture of discretize, then " central moment " of object can be approached by following dual summation form in the image scene, represent with formula (23)
For the center invariant moments is not become with proportional zoom, carry out normalized, can get normalization permanent center square and be: η Pq=D Pq/ D 00, r=(p+q)/2+1,
If p+q＜2 then can be derived two constant central moment functions of RST and be represented with formula (24),
The concrete steps of described template matching algorithm are as follows:
1) each object in template and the monitoring scene is carried out pre-service: utilize " Roberts " operator respectively the object in the monitoring scene to be carried out edge sharpening, and select appropriate threshold to carry out binaryzation.Marginal information to the object of closure is carried out the skeleton refinement, with the closed curve at direction chain code (or array) expression scenery edge, with the connectedness of the closed curve of the object edge that guarantees to represent with single pixel chain.Carry out the zone and fill, with the zone in the maximum gradation value filling object edge closed curve.If the region area of asking is greater than a threshold value (projected area of minimum vehicle is this threshold value), calculate the centre coordinate (barycentric coordinates) of scenery in the monitoring range then according to formula (23), ask for second order normalization center invariant moments according to formula (24), otherwise do not carry out to judge.
2) with the center invariant moments of the object of monitoring scene and template relatively, and get the phase difference and can tentatively think the object that template is represented less than a certain threshold value.The centre of form with template overlaps with the centre of form coordinate of object in the scene then.
3) template is carried out convergent-divergent so that the consistent size of the essentially identical object of invariant moments with it in itself and the scene.Utilize the rotational transform of coordinate to determine the anglec of rotation of object correspondence in the scene.The algorithm that convergent-divergent and rotation are adopted is respectively:
The convergent-divergent algorithm:
x 2＝N(x 1-x 0)+x 0
y 2＝N(y 1-y 0)+y 0
The rotation algorithm:
x 2＝(x 1-x 0)cosθ+(y 1-y 0)sinθ+x 0
y 2＝(y 1-y 0)cosθ+(x 1-x 0)sinθ+y 0
More than in two groups of formulas only the edge contour point to template carry out convergent-divergent and rotation processing, also need link afterwards and the closed curve interior pixel is filled.(x in the formula 2, y 2) be the new coordinate of pixel; (x 1, y 1) be former coordinate; (x 0, y 0) be the centre of form coordinate of object; N is a zoom factor; θ is the angle of rotation.
Whether 4) bianry image that above-mentioned two width of cloth have been aimed at carries out exclusive-OR operation, really be the type of car body shown in the template and vehicle with this object that how much confirms of residual pixel.
A kind of peccancy parking detector based on computer vision, comprise a plurality of vision sensors, be used for vehicle to image tracing, the microprocessor of license plate image identification, be used to be presented at the interior vehicle of monitoring scene to the image tracing situation, the vehicle general image, the display unit of vehicle license plate image and the speech play unit that is used to play the vehicles peccancy caveat, described a plurality of vision sensor connects microprocessor by video card, described a plurality of vision sensor comprises a scene monitoring vision sensor 12 and a fire ball vision sensor 1 that carries out feature candid photograph vehicle image and this vehicle license plate image on a large scale, described microprocessor comprises: image-display units is used to show the video image in the whole monitoring range, follow the tracks of the general image of vehicle object and the car plate parts of images of following the tracks of vehicle; Described large-range monitoring vision sensor 12 is an omnibearing vision sensor, and this omnibearing vision sensor is installed in the centre of large-range monitoring scene, is used to monitor the tracking vehicle object of whole monitoring scene; Described fire ball vision sensor 1, be used for the vehicle that enters in the monitoring scene is carried out the feature candid photograph, obtain the locus at vehicle place by the tracking of large-range monitoring vision sensor 12, microprocessor is captured then according to the direction rotation focusing of this positional information indication fire ball vision sensor 1 towards the locus at vehicle place; Action relationships between described large-range monitoring vision sensor 12 and the fire ball vision sensor 1 realizes by mapping table; Described omnibearing vision sensor comprises the evagination catadioptric minute surface that is used for reflecting monitoring field object, evagination catadioptric minute surface down, be used to support the transparent round body of evagination catadioptric minute surface, be used to take the camera of imaging body on the evagination mirror surface, camera facing to the evagination mirror surface up; Described microprocessor also comprises: large-range monitoring vision sensor 12 demarcating modules are used to set up the corresponding relation of monitoring scene image and the video image that is obtained; Video data Fusion Module between large-range monitoring vision sensor 12 and the fire ball vision sensor 1 is used to control the rotation and the focusing of fire ball vision sensor 1, makes fire ball vision sensor 1 can aim at the vehicle of following the tracks of and carries out the feature candid photograph; The multiple target tracking module is used to follow the tracks of the vehicle object that enters in the monitoring field; Vehicle enters the monitoring range detection module, is used for system and produces an incident automatically, all can call corresponding processing module during incident of the every generation automatically of system; Vehicle ID number and deposit the automatically-generating module of following the tracks of the vehicle image file, the vehicle object that is used for when just entering the monitoring visual field ragged edge outline line of large-range monitoring vision sensor 12 is named, and generate a file simultaneously with ID number name of this vehicle, be used to deposit the close-up image of this vehicle; Module is captured in vehicle general image location, is used to locate the image of the integral body of capturing this vehicle; Module, the image that is used to locate the suspension car plate position of this vehicle and this position is captured car plate are captured in the vehicle license plate location; The car plate identification module is used to discern the license plate number of vehicles peccancy; Caveat generates and broadcast unit, is used to warn the parking offense driver and does not want parking offense; Testing result affirmation, change, completion module are used to the license plate number of confirming whether license plate number identification is correct, recognition result, completion that change identification error do not have identification;
Described vehicle ID number, be used to produce one and can identify the major key that the vehicle object followed the tracks of, storage write down some other attribute data of time that enters the monitoring field of this vehicle object and this vehicle, vehicle ID number naming rule is: YYYYMMDDHHMMSS *With the name of 14 bit signs, YYYY-represents the year of the Gregorian calendar; MM-represents the moon; DD-represents day; HH-represents hour; MM-represents branch; SS-represents second; Automatically produce by the system for computer time;
The described automatically-generating module of following the tracks of the vehicle image file of depositing, be used to preserve the close-up image of the vehicular traffic object of tracking, when the vehicle object enters monitoring range (outline line of outermost), system automatically produce one vehicle ID number, in certain deposits the file of image, create simultaneously one with vehicle ID number file of the same name, be used for depositing the close-up image of this vehicle, the naming method of image file is to be produced automatically by the system for computer time, naming rule is HHMMSS, and HH-represents hour; MM-represents branch; SS-represents second;
Module is captured in described vehicle general image location, is used to locate the image of the integral body of capturing this vehicle; The vehicle that we enter in the monitoring field each in described multiple target tracking module is all followed the tracks of, when the vehicle object of following the tracks of enters the detection range outer contour, system can be according to the center of large-range monitoring vision sensor 12 detected vehicle objects, information such as size send instructions and carry out the candid photograph of vehicle general image location to 1 pair of vehicle object of fire ball vision sensor, owing to might not include the video information of the license plate number of this vehicle in the vehicle general image that when capturing, obtains, therefore the general image number of each vehicle object of capturing is indefinite among the present invention, in case the identified back of this vehicle license plate just stops to capture this vehicle object automatically;
In general, first character of car plate is a Chinese character; Second character is letter; Three, the 4th character is numeral and alphabetical mixing; Three of backs are numerals.According to this situation, need design numeral, letter, digital alphabet to mix and the Chinese character sorter recognition methods of adopting template matches to combine with neural network.
Described car plate identification module, be used for discerning enter vehicle license plate in the monitoring field number and for follow-up in accordance with the law to the major key of this vehicle peccancy processing procedure vehicles peccancy information retrieval; The Character segmentation unit, normalized unit, extraction feature unit and the character recognition unit that in the car plate identification module, comprise image pretreatment unit, car plate location, car plate;
Described image pretreatment unit is used for that original image is carried out various zones and handles (comprising enhancing, filtering etc.).At first license plate image is carried out grey level stretching in this unit, adopt the global threshold method that image is carried out binary conversion treatment, then it is adopted medium filtering, morphologic filtering is eliminated various interference, thus the characteristic information of outstanding car plate;
Described car plate positioning unit is used at the view picture license plate image car plate being carried out level and vertical projection location car plate; Be primarily aimed at the civilian licence plate of standard among the present invention, this class licence plate has following feature:
A) car plate background color and vehicle body, character color have bigger difference;
B) car plate has one continuously or because wearing and tearing and discontinuous frame, and character has a plurality ofly in the car plate, is substantially horizontally, so there is more rich edge in the rectangular area of licence plate, presents well-regulated textural characteristics;
C) interval between the character is more even in the car plate, and there are saltus step in character and licence plate background color on gray-scale value, and character itself all is that intensity profile is uniformly arranged with the licence plate background color;
D) concrete size, the position of licence plate is uncertain in the different images, but its length breadth ratio variation has certain limit, has a minimum and maximum length breadth ratio.
According to above-mentioned licence plate feature rough detection is carried out in the place that might have licence plate in the vehicle general image,, otherwise require system to capture vehicle image again if find have the place of similar licence plate to position;
Described characters on license plate cutting unit is used for characters on license plate is divided into single character, helps each character is carried out Feature Extraction and identification.Adopting character vertical projection histogram to combine with characters on license plate priori width information among the present invention cuts apart;
Described normalized unit is used for the feature extraction and the identification in later stage, is unified format size with character change, adopts the method for normalizing of neighbor interpolation among the present invention;
Described extraction feature unit, be used for the character recognition in later stage, from image, extract the various mathematical features that can distinguish the kinds of characters kind, adopting PCA (principal component analysis (PCA)) method to carry out character feature among the present invention extracts, and feature is carried out yojan with RS (rough set), the feature after the yojan is sent into neural network train;
Described character recognition unit is used for the characters on license plate after cutting apart is discerned, and considers that characters on license plate has different characteristic, adopts corresponding sorter that character is discerned.Adopt multistage multi-categorizer in the present invention, template matches is combined with neural network, designed numeral, letter, digital alphabet mixing and Chinese Character Recognition sorter;
Described caveat generates and broadcast unit, be used to warn the parking offense driver and do not want parking offense, above-mentioned when detecting the vehicle that newly enters the monitoring field and identifying its number-plate number, system generates caveat according to the number-plate number that is identified, and plays this caveat by voice-output unit then;
According to Figure 10 treatment scheme of the present invention is described below, at first read the video image of omni-directional visual, then the vehicle that occurs in monitoring range in this video image is followed the tracks of, vehicle parking is arranged or sail in the monitoring range if find, whether the license plate number of then judging these vehicles is again successfully identified, if exist the situation that is successfully identified vehicle, according to the position that detected this vehicle of omni-directional visual is positioned at present, indication fire ball camera head regulates corner according to mapping relations shown in Figure 6 and focal length is captured this vehicle; To the Vehicular system that just enters the monitoring field boundary can automatically generate one the vehicle ID relevant number and with the file of vehicle ID number name with system time, and the vehicle image of capturing left in this document presss from both sides, image file name was named with the time of system; Can not park cars at this to the vehicle drivers that just enters the monitoring field boundary by voice reminder simultaneously; Then whether systems inspection license plate number storage period of successfully not identifying and still being parked at present all vehicles in the monitoring range has surpassed the value of a regulation, just is judged to be parking offense if surpassed this setting; At this moment system reads location and the identification processing of carrying out vehicle license plate with the image file in the file of ID number name of this vehicle, if discerning successfully, parking offense logout of just automatic generation is kept in the database, notify the law enfrocement official to have the parking offense incident to take place in the control point by network simultaneously, so that the law enfrocement official maintains traffic order apace, behavior is educated and is punished to parking offense; Under law enfrocement official's shortage of manpower situation,, equally also can realize the effect of afterwards enforcing the law according to the result of videograph and license plate image identification because system carried out detail record with the process of parking offense incident.
In order effectively to utilize the video monitoring apparatus that has used at present, can corresponding parking offense be installed in terminal and detect software, utilize the dynamic image understanding technology among the embodiment 1, whether detection has parking behavior violating the regulations in the video monitoring apparatus monitoring range, in case having detected parking behavior violating the regulations, system takes place just immediately by the netcast caveat, caution is parking offense not, notifies the law enfrocement official to go to the scene to safeguard that traffic rules in time handle act of violating regulations as early as possible simultaneously.
Implementation algorithm among the present invention is mainly realized by Java language.
Priority Applications (1)
|Application Number||Priority Date||Filing Date||Title|
|CNB2007101644805A CN100565555C (en)||2007-12-05||2007-12-05||Peccancy parking detector based on computer vision|
Applications Claiming Priority (1)
|Application Number||Priority Date||Filing Date||Title|
|CNB2007101644805A CN100565555C (en)||2007-12-05||2007-12-05||Peccancy parking detector based on computer vision|
|Publication Number||Publication Date|
|CN101183427A CN101183427A (en)||2008-05-21|
|CN100565555C true CN100565555C (en)||2009-12-02|
Family Applications (1)
|Application Number||Title||Priority Date||Filing Date|
|CNB2007101644805A CN100565555C (en)||2007-12-05||2007-12-05||Peccancy parking detector based on computer vision|
Country Status (1)
|CN (1)||CN100565555C (en)|
Families Citing this family (66)
|Publication number||Priority date||Publication date||Assignee||Title|
|CN101667294B (en) *||2009-09-10||2011-10-05||天津工业大学||Object detecting and tracking device|
|CN101718528B (en) *||2009-12-10||2011-06-08||北京科技大学||Digital image based rapid solving method of circle parameters|
|CN102110367A (en) *||2009-12-25||2011-06-29||刘天宇||System for detecting illegal parking behaviors in real time and control method thereof|
|CN102238366A (en) *||2010-04-26||2011-11-09||鸿富锦精密工业（深圳）有限公司||Camera of realizing image tracking and monitoring and method thereof|
|KR20120062335A (en) *||2010-12-06||2012-06-14||한국전자통신연구원||Apparatus and method for managing a block of ship|
|CN102045548A (en) *||2010-12-28||2011-05-04||天津市亚安科技电子有限公司||Method for controlling automatic zoom of PTZ (pan/tilt/zoom) camera|
|CN102136194B (en) *||2011-03-22||2013-06-05||浙江工业大学||Road traffic condition detection device based on panorama computer vision|
|CN102163287B (en) *||2011-03-28||2014-06-18||北京邮电大学||Method for recognizing characters of licence plate based on Haar-like feature and support vector machine|
|CN102110366B (en) *||2011-03-28||2012-10-10||长安大学||Block-based accumulated expressway vehicle parking event detecting method|
|JP5525495B2 (en) *||2011-08-30||2014-06-18||株式会社日立製作所||Image monitoring apparatus, image monitoring method and program|
|CN103000028B (en) *||2011-09-14||2015-12-09||上海宝康电子控制工程有限公司||Number plate of vehicle recognition system and recognition methods|
|CN102520736B (en) *||2011-11-08||2014-07-02||中国华能集团清洁能源技术研究院有限公司||Load control method of coal nozzles of dry pulverized coal gasification furnace|
|TWI489395B (en) *||2011-11-28||2015-06-21||Ind Tech Res Inst||Apparatus and method for foreground detection|
|CN103176185B (en) *||2011-12-26||2015-01-21||上海汽车集团股份有限公司||Method and system for detecting road barrier|
|CN102663492A (en) *||2012-03-19||2012-09-12||南京理工大学常熟研究院有限公司||Maneuvering target tracking system based on nerve network data fusion|
|CN102680488B (en) *||2012-03-31||2015-06-17||北京农业信息技术研究中心||Device and method for identifying massive agricultural product on line on basis of PCA (Principal Component Analysis)|
|CN102637360B (en) *||2012-04-01||2014-07-16||长安大学||Video-based road parking event detection method|
|CN102722704B (en) *||2012-06-12||2014-04-30||厦门宸天电子科技有限公司||Method and system for recognizing vehicle license plate by integrating video dynamic tracking|
|KR101928876B1 (en) *||2012-09-14||2018-12-14||현대모비스 주식회사||System and Method for Reducing Problem of Double Parking|
|CN103116993A (en) *||2012-12-12||2013-05-22||樊荣||Taxi waiting notice board|
|CN103116984B (en) *||2013-01-21||2016-03-23||信帧电子技术（北京）有限公司||Detect the method for parking offense|
|CN103116985B (en) *||2013-01-21||2015-04-01||信帧电子技术（北京）有限公司||Detection method and device of parking against rules|
|CN103093249B (en) *||2013-01-28||2016-03-02||中国科学院自动化研究所||A kind of taxi identification method based on HD video and system|
|CN103136514B (en) *||2013-02-05||2016-03-30||长安大学||A kind of parking event detecting method based on bi-directional tracking|
|CN104184932B (en) *||2013-05-20||2018-03-23||浙江大华技术股份有限公司||Ball machine control method and device|
|CN103324936B (en) *||2013-05-24||2016-05-25||北京理工大学||A kind of vehicle lower boundary detection method based on Multi-sensor Fusion|
|CN103279756B (en) *||2013-06-13||2016-06-22||苏州市公安局苏州工业园区分局||Vehicle detection based on integrated classifier analyzes system and determination method thereof|
|DE102013213748A1 (en) *||2013-07-15||2015-01-15||Ford Global Technologies, Llc||Method and device for monitoring a parking authorization of a motor vehicle|
|CN103440647B (en)||2013-08-20||2016-12-07||中山大学||The detection method of a kind of vehicle sun visor state and system|
|CN103400424A (en) *||2013-08-21||2013-11-20||苏州换游信息科技有限公司||Fine operating system for automobile|
|CN103646544B (en) *||2013-11-15||2016-03-09||天津天地伟业数码科技有限公司||Based on the vehicle behavioural analysis recognition methods of The Cloud Terrace and camera apparatus|
|CN103824452B (en) *||2013-11-22||2016-06-22||银江股份有限公司||A kind of peccancy parking detector based on panoramic vision of lightweight|
|CN103824037B (en) *||2013-12-26||2017-05-03||苏州清研微视电子科技有限公司||Vehicle anti-tracking alarm device|
|CN104834886A (en) *||2014-02-11||2015-08-12||浙江大华技术股份有限公司||Method and device for detecting video image|
|CN103870832B (en) *||2014-03-21||2017-11-03||浙江宇视科技有限公司||Vehicle global feature extracting method and vehicle model recognition methods|
|CN103824458B (en) *||2014-03-25||2016-09-28||宁波市江东元典知识产权服务有限公司||Temporary parking caution system based on image recognition technology|
|CN104268887B (en) *||2014-09-30||2017-07-11||中智科创机器人有限公司||A kind of camera attitude scaling method and device suitable for the switching of many scenes|
|CN104464290B (en) *||2014-12-08||2018-12-21||无锡加视诚智能科技有限公司||A kind of road traffic parameter acquisition based on embedded dual core chip and capturing system violating the regulations|
|CN104486598A (en) *||2014-12-31||2015-04-01||国家电网公司||Video monitoring method and device|
|US10872241B2 (en)||2015-04-17||2020-12-22||Ubicquia Iq Llc||Determining overlap of a parking space by a vehicle|
|US10043307B2 (en)||2015-04-17||2018-08-07||General Electric Company||Monitoring parking rule violations|
|CN105046196B (en) *||2015-06-11||2018-04-17||西安电子科技大学||Front truck information of vehicles structuring output method based on concatenated convolutional neutral net|
|CN104965928B (en) *||2015-07-24||2019-01-22||北京航空航天大学||One kind being based on the matched Chinese character image search method of shape|
|CN105354570B (en) *||2015-10-15||2019-04-09||深圳市捷顺科技实业股份有限公司||A kind of pinpoint method and system of license plate right boundary|
|CN105574485A (en) *||2015-11-11||2016-05-11||北京四象网讯科技有限公司||Vehicle information identification method and system|
|CN105701497A (en) *||2016-01-07||2016-06-22||陈杨珑||Garbage classification processing device with rapid classification function|
|CN105678262A (en) *||2016-01-07||2016-06-15||林业城||Tunnel geological monitoring equipment capable of conducting autonomous operation|
|CN105654734A (en) *||2016-03-15||2016-06-08||深圳市佳信捷技术股份有限公司||Data processing method and device for vehicle violations|
|CN108122412A (en) *||2016-11-26||2018-06-05||沈阳新松机器人自动化股份有限公司||The method disorderly stopped for supervisory-controlled robot detection vehicle|
|US10296794B2 (en)||2016-12-20||2019-05-21||Jayant Rtti||On-demand artificial intelligence and roadway stewardship system|
|CN107066929A (en) *||2017-01-06||2017-08-18||重庆大学||The manifold freeway tunnel Parking hierarchical identification method of one kind fusion|
|CN106851196B (en) *||2017-01-25||2019-07-19||安徽达尔智能控制系统股份有限公司||City, which is disobeyed, stops capturing system|
|CN106909923A (en) *||2017-02-20||2017-06-30||汪爱民||A kind of vehicles peccancy processing system|
|CN106952476A (en) *||2017-02-22||2017-07-14||成都华安视讯科技有限公司||The camera chain for stopping dynamic monitoring is disobeyed available for vehicle|
|CN106888365A (en) *||2017-02-22||2017-06-23||成都华安视讯科技有限公司||A kind of dynamic environment monitoring system and method|
|CN107154045B (en) *||2017-03-31||2020-06-09||南京航空航天大学||Accurate positioning method for vehicles on flow guide area illegal rolling line based on computer vision|
|CN107122729A (en) *||2017-04-21||2017-09-01||奇酷互联网络科技（深圳）有限公司||Generation method, device and the mobile terminal of image|
|CN107038870A (en) *||2017-05-11||2017-08-11||南宁市正祥科技有限公司||A kind of vehicle peccancy detection method|
|CN107403553A (en) *||2017-08-02||2017-11-28||苏州蓝越软件有限公司||A kind of key road segment parking offense detecting system|
|CN107590437A (en) *||2017-08-11||2018-01-16||安徽中科新萝智慧城市信息科技有限公司||A kind of parking offense method of calling and system based on Car license recognition|
|CN107609491B (en) *||2017-08-23||2020-05-26||中国科学院声学研究所||Vehicle illegal parking detection method based on convolutional neural network|
|CN107704841A (en) *||2017-10-25||2018-02-16||云南电网有限责任公司电力科学研究院||A kind of image/video abnormal state detection identifies system|
|CN108596973B (en) *||2018-05-10||2020-04-24||东南大学||Method for fixed-point monitoring of virtual preset positions for land illegal and illegal buildings|
|CN108460972A (en) *||2018-05-17||2018-08-28||南京云思创智信息科技有限公司||A kind of more car plates based on unmanned plane independently detection and positioning system and method|
|JP2019212967A (en) *||2018-05-31||2019-12-12||キヤノン株式会社||Imaging apparatus and control method therefor|
|CN109120904A (en) *||2018-10-19||2019-01-01||武汉星巡智能科技有限公司||Binocular camera monitoring method, device and computer readable storage medium|
- 2007-12-05 CN CNB2007101644805A patent/CN100565555C/en not_active IP Right Cessation
Also Published As
|Publication number||Publication date|
|US10147004B2 (en)||Automatic image content analysis method and system|
|US9704060B2 (en)||Method for detecting traffic violation|
|Borkar et al.||A novel lane detection system with efficient ground truth generation|
|CN103105174B (en)||A kind of vehicle-mounted outdoor scene safety navigation method based on AR augmented reality|
|Fang et al.||Road-sign detection and tracking|
|US20160284097A1 (en)||Multi-cue object detection and analysis|
|Barnes et al.||Real-time speed sign detection using the radial symmetry detector|
|Song et al.||Lane detection and classification for forward collision warning system based on stereo vision|
|CN102314600B (en)||Shadow removal in image captured by vehicle-based camera for clear path detection|
|CN101142584B (en)||Method for facial features detection|
|US8391560B2 (en)||Method and system for image identification and identification result output|
|US9842266B2 (en)||Method for detecting driver cell phone usage from side-view images|
|US8233662B2 (en)||Method and system for detecting signal color from a moving video platform|
|CN102722704B (en)||Method and system for recognizing vehicle license plate by integrating video dynamic tracking|
|CN106469309B (en)||The method and apparatus of vehicle monitoring, processor, image capture device|
|CN102364496B (en)||Method and system for identifying automobile license plates automatically based on image analysis|
|JP2013529348A (en)||Lighting invariant and robust apparatus and method for detecting and recognizing various traffic signs|
|CN102442311B (en)||Determine the method and apparatus of the view data of the editor relevant with vehicle-surroundings environment|
|CN100419813C (en)||Omnibearing visual sensor based road monitoring apparatus|
|CN104616021B (en)||Traffic sign image processing method and device|
|CN103065470B (en)||Detection device for behaviors of running red light of vehicle based on machine vision with single eye and multiple detection faces|
|CN100449579C (en)||All-round computer vision-based electronic parking guidance system|
|CN101950350B (en)||Clear path detection using a hierachical approach|
|CN103235938B (en)||The method and system of car plate detection and indentification|
|CN105659304A (en)||Vision augmented navigation|
|C10||Entry into substantive examination|
|SE01||Entry into force of request for substantive examination|
|C14||Grant of patent or utility model|
|C17||Cessation of patent right|
|CF01||Termination of patent right due to non-payment of annual fee||
Granted publication date: 20091202
Termination date: 20121205