CN103824452A

CN103824452A - Lightweight peccancy parking detection device based on full view vision

Info

Publication number: CN103824452A
Application number: CN201310598319.4A
Authority: CN
Inventors: 汤一平; 王辉; 俞立; 吴越; 黄磊磊; 蔡国宁; 徐邦振; 刘森森; 杨昭
Original assignee: Zhejiang University of Technology ZJUT; Enjoyor Co Ltd
Current assignee: Yinjiang Technology Co ltd; Zhejiang University of Technology ZJUT
Priority date: 2013-11-22
Filing date: 2013-11-22
Publication date: 2014-05-28
Anticipated expiration: 2033-11-22
Also published as: CN103824452B

Abstract

The invention discloses a lightweight peccancy parking detection device based on full view vision. The device comprises a full view vision sensor which is used for acquiring a video image of a wide range monitoring zone and a microprocessor which is used for analyzing and understanding the video image and carrying out peccancy parking detection. A high definition camera is connected with the microprocessor through a video interface, and carries out real-time vision detection on vehicles on a road. When the behavior of peccancy parking is detected in a vision detection range, a peccancy parking driver is told or warned not to carry out peccancy parking through a voice playing unit. If the parking time exceeds a specified time threshold, a system snapshots a peccancy vehicle, and automatically generate a peccancy parking record. According to the invention, a video image detection method which uses points to replace a side is used to reduce spatial redundancy, thus lightweight peccancy parking vision detection is realized.

Description

A kind of peccancy parking detector based on panoramic vision of lightweight

Technical field

The invention belongs to the application in intelligent transportation field of panoramic vision sensor technology, computer vision technique, image recognition technology and the network communications technology, especially the application of road parking offense context of detection.

Background technology

Along with the fast development of Chinese national economy, the surge of vehicles number, has caused transport need to increase the series of problems such as such as traffic jam too fast and that cause, and wherein motor vehicle illegal parking phenomenon is the key factor causing obstruction to traffic.

According to the investigation of traffic control department, motor vehicle illegal parking mainly contains five harm greatly: the one, and illegal parking becomes traffic to gather around resistance source.The 2nd, illegal parking becomes scratch traffic hazard chief culprit.According to statistics, the vehicle scratch accident causing because of illegal parking accounts for 38% of this class accident.The 3rd, illegal parking becomes stolen target.The 4th, illegal parking becomes disorderly to blow a trumpet and the retrograde inducement of vehicle.The 5th, illegal parking becomes the walkway obstacle that passes through.

Mainly " law on road traffic safety " implemented " administrative penalties law " and in May, 2004 about illegal parking being punished to the law relating to." law on road traffic safety " has three clauses to stipulate illegal parking.Make a summary as follows: 56 " motor vehicle should be parked in regulation place.Forbid parking motor vehicle on walkway; But, execute except the parking position of drawing according to the 33 article of regulation of this law.Temporary parking on road, must not hinder other vehicles and pedestrians." 33 " within the scope of urban road, in the situation that not affecting pedestrian, vehicle pass-through, relevant government department can execute draws a parking position." 93 " to violating law on road traffic safety rule, rules about motor vehicle parking, temporary parking regulation, can point out illegal activities, and give verbal warning, make it sail out of immediately.Though vehicle driver not at the scene or at the scene refusal sail out of immediately, hinder other vehicles, walk, locate 20 yuan of above 200 yuan of following fine.

Current processing mode is, the illegal temporary parking of motor vehicle, and traffic police or the personnel of municipal administration will point out illegal in accordance with the law and give verbal warning, and order illegal parking people to be sailed out of immediately; Illegal parking people refusal sails out of immediately or refuses and sails out of, and traffic police can punish at scene; Not at the scene, traffic police will take to shoot with video-corder evidence obtaining mode and punish the illegal temporary parking driver of motor vehicle, and law enforcement traffic police leave before the scene of shooting with video-corder, and illegal parking people returns and approves fact of malfeasance, is ready the acceptance of punishment, and traffic police can punish then and there; Traffic police does not take to shoot with video-corder evidence obtaining mode while punishing at the scene to illegal temporary parking people, should fill in " illegal parking processing advice note " folds up under motor vehicle front windscreen glass wiper, select suitable angle shot to show the motor vehicle trade mark, be placed " illegal parking processing advice note ", the illegal place of parking prohibits and stops mark or can specify the illegal picture prints such as the significant object in place, shooting date and time of parking, and should hand over Traffic Warden Subteam to preserve and implement punishment to illegal parking involved party accordingly taking evidence obtaining data in time; Traffic police shoots with video-corder evidence obtaining data and hands over before Traffic Warden Subteam, and illegal parking involved party has arrived Traffic Warden Subteam's acceptance of punishment without demur to fact of malfeasance, can punish according to " illegal parking processing advice note "; Illegal parking involved party has objection to the traffic offence fact, and traffic control department should inform that litigant accepts illegal activities processing after shooting with video-corder data input system again.Illegal parking punishment criteria is: according to " law on road traffic safety " regulation, the illegal motor vehicle of parking is suitable for non-at-scene punishment, and to driver or motor vehicle, everyone locates 200 yuan and imposes a fine and remember 2 points; Illegal temporary parking, driver refuses to sail out of, and applicable on-the-spot punishment is located 200 yuan to driver and is imposed a fine and remember 2 points; To bicycle illegal parking involved party or vehicle, everyone is warned or is located 20 yuan of fine.

But current parking offense is processed law enforcement and is existed some disadvantages at technical elements, cause the opposition between government's law enforcement and masses' illegal activities, produce being discord of society, cause showing great attention to of society." object of traffic administration is in order to guide citizen to observe traffic rules and regulations, and should, to criticize and educate as master, punishment are as auxiliary, can not manage on behalf of another to penalize, and assigns fine as extra earning means in some people's proposition." again some people propose first slight violator, should take criticize and educate and warning as main; To slight violator for the second time, can attempt imposing a fine half; To slight violator for the third time, then fine in full." allowing driver experience law-executor is in order to safeguard traffic order really, rather than for extra earning." but to realize the support that this target needs new and high technology means to a great extent.

First law-executor and parking lawbreaker's main body is all people, monitor that facing to law-executor whether having parking offense is not a nothing the matter 24 hours every days, law-executor's illegal parking will make parking lawbreaker result genuinely convinced, oral in processing in addition, need law-executor to make great efforts greatly, with patient, to tend to exceed law-executor's working limit according to the processing mode of this hommization; On the other hand, the pained money of parking lawbreaker is that main is this law enforcement mode that does not understand traffic police on the one hand.Many people think that law enfrocement official's law enforcement behavior exists many places breach of procedural law: one, law is not forbidden, allow, and oneself place of parking cars does not significantly prohibit and stops indicating, oneself stops there without mistake; Two, even if oneself belong to disorderly parking, law enforcement agency also should be by legal procedure processing in punishment process.The focus of the problems referred to above is that law enfrocement official and the litigant's that is punished one party is caused when vacant; And each parking offense event is while occurring, require law-executor and parking lawbreaker all at the scene, in operation, be difficult to realization.Here just need a middle-agent, the information symmetrical that makes law-executor and attempt parking offense between person.In the time that the person that attempts parking offense wants to park cars, middle-agent can inform that this place of driver can not park cars, and makes driver have right to know in time; Middle-agent continues to monitor, if detecting this is apprised of driver and still ignores and advise and carry out parking offense, middle-agent just collects evidence, to punish according to the fact below, within the very first time that parking offense detected, diverse network means notify law enfrocement official to process as early as possible simultaneously, owing to there being parking offense evidence, even if law enfrocement official is not in situation at the scene, also can safeguard traffic order by middle-agent, in technological means, can guarantee has statement and explaination chance to the driver who is punished simultaneously.

Therefore, intelligently detect the vehicle of parking violating the regulations, especially parking offense detection technique in roadside has obtained the concern of departments of government.Detecting for round-the-clock, high precision, intelligentized parking offense is a challenging research work.

Along with the sixties in 19th century, the rise of intelligent transportation, facility that it brings people and quick has been enjoyed in a lot of cities, and the considerable economic benefit that it brings and social benefit this be the fact that does not need dispute.Intelligent transportation system (Intelligent Transportation System, be called for short ITS) by advanced infotech, data communication transmission technology, electron controls technology and computing machine treatment technology etc. effectively integrated use in whole transportation management system, make people, car, road and environment close fit, harmony, thus set up a kind of on a large scale in, comprehensive play a role in real time, total management system accurately and efficiently.

In the research of intelligent transportation system, video detection technology is new technology of rising, and can overcome the defect of traditional detection technology, has good development prospect.Video detection technology has the following advantages: 1) sensing range is wide; A common video camera can detect the vehicles peccancy on large area road, can reduce system cost; 2) road surface close friend, is convenient to safeguard; Video camera is arranged on the top of road, therefore installs and does not safeguard and can damage road; 3) visual; Realtime graphic can be transferred to supvr, realize the function of monitoring; 4) there are good advance, extensibility, sustainable developability etc.Therefore, video parking determination and analysis technology is a kind of new trend of following parking offense detection technique development.

Chinese invention patent, number of patent application: 201310020965.2 disclose a kind of method that detects parking offense, comprising: detect the region that has moving image in video image; Extract the unique point in the region of this moving image; If the match is successful for the unique point of described extraction and pre-recorded one group of reference characteristic point, determine and have parking offense.This video parking offense detection method mainly exists the problems such as the large and accuracy of detection of calculated amount is not high.

Chinese invention patent, number of patent application: 201310020978.X discloses a kind of parking offense detection method, comprises step: the territory, prohibition parking area of specifying the every two field picture in the video sequence gathering; Carry out foreground detection, detect the target in prospect; The target that tracking detects, has judged whether that target enters territory, described prohibition parking area, is, draws the color histogram in territory, prohibition parking area, no, continues judgement; Monitor lasting duration after the changing of described color histogram, judge whether this duration is greater than Preset Time, is, judges in territory, prohibition parking area and has parking offense, no, judge in territory, prohibition parking area and there is no parking offense.This video parking offense detection method mainly exists the problems such as the large and flase drop of calculated amount, is not also suitable for the parking offense detection on road simultaneously.

Chinese invention patent, number of patent application: 201210011198.4 disclose a kind of parking offense detection method based on video monitoring, comprise: step 1: the video sequence that utilizes code book model to take camera carries out background modeling, adopt background subtraction method to obtain prospect likelihood information image; Step 2: in filtering prospect likelihood information image, other moving target is treated the interference of monitoring vehicle; Step 3: utilize parking offense decision algorithm to carry out intelligent decision to the vehicle to be monitored in prospect likelihood information image, send warning while stopping phenomenon when disobeying appears in vehicle to be monitored.This video parking offense detection method mainly exists the problems such as the large and road background modeling difficulty of calculated amount.

Chinese invention patent, number of patent application: 200710164480.5 disclose a kind of peccancy parking detector based on computer vision, comprise the omnibearing vision sensor of the video image for obtaining guarded region, for capturing the fire ball video camera of the detailed topography of parking offense vehicle information and for video image being understood to the microprocessor of analyzing and carrying out parking offense detection, microprocessor is captured the vehicle space positional information indication fire ball camera head of this parking offense by the mapping relations between large-range monitoring vision sensor and fire ball camera head to this vehicles peccancy, then captured general image violating the regulations is carried out to car plate identification, obtain the license plate number of this vehicles peccancy, then do not want parking offense by speech play unit warning parking offense driver, advise in inoperative situation in warning, system generates a parking offense record automatically.This video parking offense detection method major defect is that calculated amount parking offense large and that be not suitable on road detects.

Chinese invention patent, number of patent application: 201310004419.X discloses a kind of parking offense detection method and device based on video, the method treats by obtaining the video information that surveyed area detects, and judges according to this video information whether current this region to be detected exists sport foreground; In the time that this region to be detected exists sport foreground, carry out vehicle identification and obtain the information of vehicles of parking in this region to be detected; Obtain and treat the video information that surveyed area detects according to the time interval of setting, in this region to be detected, carry out car plate and identify the information of obtaining the vehicle of parking in this region to be detected; The information of vehicles obtaining according to each time interval, and threshold value down time of setting, will meet the car plate of this threshold value as the car plate of parking offense.This video parking offense detection method major defect is also that calculated amount parking offense excessive and that be not suitable on road detects.

Comprehensive above-mentioned prior art research is found, the entire image that existing visible detection method need to obtain video camera is conventionally carried out the processing of overall importance such as background modeling, gray processing processing, rim detection or foreground detection, this real-time that will certainly have influence on detection to massive video data processing mode, need to spend a large amount of storage resources and computational resource; Stop thering is very large difficulty pick-up unit miniaturization and for realizing disobeying.

In order to realize the roadside fast detecting parking cars violating the regulations, and meet the demand of miniaturization, low-power consumption and the lightweight of detection system, the detection system parking cars violating the regulations that must seek the lightweight that a kind of sensing range is wide, calculated amount is little, antijamming capability is strong.

Summary of the invention

The present invention is directed to some problems that exist in current parking offense video detecting method, proposed a kind of peccancy parking detector based on panoramic vision of lightweight.The technical scheme adopting is:

The peccancy parking detector based on panoramic vision of lightweight, comprises the panoramic vision sensor of the video image for obtaining guarded region and for video image being understood to the microprocessor of analyzing and carrying out parking offense detection, described vision sensor is connected with described microprocessor by described video interface, microprocessor carries out real-time vision detection by large-range monitoring panoramic vision sensor to the vehicle on road, in the time detecting that within the scope of vision-based detection, existence is against chapter parking behavior, inform or warn parking offense driver by speech play unit and do not want parking offense, for exceeding down time after official hour threshold value, system is captured this vehicles peccancy, and automatically generate a parking offense record, in parking offense record, include vehicles peccancy photo, the parking offense record of when and where violating the regulations sends to the processing server violating the regulations of vehicle supervision department by network, processing server violating the regulations carries out car plate identification to captured vehicles peccancy general image, analysis obtains the license plate number of this vehicles peccancy, and it is single automatically to generate parking offense processing according to the single call format of processing violating the regulations, finally remind managerial personnel to confirm to process, described microprocessor comprises:

Panoramic picture data acquisition module, for reading the video image information obtaining from panoramic shooting device;

The customized module in the demarcation of panoramic vision sensor and track, roadside, for customizing the no-parking zone on monitored road and setting up a kind of material picture of space and the corresponding relation of the video image obtaining;

Automatic generation, gray-scale value inspection and the sampling point position fine setting module of sampled point, be used for the automatic uniform sampled point of the span within the scope of the track, roadside customizing, the sampled point gray-scale value generating is carried out to consistency check, the sampled point that departs from gray-scale value is carried out to locus adjustment;

There is sampled point detecting unit, detect for the foreground object of satisfying the need on sidecar road, specifically adopt background subtraction method from sampled point image, to detect the exist sampled point of reflection vehicle at track, roadside spatial distribution state;

The static sampled point detection module that exists, for detection of the static sampled point that exists on track, roadside; There is the time-space relationship of sampled point and mobile sampled point in utilization, obtains the static sampled point that exists of the static foreground object of reflection on track, roadside;

Vehicles peccancy detection module based on sampled point, for detection of the parking offense vehicle existing on road; There is sampled point to process to obtain the static piece that exists to static, and then it is carried out to the coupling of car modal, if the match is successful with regard to preliminary judgement for have stationary vehicle on road; If detect and exist stationary vehicle on road, according to the locus of this stationary vehicle, whether the front of detecting this stationary vehicle has stationary object and detects the locus of stationary vehicle on road, if near the roadside of stationary vehicle in road and stationary vehicle front does not exist stationary object, and the residence time of this stationary vehicle exceedes official hour threshold value, be just judged to be suspicious violating the regulations parking cars;

The identification module of vehicles peccancy is also the follow-up major key to this vehicle peccancy processing procedure vehicles peccancy information retrieval in accordance with the law for the vehicle license plate number of identifying parking offense; This module operates on the processing server violating the regulations of vehicle supervision department;

Can monitor scope video information large as far as possible in monitoring scene in order to reach, a solution of the present invention is to adopt panoramic vision sensor, and panoramic vision sensor becomes minute surface and the camera lens of angle just to form towards the video camera of minute surface by two; Angle between two minute surfaces is 180 ° of-2 γ, and two minute surfaces width value on front elevation is W, height value on side view is R, and the width value W of two minute surfaces and height value R are positioned at the areas imaging of video camera; On side view, the central shaft of described video camera becomes η angle with the central shaft of described vertical rod, and minute surface becomes ε angle with the surface level direction of road side; On front elevation, the angle of minute surface and described vertical rod is 90 °-γ, the central shaft of video camera and the central axes of vertical rod, and the focal length of video camera is f;

The setting height(from bottom) of panoramic vision sensor is H, and road surface is L along the visual range of road direction, 180 ° of-2 γ of angle between two minute surfaces, and formula (1) is the relation of H, L value and γ,

γ=(tan ^-1(L／H)-ω _max×H)／2 (1)

In formula, γ represents the angle of minute surface and surface level, and L is panoramic vision sensor along the visual length on surface level direction road, the setting height(from bottom) that H is panoramic vision sensor, ω _maxfor the maximum visual angle of video camera.

Further, the maximum visual angle ω of described video camera _maxit is 45 °, the setting height(from bottom) H of panoramic vision sensor is 3 meters, the visual length L along on road direction of panoramic vision sensor is greater than 200 meters, the angle γ that tries to achieve minute surface and surface level by formula (1) is 32 °, the length of minute surface is greater than W2 × cos (γ), the width of every minute surface is greater than Rcos (ε-η), and ε is the angle of the surface level direction of minute surface and road side, and η is the angle between video camera central shaft and vertical rod central shaft;

In order to obtain large as far as possible video sensing range information, another kind of solution of the present invention is to adopt common vision sensor, vision sensor is configured in to about 10 meters of the tops place in track, roadside, parallel with track, roadside direction, aim at track, roadside to declivity, the transverse axis of the imaging plane of vision sensor is parallel with ground level simultaneously, so also can obtain the video information in track, long and narrow scope roadside;

The customized module in the demarcation of described panoramic vision sensor and track, roadside, for to camera calibration, determines the mapping relations of two-dimensional imaging plane and three dimensional space coordinate point.In addition, parking offense region is customized, facilitate follow-up processing.For the parking offense on road, vehicles peccancy is all parked on track, roadside, therefore detects for the parking offense on road, in the present invention, the region-of-interest of parking offense is customized on track, roadside; The method for customizing in parking offense region is: the image first obtaining from vision sensor, select track, two roadsides marginal point according to roadside lane markings nearby in track, roadside perpendicular to track, roadside direction, then select track, two roadsides marginal point in the distant place in track, roadside on perpendicular to track, roadside direction, these four marginal points are connected and composed to lane detection region, roadside; Finally described vision sensor is demarcated;

Here adopt odd coordinate to demarcate vision sensor, computing formula as the formula (2),

[\begin{matrix} λ_{i} x_{i} \\ λ_{i} y_{i} \\ λ_{i} \end{matrix}] = [\begin{matrix} b_{11} & b_{12} & b_{13} & b_{14} \\ b_{21} & b_{22} & b_{23} & b_{24} \\ b_{31} & b_{32} & b_{33} & b_{34} \end{matrix}] [\begin{matrix} x \\ y \\ z \\ 1 \end{matrix}] - - - (2)

In formula, (x _i, y _i) be the position of pixel on the plane of delineation, (x, y, z) is the position on road ground, in vision sensor calibration process, chooses 6 known points and tries to achieve parameter b _ij; In order to solve parameter nonuniqueness problem, stipulate b here ₃₄=1; Improve real-time consideration from simplifying to calculate, ignore the effect of altitude of foreground object in scene here, as the impact of the height of vehicle, i.e. z=0, therefore, the problem of calibrating of video camera is just reduced to the mapping relations problem between road plane and imaging plane of setting up;

In the time of customization track, roadside, on the plane of delineation, select tetragonal four summits on track, roadside, obtained four marginal point coordinate informations, then measure and obtain the tetragonal width in track, roadside and the length value of customizing by reality, solve four prescription formulas according to its coordinate figure, try to achieve parameter b _ij; Adopt formula (3) to realize the demarcation of vision sensor,

\{\begin{matrix} x_{i} = \frac{b_{11} x + b_{12} y + b_{14}}{b_{31} x + b_{32} + 1} \\ y_{i} = \frac{b_{21} x + b_{22} y + b_{24}}{b_{31} x + b_{32} + 1} \end{matrix} - - - (3)

In formula, b _ijfor calibrating parameters, (x _i, y _i) be the position of pixel on the plane of delineation, (x, y, z) is the position on road ground.

Automatic generation, gray-scale value inspection and the sampling point position fine setting module of described sampled point, be used for the uniform sampled point of the automatic span in track, roadside customizing, the sampled point gray-scale value generating is carried out to consistency check, the sampled point that departs from gray-scale value is carried out to locus adjustment;

Owing to comprising a large amount of redundant informations in video image, in order to reduce the calculated load of video image processing and storage load, in the present invention, adopt the detection mode of the sampled point acquiring a special sense to detect the vehicle of parking violating the regulations on road.There is two states from the angle that has or not foreground object in the sampled point for track on road, has sampled point, exists the sampled point of foreground object; , there is not the sampled point of foreground object in the non-sampled point that exists; For there is sampled point, divide from seasonal effect in time series angle, can be divided into static sampled point and the mobile sampled point that exists of existing.The key that parking offense detects is just to detect the static sampled point that exists, and according to the static residence time that has sampled point, judges the vehicle of parking violating the regulations.Due to will be from image direct-detection go out that static to exist sampled point to exist more difficult, adopt in the present invention and from image sequence, detect and calculate the mobile sampled point that exists, then exist sampled point to calculate the static sampled point that exists according to having sampled point and moving, finally obtain the vehicle of parking violating the regulations according to the static distribution situation of sampled point and the residence time of existing; About the concrete classification of sampled point and the static computation process that has a sampled point as shown in Figure 6;

By the demarcation of vision sensor, the mapping relations of the pixel on point and the plane of delineation on the road of space are set up; In addition, in road travel direction, narrowing from the interval of video camera position sampling point far away, broadening from the interval of the position sampling point close to video camera.Because configure like this sampled point, local vehicle area occupied in image close to video camera is large, even if relatively dredging, sampled point also can detect the driving states of vehicle, but vehicle is because area occupied in image is very little in the distance, if sampled point configuration ground is not the very close state that just can not detect exactly vehicle.For this situation, determine in advance the distribution interval of sampled point according to the total higher limit of sampled point.By such setting, in guaranteeing the state of the vehicle that detects a distant place, can shorten the Check processing time.Therefore, can guarantee the region of the distant place detecting from video camera, and according to the setting of the upper limit number of sampled point, Check processing energy high speed, and do not need special hardware device.In addition, the resolution of video camera can arrange as fixed number, therefore can adopt the video camera of various different resolutions.In order to detect exactly the car status information on road, require on track, roadside, evenly to generate sampled point, the sampled point of generation projects to be spaced apart 0.5 meter on real road;

Further, the sampled point gray-scale value inspection customizing, after sampled point on customization track, roadside and track, roadside, consider by image processing techniques and sampled point further will be divided into and be had sampled point and the non-sampled point that exists, distinguishing the two is to be undertaken by the gray threshold of sampled point; Gray-scale value on road reaches unanimity substantially, therefore adds up and there is no the gray-scale value of all sampled points on track, roadside under vehicle condition and asking its mean value

as initial background value

as the non-gray-scale value that has sampled point;

Further, consider on track, roadside and have some road signs, for example right-hand rotation waits identifier, and gray-scale value and the road surface of these identifiers differ greatly, and can bring detection error to follow-up context update if sampled point just in time drops in road sign; Therefore, need to carry out gray-scale value inspection to the sampled point of all customizations, if the gray-scale value of some sampled points

depart from its initial background value

will change the position of this sampled point, the method for change is perpendicular to progressively mobile sampled point of vehicle heading, makes the gray-scale value of this sampled point

meet travel through after all sampled points, will

as the initial background gray-scale value of each sampled point here by track, roadside by vehicle heading be divided into length ratio be 2:3:5 far away, in, nearly three sections of regions, the average generation line number sampled point consistent with columns each region in;

The described sampled point detecting unit that exists, detect for the foreground object of satisfying the need on sidecar road, in this invention, adopt background subtraction method from sampled point image, to detect the sampled point that exists in tn moment, extract and represent the exist sampled point of foreground object in track, roadside space distribution; Meet

the sampled point of condition is just judged to be to exist sampled point, otherwise is the non-sampled point that exists, and obtains existing sampled point image En and the non-sampled point image that exists

exist the detailed process of detection module of sampled point as follows:

Step1: the benchmark gray level image B that sets sampled point ₀gray threshold TH1 with binaryzation;

Step2: obtain t _nthe gray level image X of moment sampled point _n;

Step3: according to the difference between formula (4) calculating sampling dot image and benchmark gray level image, obtain the background subtraction partial image D of sampled point _n;

D _n=X _n-B ₀ (4)

Step4: benchmark gray level image is carried out to context update, obtain the background model corresponding with present frame;

Step5: be used in the threshold value TH1 setting in Step1 to D _ncarry out binary conversion treatment, obtain existing sampled point bianry image ES _n, at ES _nin all sampled points all have 1 and 0 two states, 1 represents have foreground object to exist on this sampled point, has sampled point; 0 represents not have foreground object, i.e. the non-sampled point that exists on this sampled point;

Consider that Vehicle Object is a rigid body, the variation of size and shape can not occur in motion process, so can think that vehicle can describe with several adjacent sampling blocks that exists that has sampled point composition at any time; In practical application, can there is the gray-scale value situation close with road ground gray-scale value at some position of vehicle, thereby cause and will exist sampled point to be mistaken for the non-situation that has sampled point; In addition, the legacy on road, pedestrian and non motorized vehicle also can produce certain interference to testing result; These interference are corrected by isolated sampled point filter algorithm; Therefore the present invention eliminates error detection or the interference of above-mentioned sampled point by filter algorithm.Filter algorithm thought is: if the state of most of sampled points adjacent with isolated sampled point and the opposite states of this isolated sampled point can think that these sampled points are error detection or interference, need to put anti-rectification to it;

The described static sampled point detection module that exists, for detection of the static sampled point that exists on track, roadside; Because the static sampled point that exists is difficult for direct-detection out, in the present invention, first adopt frame-to-frame differences algorithm from image sequence, to detect the mobile sampled point that exists, then exist sampled point to calculate the static sampled point that exists according to having sampled point and moving, its testing process is as follows:

Step1: the image of not taking in the same time under Same Scene is carried out difference and can be obtained the pixel of the changing unit in two width images, obtain difference image, computing method as the formula (5):

Z _1n=X _n-X _n-α （5）

In formula, X _nand X _n-αbe respectively t _nand t _n-αthe gray-scale value of the each sampled point in the sampled images in moment, Z _1nfor difference image, referred to herein as the first difference image, it has represented to experience each sampled point situation of change on the road after the α time; Comprised the situation of change of the two states of sampled point at the first difference image, i.e. from " 1 " to " 0 " or the variation from " 0 " to " 1 ", be confirmed whether it is mobilely to have sampled point, also needs to observe t _nand t _{n+ α}the situation of change of the gray scale of the each sampled point in the sampled images in moment, obtains the second difference image, and computing method as the formula (6);

Z _2n=X _n+α-X _n （6）

In formula, X _{n+ α}and X _nbe respectively t _nand t _{n+ α}the gray-scale value of the each sampled point in the sampled images in moment, Z _2nfor difference image, referred to herein as the second difference image, represent to experience each sampled point situation of change on the road after the α time;

Step2: use respectively threshold value TH1 to the first difference image Z _1nwith with threshold value TH2 to the second difference image Z _2nprocess, obtain respectively First Characteristic and extract image T _1nextract image T with Second Characteristic _2n;

Step3: First Characteristic is extracted to image T _1nextract image T with Second Characteristic _2ncarrying out and computing, there is sampled point array in the movement that utilizes formula (7) to try to achieve in image;

Y _n=T _1n∧T _2n （7）

Step4: will have sampled point binary image F _nwith the mobile bianry image Y that has sampled point _nmake difference operation, utilize formula (8) to calculate the static bianry image S that has sampled point _n;

S _n=F _n-Y _n （8）

The described vehicles peccancy detection module based on sampled point, for detection of the parking offense vehicle existing on road; First exist sampled point to process to obtain the static piece that exists, then it is carried out the coupling of car modal to static, if the match is successful with regard to preliminary judgement for have stationary vehicle on road; If detect and exist stationary vehicle on road, according to the locus of this stationary vehicle, whether the front of detecting this stationary vehicle has stationary object and detects the locus of stationary vehicle on road, if near the roadside of stationary vehicle in road and stationary vehicle front does not exist stationary object, and the residence time of this stationary vehicle exceedes the threshold value of regulation, be just judged to be suspicious violating the regulations parking cars;

First, the satisfy the need static sampled point image S that exists of sidecar road space distribution _ncarry out filtering processing with auto model, remove some isolated nonstatics and have sampled point and other interference, obtain the static piece that exists that reflects that vehicle temporarily stops or being detained; Here the static sampled point image S that exists mainly formula (8) being calculated _ncarry out filtering processing, filtering processing and parking offense vehicle detecting algorithm are as follows:

Step1: set the size of car modal, adopt 3 × 5 template, vehicle occupies in a lateral direction 3 sampled points, occupies 5 sampled points on the longitudinal direction of vehicle, and decision threshold is set;

Step2: with car modal to the static sampled point image S that exists _nin region, track, roadside travel through to the other end in track, roadside from the one end in track, roadside, just judge that this region is as stationary vehicle if detect the static sampled point that exists having more than 50% with certain region in ergodic process; If exist stationary vehicle, temporarily preserve the residing spatial positional information of stationary vehicle and time of origin, the stationary vehicle of simultaneously satisfying the need in region, sidecar road is counted, and obtains count value Count;

Step3: judge whether Count=0 condition meets, if met, removes the stationary vehicle record of preserving in storage unit, forwards Step6 to;

Step4: judge whether Count=1 condition meets, if do not meet and forward Step5 to, otherwise temporarily to preserve the residing spatial positional information of stationary vehicle as matching condition, check the record that whether has had this stationary vehicle in storage unit; If existed corresponding record, the time of origin T in being recorded _loc, according to system clock time T _syscalculate the static duration T of this stationary vehicle _tur=T _sys-T _loc; If T _tur<T _alarm, system is play the advice language of " please not stopping on track " automatically; If T _alarm≤ T _tur<T _vio, system is play the warning clause of " on track, Strictly no parking, otherwise will punish by the law on road traffic safety " automatically; If T _tur>T _viosystem is play the warning clause of " law on road traffic safety has been violated in your parking offense behavior; relevant portion will be punished by the law on road traffic safety " automatically, and automatically capture the image of vehicles peccancy, candid photograph image and record violating the regulations are saved in a record violating the regulations, record violating the regulations has comprised time, place and the candid photograph image of parking offense, and send to the processing server violating the regulations of vehicle supervision department by network, so that the identification module of the vehicles peccancy in the processing server violating the regulations in vehicle supervision department is processed; Forward Step6 to;

Step5: be judged as the existence of the stationary vehicle causing due to congestion in road or traffic hazard, capture image scene and send to relevant departments to confirm processing;

Step6: EOP (end of program).

Beneficial effect of the present invention is mainly manifested in: video information on a large scale that 1) can the whole road of real-time collecting, there is sensing range wide, and can automatically detect the vehicle parking act of violating regulations on road within the scope of 200 rice diameters; 2) intellectuality that parking offense detects, the robotization of parking offense processing, the hommization of processing procedure violating the regulations have been realized; 3) adopt the mode of sampled point modeling to make calculated amount and memory space will reduce hundred times than original technology, be conducive to realize in embedded system; 4) effectively improve the dynamics that parking offense is enforced the law, reduced parking offense law enfrocement official's working strength, realized the real time automatic detection of parking offense, automatic transmission, real-time issue automatically and real-time processing automatically in real time; 5) alleviate the opposition between law-executor and parking lawbreaker, made " administrative penalties law " and " law on road traffic safety " on technological layer, have more operability, increased social harmony; 6) guiding citizen observe traffic rules and regulations, and really make parking offense driver experience the seriousness of law, improve the civilization degree of entire society.

Accompanying drawing explanation

Fig. 1 is high-definition camera is taken road condition schematic diagram from front;

Fig. 2 is near the sampled point customization schematic diagram on track, roadside, a) customization of the sampled point on track and the track on the plane of delineation of high-definition camera schematic diagram, b) customization of the sampled point on track and the track in real road plane schematic diagram;

Fig. 3 is that panoramic vision sensor is taken the schematic diagram near track, roadside state from the side;

Fig. 4 is the optical imaging concept figure of panoramic vision sensor;

Fig. 5 is the core algorithm key diagram of lightweight parking offense vision-based detection;

Fig. 6 is the computing method key diagram of the static sampled point of reflection parking offense vehicle;

Fig. 7 is the hardware structure diagram of the peccancy parking detector based on panoramic vision of lightweight;

Fig. 8 is the computation process key diagram of the static sampled point of reflection parking offense vehicle;

Fig. 9 be track, roadside with the PC of the sampled point customization function on track, roadside and embedded holographic road traffic state vision inspection apparatus between the hardware block diagram of communicating by letter;

Figure 10 be track, roadside with the PC of the sampled point customization function on track, roadside and embedded holographic road traffic state vision inspection apparatus between the schematic diagram of communicating by letter;

Figure 11 is that the software in the sampled point customization function PC on track, roadside and track, roadside is processed block diagram.

Embodiment

Below in conjunction with accompanying drawing, the invention will be further described.

Embodiment 1

With reference to Fig. 1,2,5～11, a kind of peccancy parking detector based on panoramic vision of lightweight, comprise the high-definition camera that is arranged on each measurement point on road, high-definition camera is configured in to about 10 meters of the tops place near track, roadside, parallel with track direction, aim at track to declivity, the transverse axis of the imaging plane of high-definition camera is parallel with ground level simultaneously, as shown in Figure 1.

For the microprocessor that carries out parking offense detection according to the video data of high-definition camera, described microprocessor comprises microprocessor system hardware and microprocessor software, it is characterized in that: described microprocessor hardware is made up of CPU, video memory, input block, storage unit, speech play unit, communication unit, video interface, RAM (Random Access Memory) and ROM (Read-only Memory), as shown in Figure 7; Described high-definition camera is connected with described microprocessor by described video interface, described delivery unit sends to by described communication unit the testing result of parking offense vehicle the processing server violating the regulations of vehicle supervision department through network, corresponding record data violating the regulations are identified and generated to processing server violating the regulations by the identification module of vehicles peccancy to the identity of parking offense vehicle; Below in conjunction with the annexation of each processing module in the lightweight parking offense vision-based detection treatment scheme explanation microprocessor of accompanying drawing 5; Described microprocessor software comprises:

Road background modeling module based on sampled point, for satisfying the need, sidecar road background is carried out modeling;

The customized module in the demarcation of described panoramic vision sensor and track, roadside, for high-definition camera is demarcated, determines the mapping relations of two-dimensional imaging plane and three dimensional space coordinate point.In addition, parking offense region is customized, facilitate follow-up processing.For the parking offense on road, vehicles peccancy is all parked on track, roadside, therefore detects for the parking offense on road, in the present invention, the region-of-interest of parking offense is customized on track, roadside, as the dotted portion in accompanying drawing 3, dotted portion represents track, roadside; The method for customizing in parking offense region is: the image first obtaining from high-definition camera, select track, two roadsides marginal point a according to roadside lane markings nearby in track, roadside perpendicular to track, roadside direction ₁and a ₂, then select track, two roadsides marginal point b in the distant place in track, roadside on perpendicular to track, roadside direction ₁and b ₂, these four marginal points are connected and composed to lane detection region, roadside, the dashed region in accompanying drawing 3; Finally high-definition camera is demarcated;

Here adopt odd coordinate to demarcate high-definition camera, computing formula as the formula (2),

[\begin{matrix} λ_{i} x_{i} \\ λ_{i} y_{i} \\ λ_{i} \end{matrix}] = [\begin{matrix} b_{11} & b_{12} & b_{13} & b_{14} \\ b_{21} & b_{22} & b_{23} & b_{24} \\ b_{31} & b_{32} & b_{33} & b_{34} \end{matrix}] [\begin{matrix} x \\ y \\ z \\ 1 \end{matrix}] - - - (2)

In formula, (x _i, y _i) be the position of pixel on the plane of delineation, (x, y, z) is the position on road ground, in high-definition camera calibration process, chooses 6 known points and tries to achieve parameter b _ij; In order to solve parameter nonuniqueness problem, stipulate b here ₃₄=1; Improve real-time consideration from simplifying to calculate, ignore the effect of altitude of foreground object in scene here, as the impact of the height of vehicle, i.e. z=0, therefore, the problem of calibrating of video camera is just reduced to the mapping relations problem between road plane and imaging plane of setting up;

In the time of customization track, roadside, on the plane of delineation, select tetragonal four summit a on track, roadside ₁, a ₂, b ₁and b ₂, obtain four marginal point coordinate informations, then measure and obtain the tetragonal width in track, roadside and the length value of customizing by reality, solve four prescription formulas according to its coordinate figure, try to achieve parameter b _ij; Adopt formula (3) to realize the demarcation of vision sensor,

\{\begin{matrix} x_{i} = \frac{b_{11} x + b_{12} y + b_{14}}{b_{31} x + b_{32} + 1} \\ y_{i} = \frac{b_{21} x + b_{22} y + b_{24}}{b_{31} x + b_{32} + 1} \end{matrix} - - - (3)

By the demarcation of high-definition camera, the mapping relations of the pixel on point and the plane of delineation on the road of space are set up; In addition, in road travel direction, narrowing from the interval of video camera position sampling point far away, broadening from the interval of the position sampling point close to video camera, as accompanying drawing 2a) as shown in.Because configure like this sampled point, local vehicle area occupied in image close to high-definition camera is large, even if relatively dredging, sampled point also can detect the driving states of vehicle, but vehicle is because area occupied in image is very little in the distance, if sampled point configuration ground is not the very close state that just can not detect exactly vehicle.For this situation, determine in advance the distribution interval of sampled point according to the total higher limit of sampled point.By such setting, in guaranteeing the state of the vehicle that detects a distant place, can shorten the Check processing time.Therefore, can guarantee the region of the distant place detecting from high-definition camera, and according to the setting of the upper limit number of sampled point, Check processing energy high speed, and do not need special hardware device.In addition, the resolution of high-definition camera can arrange as fixed number, therefore can adopt the video camera of various different resolutions.In order to detect exactly the car status information on road, require on track, roadside, evenly to generate sampled point, the sampled point of generation projects to be spaced apart 0.5 meter on real road;

as initial background value

as the non-gray-scale value that has sampled point;

depart from its initial background value will change the position of this sampled point, the method for change is perpendicular to progressively mobile sampled point of vehicle heading, makes the gray-scale value of this sampled point

meet

travel through after all sampled points, will

as the initial background gray-scale value of each sampled point here by track, roadside by vehicle heading be divided into length ratio be 2:3:5 far away, in, nearly three sections of regions, as shown in Figure 2, the average generation line number sampled point consistent with columns each region in;

The described road background modeling module based on sampled point, for satisfying the need, sidecar road background is carried out modeling; Owing to having customized equably sampled point on track, roadside in the time customizing sampled point, background subtraction point-score can be used for detection and has sampled point, but background subtraction point-score requires to obtain reliable, stable sampled point background gray levels; The sampled point in road customization region is subject to impact that the external environment such as illumination, weather changes greatly, need to carry out real-time update to sampled point background gray levels; The present invention adopts from existing the nearest non-gray-scale value that has sampled point of sampled point to upgrade the background that has sampled point, realizes a kind of lightweight, accurate efficiently background update method, update algorithm as shown in Equation (16),

B_{n + 1}^{i} = \{\begin{matrix} B_{n}^{i} + k (X_{n}^{i} - B_{n}^{i}) & , | X_{n}^{i} - B_{n}^{i} | < TH 1 \\ B_{n}^{\min (index (q) - index (p))} & , else \end{matrix} - - - (16)

In formula, for having the gray-scale value of the nearest non-existence sampling of sampled point from this,

for tn moment sampled point actual measurement gray-scale value,

for tn moment sampled point background gray levels,

for tn+1 moment sampled point background gray scale predicted value;

the sampled point of condition is just judged to be to exist sampled point, otherwise is the non-sampled point that exists, and obtains existing sampled point image E _nwith the non-sampled point image E that exists _n.Contrast accompanying drawing 8, exists the detailed process of detection module of sampled point as follows:

Step2: obtain t _nthe gray level image X of moment sampled point _n;

D _n=X _n-B ₀ (4)

The described static sampled point detection module that exists, for detection of the static sampled point that exists on track, roadside; Because the static sampled point that exists is difficult for direct-detection out, in the present invention, contrast accompanying drawing 8, first adopt frame-to-frame differences algorithm from image sequence, to detect the mobile sampled point that exists, then exist sampled point to calculate the static sampled point that exists according to having sampled point and moving, its testing process is as follows:

Z _1n=X _n-X _n-α （5）

Z _2n=X _n+α-X _n （6）

Y _n=T _1n∧T _2n （7）

S _n=F _n-Y _n （8）

Step4: judge whether Count=1 condition meets, if do not meet and forward Step5 to, otherwise temporarily to preserve the residing spatial positional information of stationary vehicle as matching condition, check the record that whether has had this stationary vehicle in storage unit; If existed corresponding record, the time of origin T in being recorded _loc, according to system clock time T _syscalculate the static duration T of this stationary vehicle _tur=T _sys-T _loc; If T _tur<T _alarm, system is play the advice language of " please not stopping on track " automatically; If T _alarm≤ T _tur<T _vio, system is play the warning clause of " on track, Strictly no parking, otherwise will punish by the law on road traffic safety " automatically; If T _tur>T _viosystem is play the warning clause of " law on road traffic safety has been violated in your parking offense behavior; relevant departments will punish by the law on road traffic safety " automatically, and automatically capture the image of vehicles peccancy, candid photograph image and record violating the regulations are saved in a record violating the regulations, record violating the regulations has comprised time, place and the candid photograph image of parking offense, and send to the processing server violating the regulations of vehicle supervision department by network, so that the identification module of the vehicles peccancy in the processing server violating the regulations in vehicle supervision department is processed; Forward Step6 to;

Step6: EOP (end of program).

Embodiment 2

With reference to Fig. 3～Figure 11, as preferred another kind of scheme: high-definition camera is arranged on to roadside, as shown in Figure 3; High-definition camera adopts panoramic vision sensor, and described panoramic vision sensor becomes minute surface and the camera lens of angle just to be formed towards the video camera of minute surface by two; Angle between two minute surfaces is 180 ° of-2 γ, and two minute surfaces width value on front elevation is W, height value on side view is R, and the width value W of two minute surfaces and height value R are positioned at the areas imaging of video camera; On side view, the central shaft of described video camera becomes η angle with the central shaft of described vertical rod, and minute surface becomes ε angle with surface level direction road side; On front elevation, the angle of minute surface and described vertical rod is 90 °-γ, the central shaft of video camera and the central axes of vertical rod, and the focal length of video camera is f;

The setting height(from bottom) of panoramic vision sensor is H, and road surface is L along the visual range of road direction, 180 ° of-2 γ of angle between two minute surfaces, and formula (1) is the relation of H, L value and γ, geometric relationship is as shown in Figure 4;

γ = (\tan^{- 1} (L / H) - {\overset{'}{ω}}_{\max} \times H) / 2 - - - (1)

In formula, γ represents the angle of minute surface and surface level, and L is panoramic vision sensor along the visual length on surface level direction road, the setting height(from bottom) that H is panoramic vision sensor, for the maximum visual angle of video camera.

Further, the maximum visual angle of described video camera it is 45 °, the setting height(from bottom) H of panoramic vision sensor is 3 meters, panoramic vision sensor be 100 meters along the visual length L on road direction, the angle γ that tries to achieve minute surface and surface level by formula (1) is 32 °, the length of minute surface is greater than W/2 × cos (γ), and the width of every minute surface is greater than R/cos (ε-η).

Other structures and the course of work of the present embodiment are identical with embodiment 1.

Embodiment 3

With reference to accompanying drawing 9～11, all the other are identical with enforcement 2 with enforcement 1, and difference is that the automatic generation of the customization in track, roadside, sampled point and the detection of parking offense analyzing and processing are completed respectively on two distinct devices; Wherein, being automatically created on PC and completing of the customization in track, roadside, sampled point, parking offense analyzing and processing detects and completes on embedded device; PC and the analyzing and processing of customization use detect with adopting SOCKET communication mode between embedded device, as shown in Figure 9; Embodiment is: the roadside lane information first customizing at PC and track, roadside up-sampling dot information, and the treatment scheme of track customization and generation sampled point is as shown in Figure 11; Then, communicate by letter and send embedded device to by SOCKET, as shown in Figure 10; Finally, embedded device reads after the information such as customization, constantly cycle analysis detects the parking offense vehicle on road, and testing result is sent to the processing server violating the regulations of vehicle supervision department by described communication unit, so that the identification module of the vehicles peccancy in the processing server violating the regulations in vehicle supervision department is processed;

The identification module of described vehicles peccancy is also the follow-up major key to this vehicle peccancy processing procedure vehicles peccancy information retrieval in accordance with the law for the vehicle license plate number of identifying parking offense.Capture image and relative recording for the parking offense sending over from scene, the processing server violating the regulations in vehicle supervision department is identified and is analyzed captured vehicles peccancy image; Adopting in the present invention the car plate identifying schemes based on gray level image, is mainly to utilize the large feature of car plate shade of gray changing features.Comprise following step: image frame grabber process, car plate pre-service and coarse positioning, thin location, slant correction and the cutting procedure of car plate, the automatic identifying of car plate;

Car plate pre-service and coarse positioning.In the present invention, adopting the car plate identifying schemes based on gray level image, is mainly to utilize the large feature of car plate shade of gray changing features.Its flow process is as follows: the image first video camera being obtained carries out binary conversion treatment, in image, the grey scale change of license plate area is the most obvious, binary image after the impact of filtering car light, car plate position is that a bottom horizontal projection is than the region of comparatively dense, car plate upper-lower position is positioned at two minimum wave trough position of projection from top to bottom, utilize this point can determine the upper-lower position of car plate: on the basis of horizontal projection, to carry out again afterwards vertical projection, the minimum wave trough position in two ends is the position, left and right of car plate, so just determine four coordinate points of car plate, thereby just car plate location positioning is got off.

Horizontal projection P _xand vertical projection P (y) _y(x) calculate with formula (9) and (10);

P_{x} (y) = Σ_{x = 1}^{L} f (x, y) - - - (9)

P_{y} (x) = Σ_{y = 1}^{H} f (x, y) - - - (10)

In formula, f (x, y) is the image slices vegetarian refreshments after binaryzation, the width that L, H are image and height.

In whole identifying, should use global characteristics rather than local feature as far as possible, here propose a kind of improved preprocess method and in image block, carry out 4 rank greatest gradient level error point-scores based on Sobel operator, can effectively solve the car light interference problem of positioning stage, improve the positioning precision of car plate.Specific algorithm flow process as shown in Figure 5, is described in detail as follows:

(1) due to car light open at night after through gray scale strengthen, car light regional luminance is large, and is the region of similar circle, does not meet the feature of license plate area rectangle.Judge in advance the number that in this rectangle frame, gray-scale value is greater than 230 with a moving window, if exceed 0.85 times of the total gray scale number of this rectangle frame, think that this region is a car light region, then 4 ladder degree maximum horizontal difference processing are carried out in this region, then use Sobel operator Edge detected.Otherwise, judge without car light and disturb, directly carry out rim detection with Sobel operator;

(2) because car plate is a rectangle, and inner horizontal gradient maximum, therefore these chapters and sections carry out blocked scan to two field picture, first calculate the number that in this rectangle frame, gray-scale value is greater than 230, if exceed 0.85 times of the total gray scale number of this rectangle frame, distinguish again the horizontal gradient value in unit of account piece, in actual applications, car plate depth-width ratio is 3.14:1, system None-identified in the time that the car plate height collecting is less than 20 pixels, therefore getting height is here 20 pixels, wide is searching in image of 60 pixels, horizon scan initial step length is made as 5 pixels, while upwards search, step-length is 3 pixels,

(3) carrying out the timesharing of gradient level error, we are divided into 4 × 4 block of pixels to moving window, then use the maximum horizontal gradient of 4 × 4 block of pixels to replace the maximum horizontal gradient of whole 4 × 4 block of pixels.In 4 × 4 block of pixels, three kinds of operators are used here, as shown in Figure 6; The horizontal gradient of first row pixel in computing block, maximum horizontal gradient, as the horizontal gradient of this piece, then replaces the gray-scale value of 4 × 4 by this gradient, be therefore called 4 rank greatest gradient level error point-scores;

The thin location of car plate.Car plate after coarse positioning is because the car plate that has automobile brand printed words etc. to cause on uneven illumination or license plate to cut apart is greater than positive constant width or high size, may have on the other hand the impact of the brand mark of car or the rivet of car plate top due to the bottom of car plate own, sometimes car plate side has the words such as some advertisements, all likely cause the orientation range of car plate to become large, at this moment must carry out to car plate the correction of upper and lower and left and right, it is fine positioning process, in fine positioning process, we can use gray scale enhancing, license plate binary, car plate horizontal and vertical shadow casting technique, to car plate is carried out to the correction of upper and lower and left and right,

First car plate is carried out to binaryzation, the brightness of most of characters on license plate is greater than bottom, as black matrix wrongly written or mispronounced character or wrongly written or mispronounced character of the blue end, and the gray scale 255 of character after binaryzation like this, background is 0.But the brightness of part characters on license plate is less than bottom, as white gravoply, with black engraved characters, the gray scale 0 of character after binaryzation, background is 255.Can judge car plate type by histogram analysis, white gravoply, with black engraved characters in this way, we carry out inverse processing to such car plate, and treatment scheme is very simple: what original pixel value was 0 resets to 255, and what original pixel value was 255 resets to 0;

Car plate after binaryzation is the upper-lower position of fine positioning car plate first, and revising is up and down mainly the vehicle brand character carrying in order to eliminate car plate below, and algorithm steps is as follows:

Step1: horizontal projection: the projection value iResult[(int of projection in the middle of calculating) (Height/2)], wherein iResult[i] be i point horizontal projection value, Height is car plate height, projection value iResult[i in the middle of getting here], iResult[i+l], iResult[i-1] mean value iR;

Step2: calculate iResult:[0]≤1, iResult[1]≤1, iResult[2]≤1 whether all meet, if all meet, car plate lower position is Height point, below does not need to revise, and forwards Step4 to;

Step3: from (int) (Height/2) to going up most search, when meeting iResult[k] <iR/13 or iResult[k] when >2*iR, k point position is car plate lower position;

Step4: calculate below iResult[Height-1]≤1, iResult[Height-2]≤1, iResult[Height-3]≤1 whether all meet, if all meet, car plate top position is 0 point, top does not need to revise, and forwards Step6 to;

Step5: from (int) (: Height/2) to descending most search, when meeting iResult[p] <iR/13 or iResult[k] when >2+iR, P point position is car plate top position;

Step6: finish.

It is mainly the car plate redundance causing according to unequal very strong noise for ambient light that vertical projection is carried out left and right correction, the situation that car plate often ran in the coarse positioning stage is that the vehicle brand character carrying below car plate disturbs, and the car plate that needs left and right to revise is little.

After car plate localization process, for convenient identification link below, should first carry out cutting to character, and its size of normalization, the quality of cutting quality and correctness will directly affect the correctness of recognition result below.Distinguish to some extent with the character cutting in traditional OCR, the image that characters on license plate cutting is tackled is more complicated, and because character is less, in the time of application universal character cutting algorithm, effect is poor.The main task of characters on license plate cutting is: remove the non-character zone in license plate area, as car plate frame, rivet, pollution etc. 1.; 2. the car plate that has inclination and distortion is corrected; 3. the character in car plate is cut out one by one.Its gordian technique relating to mainly can be divided into three: binaryzation, slant correction, character cutting.

After fine positioning, may, because the reasons such as camera angle cause the car plate photographing to have the situation of run-off the straight, at this moment need car plate to carry out slant correction.

At present, license plate image sloped correcting method mainly contains three kinds: Hough converter technique, rotating and projection method, principal component analysis (PCA).Hough converter technique is first license plate image to be carried out to rim detection, determine car plate frame angle of inclination by Hough conversion again, but the frame of car plate is sometimes also not obvious even be cannot see because impact of the interference such as noise, stain and binaryzation etc. makes, make that calibration result is undesirable and calculated amount is excessive.Rotating and projection method is by rotation license plate image, by marginal point to coordinate axis projection, find out the minimum value of projection after each rotation, asking for angle of inclination proofreaies and correct, it has stronger antijamming capability, but asking for of optimum angle of incidence is a searching process, carry out multiple projections and could progressively force into optimum angle of incidence, and therefore calculated amount is also very large.Principal component analysis (PCA) is converted to searching image angle of inclination problem eigenwert and the proper vector of asking for image covariance matrix, speed and be not subject to the impact of frame sharpness.

Hough conversion is bearing calibration conventional in a kind of Digital Image Processing, but this algorithm calculated amount is large, can adopt more simple bearing calibration for the higher situation of requirement of real-time.Here adopt a kind of data fitting method based on mathematical statistics, its core concept is that the left and right difference in height by asking car plate is calculated pitch angle, and therefore this algorithm can effectively solve the shortcoming that car plate sideline is unclear and other algorithm operation quantities are large.

Tilt detection and correcting algorithm are as follows:

Step1: car plate is divided into left-half and right half part according to width, and first pixel scans left-half car plate from lower-left, if this pixel value is 255, asks for distance a and a × c of this point;

Step2: been scanned, ask for all pixels that satisfy condition a value and K, a × c's and P;

Step3: ask for the value of P/K, P/K is that leftmost pixel value is 255 mathematical statistics average height;

Step4: first pixel scans right half part car plate from lower-left, if this pixel value is 255, asks for distance b and the b × d of this point;

Step5: been scanned, ask for all pixels that satisfy condition b value and M, b × d's and N;

Step6: ask for the value of N/M, N/M is that right pixels value is 255 mathematical statistics average height; The difference of left-half and right half part pixels tall mean value is the left and right difference in height h of car plate, and the tangent value θ at the pitch angle of car plate can calculate with formula (11) like this,

\tan θ = \frac{N / M - P / K}{Width / 2} - - - (11)

Step7: calculate after tiltangleθ, just can carry out to car plate the correction of horizontal direction, in correction, we have done a processing, if the point of new images in mapping outside former license plate image point, this point is generally the noise at car plate edge so, and at this moment we set to 0 this pixel value.

For Character segmentation, adopt bianry image sciagraphy to carry out Character segmentation here, this algorithm travelling speed is fast, meets the requirement of real-time.Bianry image sciagraphy is that the license plate area image after binaryzation is carried out to vertical projection, due to all at regular intervals between seven characters, on projection result figure, between each character, certainly exist peak valley, peak valley position is exactly the gap between characters on license plate, in the enterprising line scanning of drop shadow curve, just can judge the initial sum final position of car plate by the feature of trough and crest like this.Bianry image sciagraphy Character segmentation algorithm is as follows:

Step1: on binary image license plate area, from left to right carry out by row search, each lists from the top down and searches for, the number that the pixel value of adding up each pixel unit row is 255, result is saved in to array iResult[i] in, iResult[i] what preserve is vertical projection value;

Step2: search for last character left from car plate low order end, start to subtract circulation to 0.8 × Width from Width (car plate width), if the projection value of continuous three pixels all meets (iResult[i] >=2) & (iResult[i-1] >=2) & (iResult[i-2] >=2), the right point of this i point position first character.And this value is assigned to this columns group Hpoint[0];

Step3: from Hpoint[0] start to subtract circulation to 0.5 × Width, if (iResult[i] >=2) & (iResult[i+1] >=2) & (iResult[i+2] >=2), the left side point that this i point is first character, and this value is assigned to this columns group Hpoint[1];

Step4: calculate (Hpoint[0] Hpoint[1]) >=Width/15 and whether meet, whether calculating character the ratio of width to height Height/ (Hpoint[0]-Hpoint[1]) >=2 meet, if all meet Hpoint[1], Hpoint[0] position, left and right of tab character.Otherwise Hpoint[1] position give Hpoint[0], more left search;

Step5: the like, character is cut.The position of reducing is kept at array Hpoint[j] in;

Step6: the blank character " " in licence plate is eliminated, judge whether continuous three pixels tall all meet priori conditions (iResult[i-1] >=1Height/5) & (iResult[i-2] >=1Height/5) & (iResult[i] >=Height/5) and whether meet, and judge whether priori conditions character duration (Hpoint[11] Hpoint[12]) >=Width/15 meets, if do not meet, this point is blank character, now Hpoint[11] position give Hpoint[10], blank character is skipped, continue search left, until the rightest letter,

Step7: the filtering that the sideline noise that finally may exist left frame disturbs is with the filtering method of left frame line noise.

After completing the location of car plate and the cutting of character, will the character picture after cutting be identified.Character recognition is an important branch of pattern-recognition, belongs to problem of image recognition.Car plate identification is a specific question of character recognition, and its basic thought is all template matches, and the object of car plate identification is to export the number-plate number, and therefore, character recognition part is the core content of identification module in electronic police system.

In character recognition, first the single character after cutting is carried out to pre-service normalization, and extraction represents the expression-form (feature) of character pattern essence to be identified, then mate one by one with predefined standard character pattern and set (dictionary), finally differentiate by corresponding criterion, in the mode standard in dictionary, find out the expression formula approaching the most with pattern to be identified, the character that this expression-form is corresponding is judged to be recognition result.

Most of character is not of uniform size through often after cutting, before character recognition, need to be normalized character again, and be here the normal size of wide 20 pixels of high 40 pixels character normalizing.Determine four accurate frames of character by the mode of projection scanning, then carry out convergent-divergent processing and realize normalization, determine frame algorithm flow:

Step1: make i=H/2;

Step2: scanning current line white pixel (value is 255) projection value, and be recorded to array N (i) (i=1,2,3 ..., H/2);

Step3: the value that judges N (i), record makes the i of N (i) <2 & & N (i-1) <2 capable, i is exactly the coboundary of character, stops current scanning, turns STEP5; Otherwise continue;

Step4:i=i-1, judges whether i>0 sets up, if set up, description character has scanned the coboundary of cutting image, and coboundary is the coboundary of cutting image.Otherwise explanation may be that fracture may appear in character, turns STEP2, continues scanning;

Step5: start to scan lower boundary from i=H/2;

Step6: first i value of search N (i) <2 & & N (i+1) <2, record the lower boundary of i value for current character, otherwise, turn Step6;

Step7:i=i+l, judges H>i, if set up, may be breakaway poing, turns Step6; If be false the lower boundary that lower boundary of cutting image is character;

Step8: finish.

Character recognition.Determining after four accurate frames of character, work is below exactly to adopt the method for dwindling or amplify to be normalized to the character of cutting, is unified into the normal size of wide 20 pixels of high 40 pixels;

The present invention adopts a kind of reclassify device to realize the method for character recognition.First order sorter is realized the rough sort of character, and second level sorter is realized the disaggregated classification of character, and third level sorter is realized the differentiation of similar character.

(1) adopt the rough sort device of projection sequence fuzzy matching to design

First obtain the projection of the horizontal and vertical image of character picture, thereby acquisition has 60 projection sequence altogether, (horizontal projection is front, vertical projection is rear), then carry out fuzzy matching with the projection sequence of standard character template, which analytic band identification character more presses close to/group, to complete rough sort, these chapters and sections adopt the minimax approach degree in a kind of fuzzy logic to weigh two similarity degrees between projection sequence, establish p ₁, p ₂be respectively the projection sequence that two length are 60, the similarity between them can be calculated with formula (12);

N = \frac{Σ_{i = 0}^{60} Min (p_{1} (i) p_{2} (i))}{Σ_{i = 0}^{60} Max (p_{1} (i) p_{2} (i))} - - - (12)

If projection sequence is identical, similarity is 1, otherwise the value of similarity is between 0～1.

The accurate character set of bidding is T={T _i1, T _i2, T _i3..., T _im, if character C to be identified and standard character T _i1t _i2t _i3t _imsimilarity N _i1n _i2n _i3n _imall be greater than a certain threshold value, think that character C can be referred to subclass T{T _i1t _i2t _i3t _imin, that is:

T={T _iN _i-N _max<threshold} （13）

In formula, N _max=Max (N _i) (i=1,2,3 ..., 60), if im=1 shows to only have a coupling, identification just completes so.For the situation that has multiple couplings, also need secondary classification device further to carry out disaggregated classification.

(2) the segmentation classifier design of refinement character comparison

Send secondary classification device to carry out disaggregated classification the rough sort result of the first step, also standard character library is carried out to the java standard library of thinning processing as new secondary classification device simultaneously.Strategy can be expressed as: establish F and f and be expressed as the character picture to be identified after character picture to be identified and refinement, T and t represent respectively character picture after standard character image and refinement.The thought of this method is: for the some white points in refinement character, (pixel value is 255, represent foreground point) whether be the situation of white point corresponding to this point in the character for refinement, if white point, the pixel of thinking that the match is successful.Finally calculate the pixel number that the match is successful and the total white pixel of the refined image ratio of counting and determine similarity, similarity is calculated by shown in formula (14),

R_{1} = \frac{Σ_{i = 0}^{40} Σ_{j = 0}^{20} F (i, j) \cap t (i, j)}{Σ_{i = 0}^{40} Σ_{j = 0}^{20} t (i, j)} - - - (14)

Above formula represents cutting character and standard refinement character alignment similarity, uses R ₁represent.But easily there is mistake identification for the more close visibly different character of some fonts, such as character to be identified is B, may obtain result is P, and I adopt the method for two-way comparison here, except formula (15), we calculate the similarity R between T and f simultaneously ₂, calculate with formula (15),

R_{2} = \frac{Σ_{i = 0}^{40} Σ_{j = 0}^{20} T (i, j) \cap f (i, j)}{Σ_{i = 0}^{40} Σ_{j = 0}^{20} f (i, j)} - - - (15)

Finally we calculate the mean value of twice coupling, make like this efficiency greatly improve.Calculation process is as follows:

Step1: cutting character is carried out to image thinning;

Step2: calculate similarity R by formula (14) _ivalue, calculates similarity R by formula (15) ₂value;

Step3: calculate (R ₁+ R ₂) 2 value;

Step4: calculate (R ₁+ R ₂) 2 and given threshold value Threshold between size carry out second degree matches, determine the classification T'{T of character F to be identified _i1, T _i2, T _i3..., T _im, if k=1, the match is successful in explanation.If k=0, it fails to match in explanation, do not find similar character, if k>l, explanation has similar character, need to carry out reclassify device and identify.

(3) the identification and classification device of charcter topology feature design

Through two sorters above, some character can be differentiated, but for some similar characters, be also difficult to differentiate, at this time need to differentiate by the difference of charcter topology feature, for ODQ, we find that OD is symmetrical up and down, and Q is obviously asymmetric, and O is symmetrical, D left-right asymmetry, left side pixel is many.2Z significantly difference is 2 above in set of pixels, and Z is average.8B obviously difference is 8 left and right pixel symmetry, and B obviously the right pixel concentrates.Utilize character feature above, distinguish algorithm steps as follows:

Step1: judge that institute's similarity of genera belongs to ODQ, 2Z or 8B;

Step2: the horizontal projection P that calculates each similar pixel _yand vertical projection P (i) _x(i);

If ODQ type Step3:., calculates each character with value, wherein H is character height, if

this character is Q, otherwise, calculate

if

Σ_{i = 1}^{L} (L - i) * P_{x} (i) > 1.1 * Σ_{i = 1}^{L} (i) * P_{x} (i),

Wherein L is character duration, and this character is D, otherwise is O;

Step4: if 2Z type, judgement

if set up, this character is 2, otherwise this character is Z;

Step5: if 8B type, judgement

if set up, this character is B, otherwise is 8.

Claims

1. the peccancy parking detector based on panoramic vision of a lightweight, it is characterized in that: the peccancy parking detector based on panoramic vision of described a kind of lightweight, comprises the panoramic vision sensor of the video image for obtaining guarded region and for video image being understood to the microprocessor of analyzing and carrying out parking offense detection, described panoramic vision sensor is connected with described microprocessor by described video interface, microprocessor carries out real-time vision detection by large-range monitoring panoramic vision sensor to the vehicle on road, in the time detecting that within the scope of vision-based detection, existence is against chapter parking behavior, inform or warn parking offense driver by speech play unit and do not want parking offense, for exceeding down time after official hour threshold value, system is captured this vehicles peccancy, and automatically generate a parking offense record, in parking offense record, include vehicles peccancy photo, the parking offense record of when and where violating the regulations sends to the processing server violating the regulations of vehicle supervision department by network, processing server violating the regulations carries out car plate identification to captured vehicles peccancy general image, analysis obtains the license plate number of this vehicles peccancy, and it is single automatically to generate parking offense processing according to the single call format of processing violating the regulations, finally remind managerial personnel to confirm to process, described microprocessor also comprises:

Panoramic picture data acquisition module, for reading the video image information obtaining from panoramic vision sensor;

Vehicles peccancy detection module based on sampled point, for detection of the parking offense vehicle existing on road; There is sampled point to process to obtain the static piece that exists to static, and then it is carried out to the coupling of car modal, if the match is successful with regard to preliminary judgement for have stationary vehicle on road; If detect and exist stationary vehicle on road, according to the locus of this stationary vehicle, whether the front of detecting this stationary vehicle has stationary object and detects the locus of stationary vehicle on road, if near the roadside of stationary vehicle in road and stationary vehicle front does not exist stationary object, and the residence time of this stationary vehicle exceedes official hour threshold value, be just judged to be suspicious violating the regulations parking cars.

2. the peccancy parking detector based on panoramic vision of a kind of lightweight as claimed in claim 1, described panoramic vision sensor becomes minute surface and the camera lens of angle just to form towards the video camera of minute surface by two; Angle between two minute surfaces is 180 ° of-2 γ, and two minute surfaces width value on front elevation is W, height value on side view is R, and the width value W of two minute surfaces and height value R are positioned at the areas imaging of video camera; On side view, the central shaft of described video camera becomes η angle with the central shaft of described vertical rod, and minute surface becomes ε angle with the surface level direction of road side; On front elevation, the angle of minute surface and described vertical rod is 90 °-γ, the central shaft of video camera and the central axes of vertical rod, and the focal length of video camera is f;

γ=(tan ^-1(L／H)-ω _max×H)／2 (1)

Further, the maximum visual angle ω of described video camera _maxit is 45 °, the setting height(from bottom) H of panoramic vision sensor is 3 meters, the visual length L along on road direction of panoramic vision sensor is greater than 200 meters, the angle γ that tries to achieve minute surface and surface level by formula (1) is 32 °, the length of minute surface is greater than W2 × cos (γ), the width of every minute surface is greater than Rcos (ε-η), and ε is the angle of the surface level direction of minute surface and road side, and η is the angle between video camera central shaft and vertical rod central shaft.

3. the peccancy parking detector based on panoramic vision of a kind of lightweight as claimed in claim 1, described panoramic vision sensor adopts high-definition camera, high-definition camera is configured in to about 10 meters of the tops place in track, roadside, parallel with track, roadside direction, aim at track, roadside to declivity, the transverse axis of the imaging plane of high-definition camera is parallel with ground level simultaneously, so also can obtain the video information in track, long and narrow scope roadside.

4. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, the customized module in the demarcation of described panoramic vision sensor and track, roadside, for the customization to camera calibration and parking offense region, determine the mapping relations of two-dimensional imaging plane and three dimensional space coordinate point; The method for customizing in parking offense region is: the image first obtaining from described panoramic vision sensor, select track, two roadsides marginal point according to roadside lane markings nearby in track, roadside perpendicular to track, roadside direction, then select track, two roadsides marginal point in the distant place in track, roadside on perpendicular to track, roadside direction, these four marginal points are connected and composed to lane detection region, roadside; Finally described panoramic vision sensor is demarcated;

Here adopt odd coordinate to demarcate described panoramic vision sensor, computing formula as the formula (2),

[\begin{matrix} λ_{i} x_{i} \\ λ_{i} y_{i} \\ λ_{i} \end{matrix}] = [\begin{matrix} b_{11} & b_{12} & b_{13} & b_{14} \\ b_{21} & b_{22} & b_{23} & b_{24} \\ b_{31} & b_{32} & b_{33} & b_{34} \end{matrix}] [\begin{matrix} x \\ y \\ z \\ 1 \end{matrix}] - - - (2)

\{\begin{matrix} x_{i} = \frac{b_{11} x + b_{12} y + b_{14}}{b_{31} x + b_{32} + 1} \\ y_{i} = \frac{b_{21} x + b_{22} y + b_{24}}{b_{31} x + b_{32} + 1} \end{matrix} - - - (3)

5. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, automatic generation, gray-scale value inspection and the sampling point position fine setting module of described sampled point, be used for the uniform sampled point of the automatic span in track, roadside customizing, the sampled point gray-scale value generating is carried out to consistency check, the sampled point that departs from gray-scale value is carried out to locus adjustment;

By the demarcation of described panoramic vision sensor, the mapping relations of the pixel on point and the plane of delineation on the road of space are set up; On the road of space, evenly generate sampled point by these mapping relations, the sampled point generating on the plane of delineation projects to be spaced apart 0.5 meter on real space road;

as initial background value

as the non-gray-scale value that has sampled point;

Further, consider on track, roadside and have some road signs, for example right-hand rotation waits identifier, and gray-scale value and the road surface of these identifiers differ greatly, and can bring detection error to follow-up context update if sampled point just in time drops in road sign; Therefore, need to carry out gray-scale value inspection to the sampled point of all customizations, if the gray-scale value of some sampled points depart from its initial background value

meet

travel through after all sampled points, will

as the initial background gray-scale value of each sampled point here by track, roadside by vehicle heading be divided into length ratio be 2:3:5 far away, in, nearly three sections of regions, the average generation line number sampled point consistent with columns each region in.

6. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, the described sampled point detecting unit that exists, detects for the foreground object of satisfying the need on sidecar road; Concrete methods of realizing is that employing background subtraction method detects the sampled point that exists in tn moment from sampled point image, extracts and represents the exist sampled point of foreground object in track, roadside space distribution; Meet

the sampled point of condition is just judged to be to exist sampled point, otherwise is the non-sampled point that exists, and obtains existing sampled point image E _nwith the non-sampled point image that exists

exist the detailed process of detection module of sampled point as follows:

Step2: obtain t _nthe gray level image X of moment sampled point _n;

D _n=X _n-B ₀ (4)

Step5: be used in the threshold value TH1 setting in Step1 to D _ncarry out binary conversion treatment, obtain existing sampled point bianry image ES _n, at ES _nin all sampled points all have 1 and 0 two states, 1 represents have foreground object to exist on this sampled point, is and has sampled point; 0 represents not have foreground object on this sampled point, is the non-sampled point that exists.

7. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, the described static sampled point detection module that exists, for detection of the static sampled point that exists on track, roadside; Detection principle is, first adopts frame-to-frame differences algorithm from image sequence, to detect the mobile sampled point that exists, and then, according to having sampled point and mobilely existing sampled point to calculate the static sampled point that exists, its testing process is as follows:

Z _1n=X _n-X _n-α （5）

Z _2n=X _n+α-X _n （6）

Y _n=T _1n∧T _2n （7）

S _n=F _n-Y _n （8）。

8. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, the described vehicles peccancy detection module based on sampled point, for detection of the parking offense vehicle existing on road; The thought of detection algorithm is, first exists sampled point to process to obtain the static piece that exists, then it is carried out the coupling of car modal to static, if the match is successful with regard to preliminary judgement for have stationary vehicle on road; If detect and exist stationary vehicle on road, according to the locus of this stationary vehicle, whether the front of detecting this stationary vehicle has stationary object and detects the locus of stationary vehicle on road, if near the roadside of stationary vehicle in road and stationary vehicle front does not exist other any stationary objects, and the residence time of this stationary vehicle exceedes the threshold value of regulation, be just judged to be suspicious violating the regulations parking cars;

Further, the satisfy the need static sampled point image S that exists of sidecar road space distribution _ncarry out filtering processing with auto model, remove some isolated nonstatics and have sampled point and other interference, obtain the static piece that exists that reflects that vehicle temporarily stops or being detained; Here the static sampled point image S that exists mainly formula (8) being calculated _ncarry out filtering processing, filtering processing and parking offense vehicle detecting algorithm are as follows:

Step6: EOP (end of program).

9. the peccancy parking detector based on panoramic vision of a kind of lightweight as described in claim 1 or 2 or 3, the described road background modeling module based on sampled point, for satisfying the need, sidecar road background is carried out modeling; Owing to having customized equably sampled point on track, roadside in the time customizing sampled point, background subtraction point-score can be used for detection and has sampled point, but background subtraction point-score requires to obtain reliable, stable sampled point background gray levels; The sampled point in road customization region is subject to impact that the external environment such as illumination, weather changes greatly, need to carry out real-time update to sampled point background gray levels; Here adopt distance to exist the nearest non-gray-scale value that has sampled point of sampled point to upgrade the background that has sampled point, realize a kind of lightweight, accurate efficiently background update method, update algorithm as shown in Equation (16),

B_{n + 1}^{i} = \{\begin{matrix} B_{n}^{i} + k (X_{n}^{i} - B_{n}^{i}) & , | X_{n}^{i} - B_{n}^{i} | < TH 1 \\ B_{n}^{\min (index (q) - index (p))} & , else \end{matrix} - - - (16)

In formula,

for having the gray-scale value of the nearest non-existence sampling of sampled point from this,

for tn moment sampled point actual measurement gray-scale value,

for tn moment sampled point background gray levels,

for tn+1 moment sampled point background gray scale predicted value.