CN103204123B - Vehicle-pedestrian detecting, tracking and early-warning device and method - Google Patents
Vehicle-pedestrian detecting, tracking and early-warning device and method Download PDFInfo
- Publication number
- CN103204123B CN103204123B CN201310096452.XA CN201310096452A CN103204123B CN 103204123 B CN103204123 B CN 103204123B CN 201310096452 A CN201310096452 A CN 201310096452A CN 103204123 B CN103204123 B CN 103204123B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- target
- warning
- time
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention discloses vehicle-pedestrian detecting, tracking and early-warning device and method. The method includes that the early-warning device preprocesses infrared videos obtained by an infrared camera through an image preprocessing module, detects vehicles and pedestrians through a detecting module, forecast-tracks the vehicles and pedestrians through a tracking module to eliminate interference influences of vehicle vibration and virtual targets, calculates the distance between each target and the infrared camera according to corresponding relationship between world coordinates and image coordinates through a distance measuring module, judges an early-warning standard through a judging module, and displays the infrared videos carrying target identifiers and distance information on a display. The vehicle-pedestrian detecting, tracking and early-warning device has the advantages that driving safety is improved, most traffic accidents are effectively avoided, and life and property loss is lowered. The invention further discloses the vehicle and pedestrian detecting, tracking and early-warning method using the early-warning device.
Description
Technical field
The present invention relates to a kind of automobile auxiliary security control loop, particularly relate to a kind of traffic detecting and tracking prior-warning device based on monocular infrared video and method for early warning thereof, the target detect in the fields such as on-vehicle night vision, communications and transportation, video monitoring and public safety, tracking and range finding can be applied particularly to.
Background technology
Along with the fast development of automotive market and the strengthening day by day of security protection consciousness, the requirement of people to automotive safety safeguards technique is more and more higher.Existing automobile lighting system sighting distance in night, the snow visbility such as sky or dense fog lower situation is not far, one of the disadvantage potential safety hazard becoming running car of poor effect; in addition when driving a vehicle night; usually can be there is blind area by the interference of the light of opponent vehicle in chaufeur, also easily get into an accident.The principle that the infrared light intensity that passive type infrared night vision system sends according to the object of different temperatures is different carries out imaging, the temperature of object is higher, the image presented is brighter, so passive type infrared night vision system can help chaufeur to navigate in the dark, chaufeur is made to have in light and dark two kinds of situations and can see driving environment clearly, the traffic seen by passive type infrared night vision system can than road surface and trees brighter, so driver can notice vehicle and the pedestrian in front immediately.
Chaufeur also can cause Frequent Accidents due to the reason such as fatigue driving or dispersion attention in addition.Research shows, as long as chaufeur obtains 0.5 second pre-warning time before collision risk, can avoid the collision case of 60%, if obtain pre-warning time before 1.5 seconds, then can avoid more than 90% traffic accident.Therefore, automobile is equipped with the incidence that a set of intelligent anticollision forewarn system significantly can reduce road accident, may with front vehicles or pedestrian collision time, automobile can carry out sound and light alarm, the direction of prompting chaufeur dangerous article and hazard level, chaufeur can take brake measure to avoid the generation of accident if desired, also can pass through automobile CAN-bus, and forewarn system controls automobile and automatically implements braking.
At present, vehicle mounted infrared night vision device has applied to front dress some high-end vehicles in market and the rear dress market of small part, and the infrared image of catching directly shows by infrared camera before car on a liquid crystal display.During night running, vehicle driver will watch road conditions attentively, also needs constantly to watch infrared image, thus the whether dangerous appearance of comprehensive descision, this mode adds the burden of driver, is easy to cause erroneous judgement or fail to judge, and effectively can not play auxiliary effect of driving.For existing anti-collision early warning system, the product based on monocular visible image capturing head of Mobileye company of current Israel occupies most market, but it is when using night, owing to detecting visible ray, so observed range is comparatively near, limit its use at night.
Fpga chip in the market has powerful digital processing capabilities as data processing chip, FPGA alerting ability is strong, parallel processing capability is powerful, operational performance is excellent, owing to can carry out programming, debug, programmes and repeats again, therefore can carry out easily designing and developing and verifying, when circuit have change on a small quantity time, more can demonstrate the advantage of FPGA, meet the demand that IR video stream amount is large, process requirement of real-time is high, achieve data Quick Acquisition and process.ARM chip in the market, as controller, is integrated with a lot of function, has powerful transaction capabilities, just can provide diversified work, such as display, runnable interface and application program etc. by means of simple periphery.
On infrared imaging system is integrated, for improving practicality and the ease for use of infrared imaging further, infrared image processing technology has become the study hotspot of infrared imaging field and image processing field, and corresponding operation platform also develops into embedded platform from upper computer, and becomes a kind of trend.The infrared image processing embedded platform structure reported mainly contains following two kinds: (1), based on the infrared image processing device of ARM hardware structure, collection, the process and interface operation etc. of view data all complete on ARM; (2) based on the infrared image processing device of FPGA hardware structure, collection, the process and interface operation etc. of view data all complete on FPGA.Because ARM data-handling capacity is more weak, and video processnig algorithms comprises a large amount of data calculating, if be used alone ARM chip to complete video data process, cannot ensure the real-time of video processing.When video definition is higher, when data volume is larger, ARM chip cannot complete the Processing tasks of data; Fpga chip can realize the high speed processing task of mass data, but FPGA development difficulty is high, and algorithm is transplanted complicated, is difficult to host computer procedure to be transplanted in FPGA realize the functions such as interface display.
Summary of the invention
In view of this, be necessary to provide a kind of traffic detecting and tracking prior-warning device and method for early warning thereof, its object is on the basis of infrared night vision, exploitation is a kind of with target detect, follow the tracks of, the vehicle mounted infrared night vision device of range finding and anti-collision warning prompt facility, the safety of driving can be improved, effectively avoid most traffic accident, thus reduction life and property loss, the present invention is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles or pedestrians outside this vehicle are detected and follows the tracks of range finding, calculate the distance with other vehicles or pedestrians, and send early warning, thus, have and certain algorithm process is carried out to infrared image, risk object can be detected, and send alarm in time, then greatly can alleviate the burden of driver.
The present invention realizes like this, a kind of traffic detecting and tracking prior-warning device, it is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles or pedestrians outside this vehicle are detected and follows the tracks of range finding, calculate the distance with other vehicles or pedestrians, and send early warning, this traffic detecting and tracking prior-warning device comprises:
Image pre-processing module, it is for carrying out pretreatment to the infrared video obtained from this noctovisor: by filtering method filtering noise, then uses the method for adaptive histogram equalization to strengthen picture contrast;
Detection module, it, for carrying out detection & localization to other vehicles or pedestrians in this video, detects target vehicle or target pedestrian;
Tracking module, it, for carrying out predicting tracing to target vehicle or target pedestrian, follows the tracks of confirmation further;
Range finder module, it is for calculating the distance through following the tracks of target vehicle or the target pedestrian confirmed, this distance meets following formula:
Wherein, d is the distance of this vehicle-to-target vehicle or target pedestrian; F is the effective focal length of this noctovisor; α is the pitch angle of this noctovisor; H is the height of camera lens to road surface of this noctovisor; (x
0, y
0) be the optical axis of this camera lens and the intersection point of picture plane, be the initial point of photo coordinate system; (x, y) is the projection coordinate of P point in picture plane, and this P point takes from the target vehicle or target pedestrian that confirm through tracking, as the R point of the distance d with this vehicle;
Judge module, it is for judging whether this distance d is less than a predetermined collision distance;
Display and warning module, it is for when this distance d is less than this predetermined collision distance, starts audible alarm, and starts display alarm;
Wherein, this tracking module, for adopting Kalman filter method in conjunction with Mean Shift method, is set up the target model being used for vehicles or pedestrians and following the tracks of, and in tracing process, is upgraded target model parameter, the impact that this vehicle of filtering brings in vibration processes, and the empty target of filtration fraction.
As the further improvement of such scheme, according to the feature of infrared video, namely the temperature of vehicles or pedestrians is relatively high, and this detection module, for adopting the way of rim detection and feature identification, detects target vehicle or target pedestrian and locates.
As the further improvement of such scheme, this display and warning module, also for adopting different corresponding sound and light alarm modes according to different distance d sectional, make the frequency of alarm sound and icon sectional change along with the change of distance d that is strong and weak and display alarm.
As the further improvement of such scheme, this traffic detecting and tracking prior-warning device also comprises: module during survey, it is for going out instantaneous collision time according to the change calculations of target vehicle or target pedestrian dimension scale in video, and this instantaneous collision time meets following formula:
Wherein, T
mfor instantaneous collision time; △ t is instantaneous collision time T
mand the interval between two moment of current time; S is the ratio change of target vehicle or target pedestrian size in two moment videos, target vehicle or target pedestrian near time, S>1;
This judge module, it is for judging instantaneous collision time T
mwhether be less than a pre-warning time;
Display and warning module, it is at instantaneous collision time T
mwhen being less than this pre-warning time, starting audible alarm, and start display alarm.
Preferably, this display and warning module are also for according to different instantaneous collision time T
msectional adopts different corresponding sound and light alarm modes, makes the icon of the frequency of alarm sound and power and display alarm along with instantaneous collision time T
mchange and sectional change.
Preferably, this traffic detecting and tracking prior-warning device also comprises: correcting module, and it is for instantaneous collision time T
mcarry out acceleration correction, obtain the correction collision time meeting following formula:
Wherein, T represents instantaneous collision time T
mthe correction collision time obtained through acceleration correction;
represent instantaneous collision time T
mto time differentiate;
This judge module, it is for judging whether revise collision time T is less than the anti-collision warning time;
Display and warning module, it, for when revising collision time T and being less than this anti-collision warning time, starts audible alarm, and starts display alarm.
Again preferably, this display and warning module, also for adopting different corresponding sound and light alarm modes according to different correction collision time T sectional, make the frequency of alarm sound and icon sectional change along with the change of correction collision time T that is strong and weak and display alarm.
Again preferably, this traffic detecting and tracking prior-warning device also comprises: brake command forms module, its for be less than a predetermined minimum collision distance at distance d or revise collision time T be less than the predetermined minimum anti-collision warning time time, form brake command, this brake command is used for implementing braking by the CAN of this vehicle to this vehicle.
The present invention also provides a kind of traffic detecting and tracking method for early warning, it is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles or pedestrians outside this vehicle are detected and follows the tracks of range finding, calculate the distance with other vehicles or pedestrians, and send early warning, this traffic detecting and tracking method for early warning comprises the following steps:
Pretreatment is carried out to the infrared video obtained from this noctovisor: by filtering method filtering noise, then use the method for adaptive histogram equalization to strengthen picture contrast;
Detection & localization is carried out to other vehicles or pedestrians in this video, detect target vehicle or target pedestrian: according to the feature of infrared video, namely the temperature of vehicles or pedestrians is relatively high, adopts the way of rim detection and feature identification, detects target vehicle or target pedestrian and locates;
Predicting tracing is carried out to target vehicle or target pedestrian, further tracking confirms: adopt Kalman filter method in conjunction with Mean Shift method, set up the target model being used for vehicles or pedestrians and following the tracks of, and in tracing process, upgrade target model parameter, the impact that this vehicle of filtering brings in vibration processes, and the empty target of filtration fraction;
Calculate the distance through following the tracks of target vehicle or the target pedestrian confirmed, this distance meets following formula:
Wherein, d is the distance of this vehicle-to-target vehicle or target pedestrian; F is the effective focal length of this noctovisor; α is the pitch angle of this noctovisor; H is the height of camera lens to road surface of this noctovisor; (x
0, y
0) be the optical axis of this camera lens and the intersection point of picture plane, be the initial point of photo coordinate system; (x, y) is the projection coordinate of P point in picture plane, and this P point takes from the target vehicle or target pedestrian that confirm through tracking, as the R point of the distance d with this vehicle;
Judge whether this distance d is less than a predetermined collision distance;
When this distance d is less than this predetermined collision distance, starts audible alarm, and start display alarm.
As the further improvement of such scheme, this traffic detecting and tracking method for early warning is further comprising the steps of:
Change calculations according to target vehicle or target pedestrian dimension scale in video goes out instantaneous collision time, and this instantaneous collision time meets following formula:
Wherein, T
mfor instantaneous collision time; △ t is instantaneous collision time T
mand the interval between two moment of current time; S is the ratio change of target vehicle or target pedestrian size in two moment videos, target vehicle or target pedestrian near time, S>1;
Judge instantaneous collision time T
mwhether be less than a pre-warning time;
At instantaneous collision time T
mwhen being less than this pre-warning time, starting audible alarm, and start display alarm.
As the further improvement of such scheme, this traffic detecting and tracking method for early warning is further comprising the steps of:
To instantaneous collision time T
mcarry out acceleration correction, obtain the correction collision time meeting following formula:
Wherein, T represents instantaneous collision time T
mthe correction collision time obtained through acceleration correction;
represent instantaneous collision time T
mto time differentiate;
Judge whether revise collision time T is less than the anti-collision warning time;
When revising collision time T and being less than this anti-collision warning time, start audible alarm, and start display alarm.
Preferably, this traffic detecting and tracking method for early warning is further comprising the steps of: adopt different corresponding sound and light alarm modes according to different correction collision time T sectional, makes the frequency of alarm sound and icon sectional change along with the change of correction collision time T that is strong and weak and display alarm.
The invention has the advantages that: compared to visible ray, infrared video can observe longer-distance vehicle and pedestrian at night and greasy weather, in conjunction with the target detect of vehicle and pedestrian, follow the tracks of, range finding and early warning technology, constantly observe the infrared video in telltale without the need to chaufeur thus judge whether dangerous situation, but when there being the collision time of target to be less than the time of setting (or when the distance of target is less than the distance of setting), device can send sound and light alarm, and with the degree of car danger, the frequency of sound and power can change thereupon, thus chaufeur can take appropriate measures and avoids the generation of accident.In addition, in case of emergency, when being less than certain distance or collision time, when chaufeur is not taken measures, device can implement braking by the CAN of automobile to automobile.
Accompanying drawing explanation
The module composition schematic diagram of the traffic detecting and tracking forewarn system that Fig. 1 provides for better embodiment of the present invention.
Fig. 2 is the structural representation of the treatment circuit of traffic detecting and tracking forewarn system in Fig. 1.
Fig. 3 is the module composition schematic diagram of the traffic detecting and tracking prior-warning device being applied to traffic detecting and tracking forewarn system in Fig. 1.
Fig. 4 is the range measurement principle schematic diagram of traffic detecting and tracking prior-warning device in Fig. 3.
Fig. 5 is the vehicle-mounted schematic diagram of traffic detecting and tracking forewarn system in Fig. 1.
Fig. 6 is the birds-eye view of Fig. 5.
Fig. 7 is that the target detect of traffic detecting and tracking prior-warning device in Fig. 3 follows the tracks of range finding schematic diagram.
Fig. 8 is the contrast schematic diagram of instantaneous collision time and acceleration correction collision time.
Detailed description of the invention
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
Traffic detecting and tracking prior-warning device of the present invention, it is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles outside this vehicle and pedestrian are detected and followed the tracks of and finds range, calculate the distance with other vehicles or pedestrians, and send early warning.
This vehicle must possess traffic detecting and tracking forewarn system, this traffic detecting and tracking forewarn system comprises the passive type monocular noctovisor 11, telltale 12, analog-to-digital conversion module 13, power module 14 and the treatment circuit 15 that are arranged in vehicle, as shown in Figure 1.This treatment circuit 15 is shown in this telltale 12 after carrying out data processing by the video signal that this analog-to-digital conversion module 13 obtains this noctovisor 11.
In the present embodiment, the main body hardware architecture that this treatment circuit 15 adopts FPGA circuit module 16 and ARM circuit module 17 to combine, receive the analog video image signals that noctovisor 11 exports, through analog-to-digital conversion module 13 (as A/D decoding chip), carry out video signal digitalisation, FPGA circuit module 16 is utilized to complete the collection of data, and image processing algorithm is realized in FPGA circuit module 16, the view data gathered is processed, according to the difference characteristic of object and background infrared emanation, this algorithm accurately detects object edge, draw out object profile, real-time tracking display is carried out to the movement of object, and by the distance measurement function in algorithm, the target detected is found range, for follow-up use provides credible target thing and range information, thus the detection realized object in vision signal, follow the tracks of and range finding, Infrared video image data after process, send into ARM circuit module 17 to carry out showing and compress storage, realize Real-time image display, simultaneously for relevant departments provide reference of tracing to the source, thus this treatment circuit has given full play to the advantage of FPGA and ARM.
In brief, analog video image signals is inputted A/D processing module and analog-to-digital conversion module 13 by noctovisor 11, after modulus changes, send into FPGA circuit module 16 and completed the identification of object, tracking, detection range finding and warning function, the infrared video input after process is sent into ARM circuit module 17 and is completed video signal compression and store and display.
As shown in Figure 2, this FPGA circuit module 16 comprises FPGR treater, Infrared video image signal BNG input circuit (BNG input), video decoding circuit (video decode), SRAM (SRAM DDR2), warning circuit (audible alarm), I2C etc., and this ARM circuit module 17 comprises arm processor, video compression coding circuit (video compression coding), SD card memory circuit (storage of SD card), debugging communicating circuit and video and shows mutual circuit; Wherein, Infrared video image signal BNG input circuit is electrically connected at video decoding circuit, and video decoding circuit, SRAM and warning circuit are all electrically connected at FPGR treater; SD card memory circuit is electrically connected at video compression coding circuit, video compression coding circuit, debugging communicating circuit (RJ45, UART USB, SPI), video shows mutual circuit (display translation user interactions) and is all electrically connected at arm processor; FPGR treater and arm processor use the mode of bus to carry out the exchange of data, and whole treatment circuit unifies coordinative command by arm processor.This power module 14 is powered and is carried out power supply input to whole system, as produced all types of power supplys required for FPGA circuit module 16 and ARM circuit module 17.
Noctovisor 11 carries out imaging according to the infrared ray of target reflection or radiation, exports analog video image signals.The FPGA circuit module 16 for the treatment of circuit 15 receives the analog video signal of noctovisor 11 output, through A/D decoding chip, carries out video signal digitalisation.Utilize the digital processing capabilities data acquisition that the FPGA circuit module Final 16 is large, and image processing algorithm is realized in FPGA circuit module 16, according to the difference characteristic of object and background infrared emanation, object is detected, follow the tracks of and finds range, Realtime Alerts process is carried out to the risk object detected simultaneously.The fpga chip of FPGA circuit module 16 can realize the high speed processing of mass data, can meet the hardware requirement of system to Infrared video image real time signal processing.Through the IR video stream that FPGA circuit module 16 processes, send in ARM circuit module 17 with BT656 form, real-time display and the compression of carrying out video file store, for relevant departments provide reference of tracing to the source.
The electronic system of society now, depart from software substantially cannot run, on above-mentioned hardware system (i.e. traffic detecting and tracking forewarn system), develop traffic detecting and tracking method for early warning software, i.e. traffic detecting and tracking prior-warning device, the traffic detecting and tracking prior-warning device that the present invention relates to is applied in above-mentioned hardware system, and the subsequent functional realiey part that the present invention is directed to above-mentioned hardware system describes in detail.
Traffic detecting and tracking prior-warning device is applied in the vehicle being provided with passive type monocular noctovisor, namely be applied in the vehicle being provided with traffic detecting and tracking forewarn system, other vehicles outside this vehicle and pedestrian are detected and followed the tracks of and finds range, calculate the distance with other vehicles or pedestrians, and send early warning.
The System's composition of the traffic detecting and tracking prior-warning device that better embodiment of the present invention provides as shown in Figure 3, this traffic detecting and tracking prior-warning device comprise image pre-processing module 1, detection module 2, tracking module 3, range finder module 4, judge module 5, display and warning module 6, brake command formed module 7, survey time module 8.
Traffic detecting and tracking prior-warning device of the present invention, image pre-processing module 1 is adopted to carry out pretreatment to the Infrared video image signal obtained from noctovisor 11, then through detection module 2 pairs of vehicles, pedestrian detects, the obstacle of vibration impact of this vehicle is got rid of through tracking module 3, predicting tracing is carried out to the target detected, through range finder module 4, according to the corresponding relation between world coordinates and image coordinate, calculate the distance between each target and noctovisor 11, afterwards, judged by judge module 5, finally by display and warning module 6, the Infrared video image signal with target identification and range information is shown in telltale 12, and set according to user, can report to the police according to safety distance or report to the police according to the collision time calculated, in case of emergency, module 7 can be formed by brake command, by the CAN of automobile, braking be implemented to automobile, namely by the CAN of this vehicle, braking is implemented to automobile.
Subsequent detailed expansion explanation is carried out to traffic detecting and tracking prior-warning device of the present invention.
First, image pre-processing module 1 is for carrying out pretreatment to the Infrared video image signal obtained from this noctovisor 11: by filtering method filtering noise, then uses the method for adaptive histogram equalization to strengthen picture contrast.For subsequent detection, tracking, range finding and early warning link lay the foundation.
Secondly, detection module 2, for carrying out detection & localization to other vehicles or pedestrians in this video signal, detects target vehicle or target pedestrian.Detection mould 2 pieces mainly realizes the detection & localization to the target such as vehicle and pedestrian in video signal, according to the feature of infrared video, namely the temperature of the target such as vehicle and pedestrian is relatively high, adopt the way of rim detection and feature identification, each target and target vehicle or target pedestrian can be detected.
Afterwards, tracking module 3, for carrying out predicting tracing to target vehicle or target pedestrian, follows the tracks of confirmation further.Tracking module 3 mainly carries out predicting tracing to targets such as the target vehicle detected or target pedestrians, confirms further to each target.Adopt Kalman filter method in conjunction with Mean Shift method, set up the target model being used for the target trackings such as vehicles or pedestrians, and in tracing process, upgrade target model parameter, the impact that this vehicle of filtering brings in vibration processes, and the empty target of filtration fraction.
Then, range finder module 4 is for calculating the distance through following the tracks of target vehicle or the target pedestrian confirmed, as shown in Figure 4, this distance meets following formula:
Wherein, d is the distance of this vehicle-to-target vehicle or target pedestrian; F is the effective focal length of this noctovisor 11; α be this noctovisor 11 pitch angle also namely noctovisor 11 setting angle α; H is the height of camera lens to road surface of this noctovisor 11; (x
0, y
0) be the optical axis of this camera lens and the intersection point of picture plane, be the initial point of photo coordinate system; (x, y) is the projection coordinate of P point in picture plane, and this P point takes from the target vehicle or target pedestrian that confirm through tracking, as the R point of the distance d with this vehicle.
Then, judge module 5 is for judging whether this distance d is less than a predetermined collision distance.
Finally, display and warning module 6, for when this distance d is less than this predetermined collision distance, starts audible alarm, and start display alarm.This display and warning module 6 also can be used for adopting different corresponding sound and light alarm modes according to different distance d sectional, make the frequency of alarm sound and icon sectional change along with the change of distance d that is strong and weak and display alarm.
Certainly, also can show and early warning by calculating collision time to carry out.During survey, module 8 is for going out instantaneous collision time according to the change calculations of target vehicle or pedestrian dimension scale in video, and this instantaneous collision time meets following formula:
Wherein, T
mfor instantaneous collision time; △ t is instantaneous collision time T
mand the interval between two moment of current time; S is the ratio change of target vehicle or pedestrian's size in two moment videos, target vehicle or pedestrian near time, S>1.
Actual conditions may comprise accelerated movement or decelerated movement, so in order to obtain collision time more accurately, need instantaneous collision time T
mcarry out acceleration correction, correcting module is used for instantaneous collision time T
mcarry out acceleration correction, obtain the correction collision time meeting following formula:
Wherein, T represents instantaneous collision time T
mthe correction collision time obtained through acceleration correction;
represent instantaneous collision time T
mto time differentiate;
This judge module 5 is also for judging whether revise collision time T is less than the anti-collision warning time;
Display and warning module 6 also for when revising collision time T and being less than this anti-collision warning time, start audible alarm, and start display alarm.
Judge module 5 is also for judging that revising collision time T (when not having this function of design modification, adopts instantaneous collision time T
m) whether be less than a pre-warning time; Display and warning module 6 are also at correction collision time T (or instantaneous collision time T
m) when being less than this pre-warning time, starting audible alarm, and start display alarm.This display and warning module 6 also can be used for according to different correction collision time T (or instantaneous collision time T
m) sectional adopts different corresponding sound and light alarm modes, makes the icon of the frequency of alarm sound and strong and weak and display alarm along with correction collision time T (or instantaneous collision time T
m) change and sectional change.
In sum, user can setpoint distance d early warning or revise collision time T (or instantaneous collision time T
m) early warning.During setpoint distance d early warning, according to the early warning distance of setting, when there being the distance d of target to be less than early warning distance, then to sound warning, and show corresponding icon at telltale 12, and can set the sound and light alarm mode that different distances is corresponding different, the icon of the frequency of sound and power and telltale display is along with distance d change; Setting correction collision time T (or instantaneous collision time T
m) early warning time, according to the pre-warning time of setting, when there being correction collision time T (or the instantaneous collision time T of target
m) when being less than pre-warning time, then warning of sounding, and show corresponding icon at telltale 12, and different correction collision time T (or instantaneous collision time T can be set
m) corresponding different sound and light alarm mode, the icon that the frequency of sound and strong and weak and telltale 12 show is along with correction collision time T (or instantaneous collision time T
m) change.In addition, when being less than certain distance d or revising collision time T (or instantaneous collision time T
m) in emergency circumstances, brake command forms module 7 can implement braking by the CAN of automobile to automobile.
Compared to visible ray, infrared video can observe longer-distance vehicle and pedestrian at night and greasy weather, in conjunction with the target detect of vehicle and pedestrian, follow the tracks of, range finding and early warning technology, constantly observe the infrared video in telltale without the need to chaufeur thus judge whether dangerous situation, but when there being the correction collision time T of target to be less than the time of setting (or when the distance of target is less than the distance of setting), system can send sound and light alarm, and with the degree of car danger, the frequency of sound and power can change thereupon, thus chaufeur can take appropriate measures and avoids the generation of accident.In addition, in case of emergency, when being less than certain distance d or revising collision time T, when chaufeur is not taken measures, system can implement braking by the CAN of automobile to automobile.
Traffic detecting and tracking forewarn system when mounted, as shown in Figures 5 and 6, noctovisor 11 is arranged on this vehicle head, after being hidden in grid, ensure that the camera lens visual field of noctovisor 11 is not blocked, keep installation site at this left and right vehicle wheel near center location simultaneously; The hardware architecture that analog-to-digital conversion module 13, power module 14 and treatment circuit 15 are formed is arranged on below the operating console of this vehicle, by cable, the video signal of noctovisor 11 is introduced this hardware architecture, power module 14 is received on the Cigarette lighter of this vehicle, powers to noctovisor 11, treatment circuit 15 and telltale 12.Telltale 12 is arranged on the indoor correct position of this vehicular drive, as being arranged on above operating console, or hang and be arranged on the first-class position of forward sight glass, cross cable and the video signal of noctovisor 11 is introduced this hardware architecture, output to telltale 12 and show; The setting height(from bottom) h of adjustment noctovisor 11, according to setting height(from bottom) h, camera parameter and mark position location, calculates camera setting angle α.The setting height(from bottom) h of this noctovisor 11 makes camera lens visual field can cover this vehicle front region, is paved with between a half field-of-view to 2/3rds visual fields with the road surface obtained in video.Start can carry out the infrared detection of this vehicle front target, tracking, range finding and alarm, and result as shown in Figure 7.Subsequent, the present invention does more detailed application example.
Embodiment 1: the traffic detecting and tracking forewarn system of monocular infrared video is applied to truck
Noctovisor 11 is installed on the grid place of truck, records noctovisor 11 height h overhead and pitch angle and setting angle α.Noctovisor 11 is connected with the flush bonding module comprising image processing function by bnc interface, flush bonding module realizes the frequency pretreatment of infrared view image signal, filtering is carried out to noise and figure is strengthened, detection feature according to pedestrian and vehicle detects, and undertaken further confirming to obtain target pedestrian and target vehicle by track algorithm, get rid of empty target, according to height h and the pitch angle α of the noctovisor 11 recorded, location algorithm is adopted to vehicle and pedestrian, draw the distance of each target, and calculate the collision time of each target according to the change ratio of target size in two two field pictures.By AV interface, Infrared video image signal is outputted to Liquid Crystal Display, target in the infrared video of Liquid Crystal Display display adopts rectangle frame to mark, and the distance of each target is marked in rectangle frame bottom, as shown in Figure 7, the track of the automobile i.e. target in track, this vehicle place, both sides all marks with rectangle frame, and marks the distance of each target under rectangle frame.If setting safety distance is reported to the police, once there be the distance of target to be less than safety distance, then the buzzer phone on flush bonding module is reported to the police; If setting collision time is reported to the police, according to the concrete time of setting, such as 2.5s, once there be the collision time of target to be less than 2.5s, then buzzer phone is reported to the police.Whole system adopts the cigar lighter socket of automobile to power.
Fig. 8 is the contrast of instantaneous collision time Tm and acceleration correction collision time T, and suppose that the distance between two cars is 30m, the relative rate of onset of two cars is 10m/s, and relative acceleration is 2m/s
2(two cars accelerate close).When target is far away, instantaneous collision time T
mobviously be greater than acceleration correction collision time T; When target is nearer, two kinds of collision times trend towards unanimously.In addition, instantaneous collision time T
mchange along with the time presents non-linear, and instantaneous collision time T is described
mreal collision time can not be reacted exactly, then present the trend of good linear change through the correction collision time T of acceleration correction.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.
Claims (12)
1. a traffic detecting and tracking prior-warning device, it is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles or pedestrians outside this vehicle are detected and follows the tracks of range finding, calculate the distance with other vehicles or pedestrians, and send early warning, it is characterized in that, this traffic detecting and tracking prior-warning device comprises:
Image pre-processing module, it is for carrying out pretreatment to the infrared video obtained from this noctovisor: by filtering method filtering noise, then uses the method for adaptive histogram equalization to strengthen picture contrast;
Detection module, it, for carrying out detection & localization to other vehicles or pedestrians in this video, detects target vehicle or target pedestrian;
Tracking module, it, for carrying out predicting tracing to target vehicle or target pedestrian, follows the tracks of confirmation further;
Range finder module, it is for calculating the distance through following the tracks of target vehicle or the target pedestrian confirmed, this distance meets following formula:
Wherein, d is the distance of this vehicle-to-target vehicle or target pedestrian; F is the effective focal length of this noctovisor; α is the pitch angle of this noctovisor; H is the height of camera lens to road surface of this noctovisor; (x
0, y
0) be the optical axis of this camera lens and the intersection point of picture plane, be the initial point of photo coordinate system; (x, y) is the projection coordinate of P point in picture plane, and this P point takes from the target vehicle or target pedestrian that confirm through tracking, as the R point of the distance d with this vehicle;
Judge module, it is for judging whether this distance d is less than a predetermined collision distance;
Display and warning module, it is for when this distance d is less than this predetermined collision distance, starts audible alarm, and starts display alarm;
Wherein, this tracking module adopts Kalman filter method in conjunction with Mean Shift method, sets up the target model being used for vehicles or pedestrians and following the tracks of, and in tracing process, upgrades target model parameter, the impact that this vehicle of filtering brings in vibration processes, and the empty target of filtration fraction.
2. traffic detecting and tracking prior-warning device as claimed in claim 1, it is characterized in that, according to the feature of infrared video, namely the temperature of vehicles or pedestrians is relatively high, this detection module, for adopting the way of rim detection and feature identification, detects target vehicle or target pedestrian and locates.
3. traffic detecting and tracking prior-warning device as claimed in claim 1, it is characterized in that, this display and warning module, also for adopting different corresponding sound and light alarm modes according to different distance d sectional, make the frequency of alarm sound and icon sectional change along with the change of distance d that is strong and weak and display alarm.
4. traffic detecting and tracking prior-warning device as claimed in claim 1, it is characterized in that, this traffic detecting and tracking prior-warning device also comprises: module during survey, it is for going out instantaneous collision time according to the change calculations of target vehicle or target pedestrian dimension scale in video, and this instantaneous collision time meets following formula:
Wherein, T
mfor instantaneous collision time; △ t is instantaneous collision time T
mand the interval between two moment of current time; S is the ratio change of target vehicle or target pedestrian size in two moment videos, target vehicle or target pedestrian near time, S>1;
This judge module, it is for judging instantaneous collision time T
mwhether be less than a pre-warning time;
Display and warning module, it is at instantaneous collision time T
mwhen being less than this pre-warning time, starting audible alarm, and start display alarm.
5. traffic detecting and tracking prior-warning device as claimed in claim 4, is characterized in that, this display and warning module are also for according to different instantaneous collision time T
msectional adopts different corresponding sound and light alarm modes, makes the icon of the frequency of alarm sound and power and display alarm along with instantaneous collision time T
mchange and sectional change.
6. traffic detecting and tracking prior-warning device as claimed in claim 4, it is characterized in that, this traffic detecting and tracking prior-warning device also comprises: correcting module, and it is for instantaneous collision time T
mcarry out acceleration correction, obtain the correction collision time meeting following formula:
Wherein, T represents instantaneous collision time T
mthe correction collision time obtained through acceleration correction;
represent instantaneous collision time T
mto time differentiate;
This judge module, it is for judging whether revise collision time T is less than the anti-collision warning time;
Display and warning module, it, for when revising collision time T and being less than this anti-collision warning time, starts audible alarm, and starts display alarm.
7. traffic detecting and tracking prior-warning device as claimed in claim 6, it is characterized in that, this display and warning module, also for adopting different corresponding sound and light alarm modes according to different correction collision time T sectional, make the frequency of alarm sound and icon sectional change along with the change of correction collision time T that is strong and weak and display alarm.
8. traffic detecting and tracking prior-warning device as claimed in claim 6, it is characterized in that, this traffic detecting and tracking prior-warning device also comprises: brake command forms module, its for be less than a predetermined minimum collision distance at distance d or revise collision time T be less than the predetermined minimum anti-collision warning time time, form brake command, this brake command is used for implementing braking by the CAN of this vehicle to this vehicle.
9. a traffic detecting and tracking method for early warning, it is applied in the vehicle being provided with passive type monocular noctovisor, other vehicles or pedestrians outside this vehicle are detected and follows the tracks of range finding, calculate the distance with other vehicles or pedestrians, and send early warning, it is characterized in that, this traffic detecting and tracking method for early warning comprises the following steps:
Pretreatment is carried out to the infrared video obtained from this noctovisor: by filtering method filtering noise, then use the method for adaptive histogram equalization to strengthen picture contrast;
Detection & localization is carried out to other vehicles or pedestrians in this video, detect target vehicle or target pedestrian: according to the feature of infrared video, namely the temperature of vehicles or pedestrians is relatively high, adopts the way of rim detection and feature identification, detects target vehicle or target pedestrian and locates;
Predicting tracing is carried out to target vehicle or target pedestrian, further tracking confirms: adopt Kalman filter method in conjunction with Mean Shift method, set up the target model being used for vehicles or pedestrians and following the tracks of, and in tracing process, upgrade target model parameter, the impact that this vehicle of filtering brings in vibration processes, and the empty target of filtration fraction;
Calculate the distance through following the tracks of target vehicle or the target pedestrian confirmed, this distance meets following formula:
Wherein, d is the distance of this vehicle-to-target vehicle or target pedestrian; F is the effective focal length of this noctovisor; α is the pitch angle of this noctovisor; H is the height of camera lens to road surface of this noctovisor; (x
0, y
0) be the optical axis of this camera lens and the intersection point of picture plane, be the initial point of photo coordinate system; (x, y) is the projection coordinate of P point in picture plane, and this P point takes from the target vehicle or target pedestrian that confirm through tracking, as the R point of the distance d with this vehicle;
Judge whether this distance d is less than a predetermined collision distance;
When this distance d is less than this predetermined collision distance, starts audible alarm, and start display alarm.
10. traffic detecting and tracking method for early warning as claimed in claim 9, it is characterized in that, this traffic detecting and tracking method for early warning is further comprising the steps of:
Change calculations according to target vehicle or target pedestrian dimension scale in video goes out instantaneous collision time, and this instantaneous collision time meets following formula:
Wherein, T
mfor instantaneous collision time; △ t is instantaneous collision time T
mand the interval between two moment of current time; S is the ratio change of target vehicle or target pedestrian size in two moment videos, target vehicle or target pedestrian near time, S>1;
Judge instantaneous collision time T
mwhether be less than a pre-warning time;
At instantaneous collision time T
mwhen being less than this pre-warning time, starting audible alarm, and start display alarm.
11. traffic detecting and tracking method for early warning as claimed in claim 10, is characterized in that, this traffic detecting and tracking method for early warning is further comprising the steps of:
To instantaneous collision time T
mcarry out acceleration correction, obtain the correction collision time meeting following formula:
Wherein, T represents instantaneous collision time T
mthe correction collision time obtained through acceleration correction;
represent instantaneous collision time T
mto time differentiate;
Judge whether revise collision time T is less than the anti-collision warning time;
When revising collision time T and being less than this anti-collision warning time, start audible alarm, and start display alarm.
12. traffic detecting and tracking method for early warning as claimed in claim 11, it is characterized in that, this traffic detecting and tracking method for early warning is further comprising the steps of: adopt different corresponding sound and light alarm modes according to different correction collision time T sectional, makes the frequency of alarm sound and icon sectional change along with the change of correction collision time T that is strong and weak and display alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310096452.XA CN103204123B (en) | 2013-03-25 | 2013-03-25 | Vehicle-pedestrian detecting, tracking and early-warning device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310096452.XA CN103204123B (en) | 2013-03-25 | 2013-03-25 | Vehicle-pedestrian detecting, tracking and early-warning device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103204123A CN103204123A (en) | 2013-07-17 |
| CN103204123B true CN103204123B (en) | 2015-07-08 |
Family
ID=48751597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310096452.XA Active CN103204123B (en) | 2013-03-25 | 2013-03-25 | Vehicle-pedestrian detecting, tracking and early-warning device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103204123B (en) |
Families Citing this family (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104553983B (en) * | 2013-10-24 | 2017-02-15 | 鸿富锦精密工业（深圳）有限公司 | Vehicle pre-warning system and vehicle pre-warning method |
| CN103617747B (en) * | 2013-11-07 | 2016-03-23 | 北京智谷睿拓技术服务有限公司 | Information processing method and car-mounted terminal, handheld device |
| CN104044503A (en) * | 2014-07-03 | 2014-09-17 | 安庆市鸿裕工业产品设计有限公司 | Trailing vehicle identifying and prompting device |
| JP6239114B2 (en) * | 2014-07-22 | 2017-11-29 | 三菱電機株式会社 | Driving support system, driving support method and program |
| CN104966064A (en) * | 2015-06-18 | 2015-10-07 | 奇瑞汽车股份有限公司 | Pedestrian ahead distance measurement method based on visual sense |
| CN105203034B (en) * | 2015-07-29 | 2018-07-17 | 四川大学 | A kind of survey height survey area method based on monocular cam three-dimensional ranging model |
| US9604639B2 (en) * | 2015-08-28 | 2017-03-28 | Delphi Technologies, Inc. | Pedestrian-intent-detection for automated vehicles |
| CN105882523A (en) * | 2015-12-08 | 2016-08-24 | 乐视移动智能信息技术（北京）有限公司 | Detection method and device of safe driving |
| CN106909929A (en) * | 2015-12-22 | 2017-06-30 | 比亚迪股份有限公司 | Pedestrian's distance detection method and device |
| CN105706145A (en) * | 2016-01-31 | 2016-06-22 | 冯旋宇 | Vehicle image distance measuring method and system |
| CN106156725A (en) * | 2016-06-16 | 2016-11-23 | 江苏大学 | A kind of method of work of the identification early warning system of pedestrian based on vehicle front and cyclist |
| US10127465B2 (en) * | 2016-06-27 | 2018-11-13 | Mobileye Vision Technologies Ltd. | Controlling host vehicle based on detected movement of a target vehicle |
| CN107226023A (en) * | 2017-04-10 | 2017-10-03 | 江苏车视杰电子有限公司 | Image procossing real-time early warning system and method |
| CN107226024B (en) * | 2017-04-24 | 2019-06-21 | 惠州市凯越电子股份有限公司 | A kind of high-precision vehicle sighting distance acquisition and processing system and method |
| EP3416307B1 (en) * | 2017-06-12 | 2022-07-27 | STMicroelectronics (Research & Development) Limited | Vehicle communications using visible light communications |
| CN107591026B (en) * | 2017-09-11 | 2021-04-02 | 苏州莱孚斯特电子科技有限公司 | Pedestrian detection and early warning method |
| CN109583267B (en) * | 2017-09-28 | 2022-04-08 | 京东方科技集团股份有限公司 | Vehicle target detection method, vehicle target detection device, and vehicle |
| CN109747566A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of automatic night vision system |
| CN107959796A (en) * | 2017-12-08 | 2018-04-24 | 朱明君 | It is a kind of that there is the camera device of target detection |
| CN110001517A (en) * | 2018-01-05 | 2019-07-12 | 聚晶半导体股份有限公司 | Driving warning method and driving warning system |
| CN110622080B (en) * | 2018-03-26 | 2023-07-25 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle tracking processing method and control terminal |
| DE112018008107B4 (en) * | 2018-11-26 | 2022-05-19 | Mitsubishi Electric Corporation | INFORMATION PRESENTATION CONTROL DEVICE, INFORMATION PRESENTATION CONTROL DEVICE, INFORMATION PRESENTATION CONTROL METHOD, PROGRAM AND RECORDING MEDIUM |
| US11605298B2 (en) | 2020-01-29 | 2023-03-14 | Toyota Motor Engineering & Manufacturing North America, Inc. | Pedestrian navigation based on vehicular collaborative computing |
| CN117409070A (en) * | 2020-10-30 | 2024-01-16 | 深圳市优必选科技股份有限公司 | Vehicle operation control method and device, electronic equipment and storage medium |
| CN112356815B (en) * | 2020-12-01 | 2023-04-25 | 吉林大学 | Pedestrian active collision avoidance system and method based on monocular camera |
| CN113401082A (en) * | 2021-08-02 | 2021-09-17 | 姜春诗 | Self-judging automobile safety automatic braking system and method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06270780A (en) * | 1993-03-23 | 1994-09-27 | Mazda Motor Corp | Safety device for vehicle |
| DE102004046101A1 (en) * | 2004-09-23 | 2006-09-21 | Daimlerchrysler Ag | Method for early detection of motor vehicle collision involves estimation of collision danger on the basis of determined time to collision and introducing collision reducing measures with exceeding of predetermined collision danger |
| CN101739843A (en) * | 2009-12-04 | 2010-06-16 | 河海大学常州校区 | Device and method for real-time three-dimensionally sensing safe driving of vehicle having mixed multi-visual information |
| CN201800639U (en) * | 2010-08-20 | 2011-04-20 | 叶春林 | Vehicle sight line monitoring and warning device |
| CN201901079U (en) * | 2009-05-05 | 2011-07-20 | 杨斌 | Automobile collision-prevention device |
| CN102963326A (en) * | 2011-08-30 | 2013-03-13 | 通用汽车环球科技运作有限责任公司 | Detection apparatus and method for detecting a carrier of a transceiver, motor vehicle |
-
2013
- 2013-03-25 CN CN201310096452.XA patent/CN103204123B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06270780A (en) * | 1993-03-23 | 1994-09-27 | Mazda Motor Corp | Safety device for vehicle |
| DE102004046101A1 (en) * | 2004-09-23 | 2006-09-21 | Daimlerchrysler Ag | Method for early detection of motor vehicle collision involves estimation of collision danger on the basis of determined time to collision and introducing collision reducing measures with exceeding of predetermined collision danger |
| CN201901079U (en) * | 2009-05-05 | 2011-07-20 | 杨斌 | Automobile collision-prevention device |
| CN101739843A (en) * | 2009-12-04 | 2010-06-16 | 河海大学常州校区 | Device and method for real-time three-dimensionally sensing safe driving of vehicle having mixed multi-visual information |
| CN201800639U (en) * | 2010-08-20 | 2011-04-20 | 叶春林 | Vehicle sight line monitoring and warning device |
| CN102963326A (en) * | 2011-08-30 | 2013-03-13 | 通用汽车环球科技运作有限责任公司 | Detection apparatus and method for detecting a carrier of a transceiver, motor vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103204123A (en) | 2013-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103204123B (en) | Vehicle-pedestrian detecting, tracking and early-warning device and method | |
| CN103158620B (en) | A kind of traffic detecting and tracking forewarn system | |
| Garcia et al. | Data fusion for overtaking vehicle detection based on radar and optical flow | |
| US10889232B2 (en) | Vehicle control method that compares a light distribution pattern extracted from an image captured by a camera with a reference light distribution pattern | |
| CN103909926B (en) | The lateral collision-proof method of vehicle, equipment and system | |
| CN102361777B (en) | Device for generating sound to outside of vehicle | |
| CN203854604U (en) | Intelligent blind spot monitoring device for automobile | |
| CN106355919B (en) | A kind of bend speed speed limit calculation method and hypervelocity alarming device based on machine vision | |
| CN104786933A (en) | Panoramic image driving auxiliary device and panoramic image driving auxiliary method | |
| CN107031508A (en) | Back-sight visual system and its application method for vehicle | |
| CN107284355A (en) | A kind of Safety vehicle door opens processing method and system | |
| CN110329152A (en) | It is a kind of using algorithm of target detection as the driving blind area early warning system of core | |
| CN105620489A (en) | Driving assistance system and real-time warning and prompting method for vehicle | |
| CN101016052A (en) | Warning method and system for preventing deviation for vehicle on high standard highway | |
| US8140226B2 (en) | Security system and a method to derive a security signal | |
| CN103072537A (en) | Automotive collision avoidance safety protecting device and protecting method based on infrared image processing | |
| CN203332106U (en) | Automobile warning anti-collision system based on monocular vision | |
| CN107408338A (en) | Driver assistance system | |
| CN104773177A (en) | Aided driving method and aided driving device | |
| KR101296248B1 (en) | Virtual Roadway Safety Tripod | |
| CN109001743A (en) | Tramcar anti-collision system | |
| CN108664029A (en) | A kind of unpiloted intelligent automobile and its control method | |
| KR101861525B1 (en) | Pedestrian Protection System for autonomous car | |
| KR20160091040A (en) | Vehicle and Control Method Thereof | |
| JP2022013838A (en) | System and method for providing truck-mounted sensor to detect trailer following vehicle and trailer condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |