CN105844257B - The early warning system and method for road sign are missed based on machine vision travelling in fog day - Google Patents

The early warning system and method for road sign are missed based on machine vision travelling in fog day Download PDF

Info

Publication number
CN105844257B
CN105844257B CN201610218031.3A CN201610218031A CN105844257B CN 105844257 B CN105844257 B CN 105844257B CN 201610218031 A CN201610218031 A CN 201610218031A CN 105844257 B CN105844257 B CN 105844257B
Authority
CN
China
Prior art keywords
image
road
classifier
module
driver
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201610218031.3A
Other languages
Chinese (zh)
Other versions
CN105844257A (en
Inventor
金立生
陈梅
张豪翊
王发继
刘辉
程蕾
杨诚
李科勇
高琳琳
谢宪毅
管信
姬生远
徐俊
郑义
张昊
郭柏苍
岳欣羽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jilin University
Original Assignee
Jilin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jilin University filed Critical Jilin University
Priority to CN201610218031.3A priority Critical patent/CN105844257B/en
Publication of CN105844257A publication Critical patent/CN105844257A/en
Application granted granted Critical
Publication of CN105844257B publication Critical patent/CN105844257B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G06T5/73—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8006—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/804—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for lane monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30268—Vehicle interior

Abstract

The present invention is based on early warning systems and method that machine vision travelling in fog day misses road sign, belong to intelligent vehicle safety auxiliary driving technology field.The present invention identifies neuroid classifier, driver fatigue pattern classifier by road and traffic sign plates recognition classifier, road and traffic sign plates character classification, has fogless Image Classifier, image defogging model, constructs Real-time Road landmark identification prediction policy.Track, frequency of wink, PERCLOS value are watched attentively according to driver's sight, if it is determined that driver's driving fatigue or in driving procedure execute subtask lead to miss road signs information, then vehicle-mounted loudspeaker issues the audible alert of real-time sign board information and in the flag information of car-mounted display screen display warning, simultaneously if it is determined that the greasy weather, it equally gives driver and vision, aural alert is provided, it realizes under the conditions of missing road sign in greasy weather and driver, driver is made to obtain road and traffic sign plates information and traffic safety early warning.

Description

The early warning system and method for road sign are missed based on machine vision travelling in fog day
Technical field
The invention mainly relates to intelligent vehicle safeties to assist driving technology field, in particular to a kind of to be based on machine vision mist It, which is driven a vehicle, misses the early warning system and method for road sign.
Background technique
Traffic sign provides the information such as condition of road surface and traffic condition for driver, the generation to road traffic accident is reduced It is of great significance.According to statistics, every year due to according to traffic instruction traveling illegal activities up to 30,000,000, driver because Road markings information has not been obtained and caused by traffic accident occupy sizable specific gravity, and specific gravity in whole traffic accident numbers It is high, quite a few the reason is that: execute subtask due to driver's driving fatigue or in driving procedure, cause Miss road caution mark；Night is bad, foggy weather leads to low visibility, also results in driver and misses road letter Breath causes accident thereby executing faulty operation.Currently, although vehicle GPS onboard navigation system is in terms of positioning road sign, Develop fairly perfect, but still have the following deficiencies: if navigation system updates not in time will appear navigational error, obtain it is certain The information in unstructured road area, the defects such as speed is slow, precision is low.It would therefore be highly desirable to a kind of intelligent driving auxiliary system, it can It assists driver to obtain road sign information, improves driving safety.
Currently, sign board information detecting method is broadly divided into detection method based on sign board shape, based on template matching Detection method, be based on GPS technology detection method.Wherein, that there are detection effects is low, real for detection method based on sign board shape The problems such as when property is bad；Detection method based on template matching, there are the algorithm process time is longer, discrimination is very low, cannot get It is expected that the disadvantages of desired result；Detection method based on GPS technology, this method can generate large error when shade is with blocking. The mark type identified in existing traffic mark board recognition methods is limited, therefore driver obtains Limited information, and is applied to It is seldom in real road environment.The early warning system that road sign is missed based on machine vision travelling in fog day that this patent proposes And method, identification process will not the normal driving behavior to driver interfere, overcome because of driving fatigue, in driving procedure It executes that subtask, night be bad and foggy weather low visibility, driver is caused to miss traffic instruction information, and cause The generation of road safety accident.Using machine vision blending image defogging technology, road instruction information is obtained in real time, while being sentenced Disconnected driver warns since fatigue is missed road sign and given, and makes driver under complicated road environment, specifies road environment spy Point reduces the probability that traffic accident occurs, has become domestic and international research hotspot.
Summary of the invention
Road sign is missed based on machine vision travelling in fog day the technical problems to be solved by the present invention are: providing one kind Early warning system and method, for solve driver fatigue drive, in driving procedure execute subtask and miss road sign Or since night is bad and foggy weather low visibility, driver is caused not understand road traffic environment information, and The technical issues of causing road traffic accident.
The present invention compensates for existing vehicle mounted GPS guidance system and is obtaining following defect existing for road sign information: if GPS navigation system update not in time will appear navigational error, obtain the information rate in certain unstructured road areas slowly, precision The problems such as low.
The present invention is implemented with the following technical solutions:
A kind of early warning system for missing road sign based on machine vision travelling in fog day, it is characterised in that: including power supply, Transformation plug I, infrared camera I, transformation plug II, onboard electronic control unit module, image classification module, image defogging module, Sign board identification module, sign board character recognition module, driving behavior categorization module, warning module, vehicle-carrying display screen, automobile sound Equipment, vehicle-mounted loudspeaker and infrared camera II are rung, the power supply is connect by transformation plug I with infrared camera I, and power supply is logical Transformation plug II is crossed to connect with infrared camera II；
The onboard electronic control unit module includes image classification module, image defogging module, sign board identification module, mark Board character recognition module, driving behavior categorization module, warning module, one end of described image categorization module by conducting wire with it is infrared Camera I connects, and the other end of image classification module is connect by conducting wire with image defogging module；Described image defogging module is logical Conducting wire is crossed to connect with sign board identification module；The sign board identification module is connected by conducting wire and sign board character recognition module It connects；The sign board character recognition module is connect by conducting wire with warning module；One end of the driving behavior categorization module is logical It crosses conducting wire to connect with infrared camera II, the other end of driving behavior categorization module is connect by conducting wire with warning module；
The vehicle-carrying display screen is connect by conducting wire with warning module；One end of the hoot device by conducting wire with The other end of warning module connection, hoot device is connect by conducting wire with vehicle-mounted loudspeaker.
A kind of identification of vehicle mounted road sign board and early warning system method in machine vision, it is characterised in that:
Include the following steps
Step 1: establishing road and traffic sign plates identifying system
A, road and traffic sign plates gauss hybrid models GMM classifier is established
1. infrared camera acquisition N has the infrared road image of road and traffic sign plates and infrared without road and traffic sign plates Road image, and acquired image is transferred to sign board identification module, including N1Opening has road and traffic sign plates red Outer road image and N2It opens without the infrared road image of road and traffic sign plates, N, N1、N2It is natural number, identifies mould in sign board The training set of images of road and traffic sign plates classifier is established in block；
2. carrying out binaryzation, slant correction and contracting to the image in the training set of images of road and traffic sign plates classifier Size normalized is put, the image training library of the road and traffic sign plates classifier of uniform sizes is obtained；
3. using the image training library off-line training gauss hybrid models GMM classifier of road and traffic sign plates classifier, Gauss hybrid models GMM classifier is special according to the textural characteristics, area features, chamfered shape of road and traffic sign plates metal material Sign and road and traffic sign plates histogram feature, according to the infrared road image progress whether there is or not road and traffic sign plates to acquisition Classification obtains the characteristics of image for having road and traffic sign plates, and gauss hybrid models GMM classifier, which is established, to be completed；
B, the neuroid classifier for the identification of road and traffic sign plates character classification is established
1. infrared camera acquires L road and traffic sign plates images, L is natural number, to the road signs of acquisition Board image carries out histogram equalization processing, threshold filter, lookup profile algorithm and is split to character therein, to segmentation Character size normalized, uniformly reduce its size to 32*16 pixel；
2. establishing Chinese character network, alphabetical network, digital network, graphical symbol network to friendship according to the information on sign board Logical sign board information is classified；
Using all 6763 Chinese characters and 571 characters of the GB2312-80 in Chinese character base CCLIB, traffic sign is established Chinese character, letter and the digital dot array character repertoire of board；
Figure sample database is established, K traffic mark board images of acquisition, K is natural number, extracts logotype therein, root According to the graphic characteristics of traffic mark board, extracts its graphical symbol and establish graphical symbol dot chart and be stored in figure sample database；
Chinese character, letter and digital dot array character repertoire and figure sample database are merged into sign board information identification library；
3. constructing Back propagation neural metanetwork classifier structure
A) input layer number
To the character for being normalized to 32*16 dot matrix size, with each pixel for a grid, input layer number Take 512；
B) output layer neuron number
Being respectively as follows: Chinese character according to heterogeneous networks output layer neuron number is 1000；Letter 26；Number 10；Graphical symbol It is 500；
C) hidden layers numbers are one layer, One hidden layer neuron number calculation formula are as follows:
In formula, h_num is hidden neuron number, and i_num is input layer number, and o_num is output layer nerve First number；
D) activation primitive selects logistic functional form as follows:
In formula, vjFor the local field of neuron j,For the output function of hidden layer；
4. artificial neural network's training
Sign board information is identified that four kinds of Chinese character, letter, number and figure samples in library are separately input to corresponding mind Through in metanetwork, and Studying factors, factor of momentum, error target value and threshold range are carried out to four networks respectively and set It is fixed, the recognition training of artificial neural network is then carried out, input layer to hidden layer weight or hidden layer are not being set to output layer weight In the threshold range set, then shows artificial neural network's failure to train, exit training；Input layer is arrived to hidden layer weight and hidden layer Output layer weight then saves weight in the threshold range of setting, and artificial neural network trains successfully；
5. sign board character recognition
The weight matrix that training obtains is input in the weight matrix of corresponding neuroid structure, by word to be identified Symbol sample is saved in file in the matrix form, the classification of the character sample is identified using neuroid after training, and defeated Result out；
6. artificial neural network tests, the accuracy rate of identification is calculated,
Manual record exports result for 100 times or more, and after being trained neuroid recognition accuracy, if accurately Rate is lower than 95%, then repeats step 4. artificial neural network's training and step 5. sign board character recognition, if accurately Rate is 95% or 95% or more, then the neuroid classifier training with robustness is completed；
Step 2: establishing Fatigue pattern classifier
1. acquiring driving behavior data of the M1 drivers under normal driving state, M2 drivers of acquisition drive in fatigue Sail the driving behavior data under state, wherein M1, M2 are natural number；
2. extracting fatigue driving characteristic parameter using feature extracting method, normal driving is obtained using the method for statistical analysis With the characteristic parameter of fatigue driving, determine that driver's sight track, frequency of wink and PERCLOS are fatigue driving characteristic parameter；
3. infrared camera II (15) tracks driver's sight track, drives the blinkpunkt position of human eye and identify Traffic mark board position is inconsistent, then sends the signal that driver misses road signs；Drive human eye watches point attentively The traffic mark board position consistency set and identified, then record the time that driver watches the region attentively, and the time is no more than 1 second, then Send the signal that driver misses road signs；Time is more than 1 second, then carries out in next step；
4. infrared camera II (15) records the frequency of wink and wink time of driver, frequency of wink threshold range is set Be 2 seconds/time~4 seconds/time, each wink time threshold range be 0.25 second~0.3 second, the driver of record be frequency of wink or Wink time in given threshold range, does not then send the signal that driver misses road signs；The driver of record is Frequency of wink and wink time then carry out in next step in given threshold range；
5. according to measurement fatigue/drowsiness physical quantity PERCLOS,
The eyelid size under 100 groups of normal driving states is taken to obtain driver's left eyelid average-size L driverELMWith Driver's right eyelid average-size RELM,
Eyes closed degree is more than 80% frequency n in unit timepCalculation formula be
In formula, LELJFor the left eyelid size of jth frame image, RELJFor the right eyelid size of jth frame image；Unit time Interior eyes open degree less than 20% shared by ratio P80Evaluation criterion P as PERCLOS80Calculation formula be
In formula, f0For sample frequency, TP80For computation window size；
The calculation formula of measurement fatigue/drowsiness physical quantity PERCLOS is as follows:
Eyes closed is set to 80% and 80% or more shared time threshold as 30 seconds/point, infrared camera II detects Eyes closed is more than given threshold value, then sends driver and miss to 80% and 80% or more shared time in the unit time The signal of road signs；
6. establishing characteristic ginseng value under the conditions of fatigue driving according to the driving behavior data statistics under normal and fatigue state Database, utilize machine learning method construct Fatigue pattern classifier；
Step 3: being built with fog free images classifier
A, road image support vector machines classifier is established
1. infrared camera acquires X infrared road images and infrared non-rice habitats images, and acquired image is transmitted To there is fogless image classification module, including X1Open infrared road image and X2Open infrared non-rice habitats image, X, X1、X2It is Natural number establishes the image training library of road image classifier in having fogless image classification module；
2. by the image training library off-line training support vector machines classifier of road image classifier, supporting vector Machine SVM classifier obtains road image feature according to road texture, to road image and non-rice habitats image classification, and obtains infrared Road image feature, road image support vector machines classifier, which is established, to be completed；
B, establishing has fogless Image Classifier
1. infrared camera acquires the infrared foggy image and infrared fog free images of Y different mistiness degree, including Y1 Opening has mist infrared image and Y2Open fogless infrared image, Y, Y1、Y2It is natural number, is established in having fogless image classification module There is the image training library of fogless Image Classifier；
2. by having the image of fogless Image Classifier training library off-line training gauss hybrid models GMM classifier, Gauss Mixed model GMM classifier extracts the one-dimensional gray scale of foggy image and fog free images according to foggy image gray value frequecy characteristic Histogram image feature, and classified according to the histogram of image to foggy image and fog free images, there is fog free images Gauss Mixed model GMM classifier, which is established, to be completed；
3. by have the image of fogless Image Classifier training library off-line training support vector machines classifier, support to Amount machine SVM classifier is according to the Fourier transformation frequecy characteristic of foggy image and fog free images, to foggy image and fog free images Classify, and extract the Fourier transformation frequecy characteristic of foggy image and fog free images, there are fog free images support vector machines Classifier, which is established, to be completed；
By the foggy image for thering is fog free images gauss hybrid models GMM classifier to sort out and there is fog free images supporting vector The foggy image that machine SVM classifier sorts out takes union, obtains foggy image sample database；
Step 4: establishing image defogging model
1. atmospheric optics model is I (x)=J (x) e-βd+A(1-e-βd), I (x) is sorted foggy image, and J (x) is Clear image after defogging, A are global atmosphere light ingredient, e-βdIt is atmospheric extinction coefficient for atmospheric transmissivity value t, β, d is energy See angle value；
2. smallest passage gray level image of the mist road image in RGB RGB triple channel image is taken, then again to obtaining The gray level image taken does mini-value filtering, obtains foggy image dark:Wherein, JdarkRefer to J Dark, JCIndicate each channel of color image, C is RGB triple channel；Ω is the whole image window for having all pixels,For the minimum value pixel in whole image window all pixels,For tri- channels each pixel RGB of whole image window The minimum pixel value of component；
3. preceding 0.1% pixel is taken according to brightness size from dark channel image, then it is original have found in mist figure pair The value for the point with maximum brightness answered, the signal component value A as atmosphere light；
4. handling atmospheric optics model, form is as follows:
I (x)=J (x) e-βd+A(1-e-βd)
Wherein C is RGB triple channel, asks dark to ask minimum operation twice, then benefit to above formula both sides above formula both ends With dark gray value close to zero, therefore, can derive:
Wherein t is atmospheric transmissivity value, since the presence of mist makes one to feel the presence of the depth of field,The middle factor ω introduced between one [0,1], obtains atmospheric transmissivity figure,
5. there is loss at the edge and grain details that obtain transmittance figure, side is carried out to transmittance figure using median filter The filter optimization that edge is kept obtains refinement transmittance figure；
6. threshold value t is arranged0, when t value is less than t0When, enable t=t0, with t0=0.1 is standard, will treated triple channel figure As synthesis, the clear image J (x) after recovering defogging,
Step 5: building road sign identifies prediction policy
1. infrared camera I acquires realtime graphic
In vehicular motion, infrared camera I acquires realtime graphic, and acquired image has been sequentially inputted to Fog free images classifier and image defogging model carry out image defogging to foggy image；
2. by after defogging image or real-time fog free images be input to road and traffic sign plates identifying system, by acquisition Road and traffic sign plates image is transferred to sign board character recognition module as input, will in sign board character recognition module After road and traffic sign plates image carries out image segmentation pretreatment, carried out using the neuroid classifier that off-line training is completed Character recognition；Road signs information after identification inputs warning module and makes a sound warning simultaneously by vehicle-mounted loudspeaker In the car-mounted display screen display road signs information；
3. the acquisition of infrared camera II driver's sight blinkpunkt, fixation time, frequency of wink and PERCLOS are input to tired Labor pattern classifier, driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS are within the scope of given threshold value, then It is not applied to driver's warning information；Driver's sight blinkpunkt, fixation time, frequency of wink or PERCLOS be not in given threshold It is worth in range, then warning module is made a sound by vehicle-mounted loudspeaker and alerted and in the car-mounted display screen display road traffic mark Will information.
N >=3000 in the step 1, N1>=1000, N2≥2000。
L >=1000 in the step 1, K >=2000.
M in the step 21>=50, M2≥50。
X >=3000 in the step 3, X1>=1000, X2≥2000。
Y >=3000 in the step 3, Y1>=1000, Y2≥2000。
Through the above design, the present invention can be brought the following benefits:
1, the present invention is based on early warning systems and method that machine vision travelling in fog day misses road sign, by road traffic Road and traffic sign plates character classification is established in sign board gauss hybrid models GMM classifier, identification road and traffic sign plates position Identify that neuroid classifier, real-time grading identification road sign character information establish eye tracker detection model, real-time root According to eye tracker detection model judge driver driving status whether fatigue or perform subtask, judge whether driver wrong Traffic route sign board character information is crossed, early warning is carried out.
In practical driving conditions, the road video image in vehicular motion is acquired in real time, according to off-line training Road sign image classification library, whether real-time judge image has road sign, if it is judged that have road sign, according to The road sign character classification of off-line training identifies library, and real-time grading identifies the character of road sign board, and information is shown On vehicle-carrying display screen；In real time according to eye tracker detection model, judge whether driver misses road sign information, gives driver Vision and aural alert realize the early warning dispersed to driver attention in driving procedure, and driver is made to obtain sufficient road Environmental information weakens the influence that dispersion attention generates driver's normal driving.
2, signal processing time is short between inside modules of the invention, module and module, can satisfy the requirement of real-time.
3, the present invention carries out Classification and Identification to road sign character, the accuracy rate of character recognition is improved, to driver The road environment information missed is supplemented, and is enhanced visual information content, is beneficial to promote and apply, driver can be greatly reduced and exist Driving fatigue executes driving subtask because missing road signs information, and the probability of pernicious traffic accident occurs.
Detailed description of the invention
Below in conjunction with the drawings and specific embodiments, the present invention is further illustrated:
Fig. 1 is the flow diagram for missing the method for early warning of road sign in the present invention based on machine vision travelling in fog day.
Fig. 2 is the structural block diagram for missing the early warning system of road sign in the present invention based on machine vision travelling in fog day.
1- power supply in figure, 2- transformation plug I, 3- infrared camera I, 4- transformation plug II, 5- onboard electronic control unit module, 6- image classification module, 7- image defogging module, 8- sign board identification module, 9- sign board character recognition module, 10- drive row For categorization module, 11- warning module, 12- vehicle-carrying display screen, 13- hoot device, 14- vehicle-mounted loudspeaker, 15- is infrared takes the photograph As head II.
Specific embodiment
As shown, a kind of early warning system for missing road sign based on machine vision travelling in fog day, it is characterised in that: Including power supply 1, transformation plug I 2, infrared camera I 3, transformation plug II 4, onboard electronic control unit module 5, image classification module 6, image defogging module 7, sign board identification module 8, sign board character recognition module 9, driving behavior categorization module 10, early warning mould Block 11, vehicle-carrying display screen 12, hoot device 13, vehicle-mounted loudspeaker 14 and infrared camera II 15, the power supply 1 pass through change Pressure plug I 2 is connect with infrared camera I 3, and power supply 1 is connect by transformation plug II 4 with infrared camera II 15；
The onboard electronic control unit module 5 include image classification module 6, image defogging module 7, sign board identification module 8, Sign board character recognition module 9, driving behavior categorization module 10, warning module 11, one end of described image categorization module 6 pass through Conducting wire is connect with infrared camera I 3, and the other end of image classification module 6 is connect by conducting wire with image defogging module 7；It is described Image defogging module 7 is connect by conducting wire with sign board identification module 8；The sign board identification module 8 passes through conducting wire and mark Board character recognition module 9 connects；The sign board character recognition module 9 is connect by conducting wire with warning module 11；The driving One end of behavior categorization module 10 is connect by conducting wire with infrared camera II 15, and the other end of driving behavior categorization module 10 is logical Conducting wire is crossed to connect with warning module 11；
The vehicle-carrying display screen 12 is connect by conducting wire with warning module 11；One end of the hoot device 13 passes through Conducting wire is connect with warning module 11, and the other end of hoot device 13 is connect by conducting wire with vehicle-mounted loudspeaker 14.
The method for early warning of road sign is missed based on machine vision travelling in fog day, comprising: pass through the road of off-line training Sign board information classifier establishes road sign information identification system；The driving behavior data in vehicular motion are acquired, It is input to the Fatigue pattern classifier of off-line training, realizes the identification to driver fatigue state；It acquires in vehicular motion Road ahead environment image is input to the presence or absence of off-line training mist classifier, realizes the identification to state of weather；To foggy image Using physics defogging technology, vision enhancement image is obtained；In driver fatigue state and under having greasy weather gaseity, according to road road sign Will board information identifies library, identifies present road knowledge information, road environment flag information is alerted in the form of vision and the sense of hearing Driver.
A kind of method for early warning for missing road sign based on machine vision travelling in fog day, it is characterised in that:
Include the following steps
Step 1: establishing road and traffic sign plates identifying system
A, road and traffic sign plates gauss hybrid models GMM classifier is established
1. infrared camera acquisition N has the infrared road image of road and traffic sign plates and infrared without road and traffic sign plates Road image, and acquired image is transferred to sign board identification module, including N1Opening has road and traffic sign plates red Outer road image and N2It opens without the infrared road image of road and traffic sign plates, N, N1、N2It is natural number, identifies mould in sign board The training set of images of road and traffic sign plates classifier is established in block；
2. carrying out binaryzation, slant correction and contracting to the image in the training set of images of road and traffic sign plates classifier Size normalized is put, the image training library of the road and traffic sign plates classifier of uniform sizes is obtained；
3. using the image training library off-line training gauss hybrid models GMM classifier of road and traffic sign plates classifier, Gauss hybrid models GMM classifier is special according to the textural characteristics, area features, chamfered shape of road and traffic sign plates metal material Sign and road and traffic sign plates histogram feature, according to the infrared road image progress whether there is or not road and traffic sign plates to acquisition Classification obtains the characteristics of image for having road and traffic sign plates, and gauss hybrid models GMM classifier, which is established, to be completed；
B, the neuroid classifier for the identification of road and traffic sign plates character classification is established
1. infrared camera acquires L different road and traffic sign plates images, L is natural number, to the road traffic of acquisition Sign board image carries out histogram equalization processing, threshold filter, lookup profile algorithm and is split to character therein, right The character size normalized of segmentation uniformly reduces it having a size of 32*16 pixel；
2. designing four neural networks: Chinese character network, alphabetical network, number according to the difference of the information category on sign board Word network, graphical symbol network realize the classification to sign board information.Using complete in the GB2312-80 in Chinese character base CCLIB 6763, portion Chinese character and 571 characters construct Chinese character, letter, the digital dot array character repertoire of sign board in this system.According to statistics, Road signs wide variety includes 46 kinds of caution sign, and 43 kinds of prohibitory sign, 30 kinds of Warning Mark, 57 kinds of road sign, It 40 kinds of fingerpost, 39 kinds of road traffic marking, 16 kinds of auxiliary sign, travels 17 kinds of distinctive emblem, road construction safety sign 26 Kind.Wherein logotype or the sign board comprising logotype account for 90% or more, therefore above-mentioned Chinese character, letter, digital dot array Character repertoire can not meet actual graphical landmark identification demand, in the invention patent, will establish in addition to Chinese character, letter, number Logotype sample database.K sign board images are acquired, extract logotype therein, the characteristics of analysis mark board figure: alert The black figure in logotype board Huang bottom, the black figure red diagonal of ban logotype white background, the white figure in Warning Mark indigo plant bottom are accused, its figure symbol is extracted Number construction dot chart deposit figure sample database in.Chinese character, letter, digital dot array character repertoire and figure sample database are merged into mark Board information identifies library, and the sign board of graphic character combination can identify library identification tag information according to sign board information.
3. constructing Back propagation neural metanetwork classifier structure
A) input layer number, to the character for being normalized to 32*16 dot matrix size, with each pixel for a net Lattice, input layer number take 512；
B) output layer neuron number is respectively as follows: Chinese character 1000 according to heterogeneous networks output layer neuron number；Letter 26；Number 10；Graphical symbol is 500；
C) hidden layers numbers are one layer, have theoretical research it is found that there is the neural network of input layer output layer and single hidden layer enough Execute arbitrarily complicated Function Mapping, One hidden layer neuron number calculation formula are as follows:
In formula, h_num indicates hidden neuron number, and i_num indicates input layer number, and o_num indicates output Layer neuron number.
D) activation primitive selects Logistic logistic functional form as follows:
In formula, vjFor the local field of neuron j,For the output function of hidden layer, e refers to being approximately equal to 2.71828 Natural logrithm bottom.
4. it is corresponding that sign board information is identified that four kinds of Chinese character, letter, number and figure samples in library are separately input to In neuroid, and Studying factors, factor of momentum, error target value and threshold range are carried out to four networks respectively and set It is fixed, the recognition training of artificial neural network is then carried out, input layer to hidden layer weight, hidden layer are wherein any to output layer weight One weight then shows artificial neural network's failure to train, exits training not in the threshold range of setting；Input layer is to hidden Layer weight and hidden layer to output layer weight in the threshold range of setting, then save weight, artificial neural network is trained to Function.
5. sign board character recognition: after creating neuroid by above step, trained weight matrix being input to In the weight matrix of corresponding neuroid structure, character sample to be identified is saved in file in the matrix form, is opened The classification of beginning identification sample, and export result
6. artificial neural network tests, the accuracy rate of identification is calculated, trained process is the mistake of continuous iteration Journey need to evaluate result, if accuracy rate is lower than 95%, needs to continuously improve in the training process, have to train The neuroid classifier of robustness, if accuracy rate is 95% or 95% or more, the neuroid with robustness Classifier training is completed；
Step 2: establishing Fatigue pattern classifier
1. acquiring driving behavior data of the M1 drivers under normal driving state, M2 drivers of acquisition drive in fatigue Sail the driving behavior data under state, wherein M1, M2 are natural number；
2. extracting fatigue driving characteristic parameter using feature extracting method, normal driving is obtained using the method for statistical analysis With the characteristic parameter of fatigue driving, determine that driver's sight track, frequency of wink and PERCLOS are fatigue driving characteristic parameter；
3. infrared camera II tracks driver's sight track, the blinkpunkt position of human eye and the traffic identified are driven Sign board position is inconsistent, then sends the signal that driver misses road signs；Drive human eye blinkpunkt position with The traffic mark board position consistency identified then records the time that driver watches the region attentively, and the time is no more than 1 second, then sends Driver misses the signal of road signs；Time is more than 1 second, then carries out in next step；
4. it is frequency of wink and wink time that infrared camera II, which records driver, frequency of wink threshold range is set as 2 Second/time~4 seconds/time, each wink time threshold range is 0.25 second~0.3 second, and the driver of record is frequency of wink, blink Time, wherein any one parameter value then sent the signal that driver misses road signs not in given threshold range； The driver of record is frequency of wink and wink time in given threshold range, then carries out in next step；
5. according to measurement fatigue/drowsiness physical quantity PERCLOS,
The eyelid size under 100 groups of normal driving states is taken to obtain driver's left eyelid average-size L driverELMWith Driver's right eyelid average-size RELM,
Eyes closed degree is more than 80% frequency n in unit timepCalculation formula be
In formula, LELJFor the left eyelid size of jth frame image, RELJFor the right eyelid size of jth frame image；Unit time Interior eyes open degree less than 20% shared by ratio P80Evaluation criterion P as PERCLOS80Calculation formula be
In formula, f0For sample frequency, TP80For computation window size；
The calculation formula of measurement fatigue/drowsiness physical quantity PERCLOS is as follows:
Eyes closed is set to 80% and 80% or more shared time threshold as 30 seconds/point, infrared camera II 15 is examined Eyes closed in the unit time is measured to be more than given threshold value, then send driver's mistake to 80% and 80% or more shared time Lose the signal of road signs；
6. establishing characteristic ginseng value under the conditions of fatigue driving according to the driving behavior data statistics under normal and fatigue state Database, utilize machine learning method construct Fatigue pattern classifier；
Step 3: being built with fog free images classifier
A, road image support vector machines classifier is established
1. infrared camera acquires X infrared road images and infrared non-rice habitats images, and acquired image is transmitted To there is fogless image classification module, including X1Open infrared road image and X2Open infrared non-rice habitats image, X, X1、X2It is Natural number establishes the image training library of road image classifier in having fogless image classification module；
2. by the image training library off-line training support vector machines classifier of road image classifier, supporting vector Machine SVM classifier obtains road image feature according to road texture, to road image and non-rice habitats image classification, and obtains infrared Road image feature, road image support vector machines classifier, which is established, to be completed；
B, establishing has fogless Image Classifier
1. infrared camera acquires the infrared foggy image and infrared fog free images of Y different mistiness degree, including Y1 Opening has mist infrared image and Y2Open fogless infrared image, Y, Y1、Y2It is natural number, is established in having fogless image classification module There is the image training library of fogless Image Classifier；
2. by having the image of fogless Image Classifier training library off-line training gauss hybrid models GMM classifier, Gauss Mixed model GMM classifier extracts the one-dimensional gray scale of foggy image and fog free images according to foggy image gray value frequecy characteristic Histogram image feature, and classified according to the histogram of image to foggy image and fog free images, there is fog free images Gauss Mixed model GMM classifier, which is established, to be completed；
3. by have the image of fogless Image Classifier training library off-line training support vector machines classifier, support to Amount machine SVM classifier is according to the Fourier transformation frequecy characteristic of foggy image and fog free images, to foggy image and fog free images Classify, and extract the Fourier transformation frequecy characteristic of foggy image and fog free images, there are fog free images support vector machines Classifier, which is established, to be completed；
By the foggy image for thering is fog free images gauss hybrid models GMM classifier to sort out and there is fog free images supporting vector The foggy image that machine SVM classifier sorts out takes union, obtains foggy image sample database；
Step 4: establishing image defogging model
1. atmospheric optics model is I (x)=J (x) e-βd+A(1-e-βd), I (x) is sorted foggy image, and J (x) is Clear image after defogging, A are global atmosphere light ingredient, e-βdIt is atmospheric extinction coefficient for atmospheric transmissivity value t, β, d is energy See angle value；
2. smallest passage gray level image of the mist road image in RGB RGB triple channel image is taken, then again to obtaining The gray level image taken does mini-value filtering, obtains foggy image dark:Wherein, JdarkRefer to J Dark, JCIndicate each channel of color image, C is RGB triple channel；Ω is the whole image window comprising all pixels Mouthful,For the minimum value pixel in whole image window all pixels,It is logical for each pixel RGB of whole image window tri- The minimum pixel value of road component；
3. preceding 0.1% pixel is taken according to brightness size from dark channel image, then it is original have found in mist figure pair The value for the point with maximum brightness answered, the signal component value A as atmosphere light；
4. handling atmospheric optics model, form is as follows:
I (x)=J (x) e-βd+A(1-e-βd)
Wherein C is RGB triple channel, asks dark to ask minimum operation twice, then benefit to above formula both sides above formula both ends With dark gray value close to zero, therefore, can derive:
Wherein t is atmospheric transmissivity value, since the presence of mist makes one to feel the presence of the depth of field,The middle factor ω introduced between one [0,1], obtains atmospheric transmissivity figure,
5. there is loss at the edge and grain details that obtain transmittance figure, side is carried out to transmittance figure using median filter The filter optimization that edge is kept obtains refinement transmittance figure；
6. threshold value t is arranged0, when t value is less than t0When, enable t=t0, with t0=0.1 is standard, will treated triple channel figure As synthesis, the clear image J (x) after recovering defogging,
Step 5: building road sign identifies prediction policy
1. infrared camera I 3 acquires realtime graphic；
This system requires to be the process handled in real time in vehicle travel process, therefore the system is imaged using infrared (IR) Machine obtains data, for the requirement such as generality, real-time of the invention patent, the complexity of road environment, it is contemplated that shooting Ambient light is dim fuzzy, and with ground is not parallel, road sign has slight distortion in the picture, using structural red The video camera of outer optical projectors, so that it may obtain infrared light only to get the image of high-quality is arrived.It, will in vehicular motion The image of acquisition is input to Misty Image classifier and image defogging model, if it is determined that foggy image, then to foggy image Image defogging is carried out, if it is determined that fog free images, then carry out in next step；
2. by after defogging image or real-time fog free images be input to sign board character recognition system, by the road of acquisition Traffic mark board image is transferred to sign board character recognition module 9, sign board image is carried out figure in the module as input After the pretreatment of picture segmentation, utilizes and have been off the identification that trained neuroid classifier carries out character；After identification Road signs information, input warning module 11 issue the audible alert in relation to sign board information simultaneously by vehicle-mounted loudspeaker 14 The flag information of warning is shown on vehicle-carrying display screen 12；
3. infrared camera II 15 acquires driver's sight blinkpunkt, fixation time, frequency of wink and PERCLOS and is input to Fatigue pattern classifier, driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS within the scope of given threshold value, Then it is not applied to driver's warning information；Driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS are wherein any one A parameter value is not within the scope of given threshold value, then warning module 11 makes a sound warning by vehicle-mounted loudspeaker 14 and shows vehicle-mounted Show the screen display road signs information.
N >=3000 in the step 1, N1>=1000, N2≥2000。
L >=1000 in the step 1, K >=2000.
M in the step 21>=50, M2≥50。
X >=3000 in the step 3, X1>=1000, X2≥2000。
Y >=3000 in the step 3, Y1>=1000, Y2≥2000。
A specific embodiment of this method is given below:
A kind of method for early warning for missing road sign based on machine vision travelling in fog day, it is characterised in that:
Include the following steps
Step 1: establishing road and traffic sign plates recognition classifier
A, road and traffic sign plates gauss hybrid models GMM classifier is established
1. infrared camera I 3, which acquires 3000, includes the infrared road image of sign board and the infrared road image of no marks board, And acquired image is transferred to image classification module 6, including 1000 comprising the infrared road image of sign board and 2000 infrared road images of no marks board establish the image instruction of road and traffic sign plates classifier in image classification module 6 Practice library；
2. carrying out binaryzation, slant correction and contracting to the image in the training set of images of road and traffic sign plates classifier Size normalized is put, the image training library of the road and traffic sign plates classifier of uniform sizes is obtained；
3. using the image training library off-line training gauss hybrid models GMM classifier of road and traffic sign plates classifier, Gauss hybrid models GMM classifier is special according to the textural characteristics, sign board area features, profile of road sign sheet metal It is three-dimensional special that sign, usually regular rectangular shape, circle or triangle and road sign histogram feature establish gauss hybrid models Space is levied, training includes the infrared road image of sign board and the infrared road image of no marks board, the characteristics of image of sign board is obtained, Road and traffic sign plates classifier, which is established, to be completed；
B, the neuroid classifier for the identification of road and traffic sign plates character classification is established
1. infrared camera acquires 1000 different road and traffic sign plates images, to the road and traffic sign plates figure of acquisition Character therein is split as carrying out histogram equalization processing, threshold filter, searching profile algorithm, to the word of segmentation Size normalized is accorded with, uniformly reduces it having a size of 32*16 pixel；
2. designing four neural networks: Chinese character network, alphabetical network, number according to the difference of the information category on sign board Word network, graphical symbol network realize the classification to sign board information.Using complete in the GB2312-80 in Chinese character base CCLIB 6763, portion Chinese character and 571 characters construct Chinese character, letter, the digital dot array character repertoire of sign board in this system.According to statistics, Road signs wide variety includes 46 kinds of caution sign, and 43 kinds of prohibitory sign, 30 kinds of Warning Mark, 57 kinds of road sign, It 40 kinds of fingerpost, 39 kinds of road traffic marking, 16 kinds of auxiliary sign, travels 17 kinds of distinctive emblem, road construction safety sign 26 Kind.Wherein logotype or the sign board comprising logotype account for 90% or more, therefore above-mentioned Chinese character, letter, digital dot array Character repertoire can not meet actual graphical landmark identification demand, in the invention patent, will establish in addition to Chinese character, letter, number Logotype sample database.2000 sign board images are acquired, logotype therein, the spy of analysis mark board figure are extracted Point: the black figure in warning logotype board Huang bottom, the black figure red diagonal of ban logotype white background, the white figure in Warning Mark indigo plant bottom extract it Graphical symbol constructs in dot chart deposit figure sample database.Chinese character, letter, digital dot array character repertoire and figure sample database are merged Library is identified for sign board information, and the sign board of graphic character combination can identify library identification tag information according to sign board information.
3. constructing Back propagation neural metanetwork classifier structure
A) input layer number, to the character for being normalized to 32*16 dot matrix size, with each pixel for a net Lattice, input layer number take 512；
B) output layer neuron number is respectively as follows: Chinese character 1000 according to heterogeneous networks output layer neuron number；Letter 26；Number 10；Graphical symbol is 500；
C) hidden layers numbers are one layer, have theoretical research it is found that there is the neural network of input layer output layer and single hidden layer enough Execute arbitrarily complicated Function Mapping, One hidden layer neuron number calculation formula are as follows:
In formula, h_num indicates hidden neuron number, and i_num indicates input layer number, and o_num indicates output Layer neuron number.
D) activation primitive selects Logistic logistic functional form as follows:
In formula, vjFor the local field of neuron j,For the output function of hidden layer, e refers to being approximately equal to 2.71828 Natural logrithm bottom.
4. sign board information is identified Chinese character, letter, number and the figure four in library by artificial neural network's training process Kind sample is separately input in corresponding neuroid, and carries out Studying factors, factor of momentum, error to four networks respectively The setting of target value and threshold range, then carry out artificial neural network recognition training, input layer to hidden layer weight or To output layer weight, wherein any one weight then shows artificial neural network's training not in the threshold range of setting to hidden layer Failure, exits training；Input layer in the threshold range of setting, then saves power to hidden layer weight and hidden layer to output layer weight Value, artificial neural network train successfully；
5. sign board character recognition
The weight matrix that training obtains is input in the weight matrix of corresponding neuroid structure, by word to be identified Symbol sample is saved in file in the matrix form, the classification of the character sample is identified using neuroid after training, and defeated Result out；
6. artificial neural network tests, the accuracy rate of identification is calculated,
Manual record exports result 100 times, and the sign board information after Real-Time Character Recognition is really believed with real sign board Breath compares, the recognition result last to the algorithm of the Classification and Identification, carries out accuracy rate evaluation, and neuron net after being trained The recognition accuracy of network, if accuracy rate is lower than 95%, repeating step, 4. 5. artificial neural network is trained with step Sign board character recognition, to real-time grading identification information in there is the lower Chinese character of recognition accuracy, number, letter and figure Case is trained again according to recognition effect adhesion situation, increases the sample database of adhesion character until recognition accuracy is 95% Or 95% or more, then the neuroid classifier training with robustness is completed；
Step 2: establishing Fatigue pattern classifier
1. acquiring driving behavior data of 50 drivers under normal driving state, acquires 50 drivers and driven in fatigue Sail the driving behavior data under state；
2. extracting fatigue driving characteristic parameter using feature extracting method, normal driving is obtained using the method for statistical analysis With the characteristic parameter of fatigue driving, determine that driver's sight track, frequency of wink and PERCLOS are fatigue driving characteristic parameter；
3. infrared camera II 15 tracks driver's sight track, the blinkpunkt position of human eye and the friendship identified are driven Logical sign board position is inconsistent, then sends the signal that driver misses road signs；Drive the blinkpunkt position of human eye With the traffic mark board position consistency identified, then the time that driver watches the region attentively is recorded, the time is no more than 1 second, then sends out The signal for sending driver to miss road signs；Time is more than 1 second, then carries out in next step；
It is frequency of wink and wink time that 4. infrared camera II 15, which records driver, set frequency of wink threshold range as 2 seconds/time~4 seconds/time, each wink time threshold range is 0.25 second~0.3 second, and the driver of record is frequency of wink, blinks Wherein any one parameter value then sends the letter that driver misses road signs not in given threshold range between at the moment Number；The driver of record is frequency of wink and wink time in given threshold range, then carries out in next step；
5. according to measurement fatigue/drowsiness physical quantity PERCLOS,
The eyelid size under 100 groups of normal driving states is taken to obtain driver's left eyelid average-size L driverELMWith Driver's right eyelid average-size RELM,
Eyes closed degree is more than 80% frequency n in unit timepCalculation formula be
In formula, LELJFor the left eyelid size of jth frame image, RELJFor the right eyelid size of jth frame image；Unit time Interior eyes open degree less than 20% shared by ratio P80Evaluation criterion P as PERCLOS80Calculation formula be
In formula, f0For sample frequency, TP80For computation window size；
The calculation formula of measurement fatigue/drowsiness physical quantity PERCLOS is as follows:
Eyes closed is set to 80% and 80% or more shared time threshold as 30 seconds/point, infrared camera II (15) Detect that eyes closed is more than given threshold value, then sends driver to 80% and 80% or more shared time in the unit time Miss the signal of road signs；
6. establishing characteristic ginseng value under the conditions of fatigue driving according to the driving behavior data statistics under normal and fatigue state Database, utilize machine learning method construct Fatigue pattern classifier；
Step 3: being built with fog free images classifier
A, road image support vector machines classifier is established
1. infrared camera acquires 3000 infrared road images and infrared non-rice habitats image, and acquired image is passed It is defeated to there is fogless image classification module, including 1000 infrared road images and 2000 infrared non-rice habitats image, having The image training library of road image classifier is established in fog free images categorization module；
2. by the image training library off-line training support vector machines classifier of road image classifier, supporting vector Machine SVM classifier obtains road image feature according to road texture, to road image and non-rice habitats image classification, and obtains infrared Road image feature, road image support vector machines classifier, which is established, to be completed；
B, establishing has fogless Image Classifier
1. infrared camera acquires the infrared foggy image and infrared fog free images of 3000 different mistiness degree, including 1000 have mist infrared image and 2000 fogless infrared images, and establishing in having fogless image classification module has fog free images point The image training library of class device；
2. by having the image of fogless Image Classifier training library off-line training gauss hybrid models GMM classifier, Gauss Mixed model GMM classifier extracts the one-dimensional gray scale of foggy image and fog free images according to foggy image gray value frequecy characteristic Histogram image feature, and classified according to the histogram of image to foggy image and fog free images, there is fog free images Gauss Mixed model GMM classifier, which is established, to be completed；
3. by have the image of fogless Image Classifier training library off-line training support vector machines classifier, support to Amount machine SVM classifier is according to the Fourier transformation frequecy characteristic of foggy image and fog free images, to foggy image and fog free images Classify, and extract the Fourier transformation frequecy characteristic of foggy image and fog free images, there are fog free images support vector machines Classifier, which is established, to be completed；
By the foggy image for thering is fog free images gauss hybrid models GMM classifier to sort out and there is fog free images supporting vector The foggy image that machine SVM classifier sorts out takes union, obtains foggy image sample database；
Step 4: establishing image defogging model
1. atmospheric optics model is I (x)=J (x) e-βd+A(1-e-βd), I (x) is sorted foggy image, and J (x) is Clear image after defogging, A are global atmosphere light ingredient, e-βdIt is atmospheric extinction coefficient for atmospheric transmissivity value t, β, d is energy See angle value；
2. smallest passage gray level image of the mist road image in RGB RGB triple channel image is taken, then again to obtaining The gray level image taken does mini-value filtering, obtains foggy image dark:Wherein, JdarkRefer to J Dark, JCIndicate each channel of color image, C is RGB triple channel；Ω is the whole image window comprising all pixels Mouthful,For the minimum value pixel in whole image window all pixels,It is logical for each pixel RGB of whole image window tri- The minimum pixel value of road component；
3. preceding 0.1% pixel is taken according to brightness size from dark channel image, then it is original have found in mist figure pair The value for the point with maximum brightness answered, the signal component value A as atmosphere light；
4. handling atmospheric optics model, form is as follows:
I (x)=J (x) e-βd+A(1-e-βd)
Wherein C is RGB triple channel, asks dark to ask minimum operation twice, then benefit to above formula both sides above formula both ends With dark gray value close to zero, therefore, can derive:
Wherein t is atmospheric transmissivity value, since the presence of mist makes one to feel the presence of the depth of field,The middle factor ω introduced between one [0,1], obtains atmospheric transmissivity figure,
5. there is loss at the edge and grain details that obtain transmittance figure, side is carried out to transmittance figure using median filter The filter optimization that edge is kept obtains refinement transmittance figure；
6. threshold value t is arranged0, when t value is less than t0, t=t0 is enabled, with t0=0.1 is standard, will treated triple channel figure As synthesis, the clear image J (x) after recovering defogging,
Step 5: building road sign identifies prediction policy
1. infrared camera I 3 acquires distance in 100 meters~250 meters remote sign image information in real time；
This system requires to be the process handled in real time in vehicle travel process, therefore the system is imaged using infrared (IR) Machine obtains data, for the requirement such as generality, real-time of the invention patent, the complexity of road environment, it is contemplated that shooting Ambient light is dim fuzzy, and with ground is not parallel, road sign has slight distortion in the picture, using structural red The video camera of outer optical projectors, so that it may obtain infrared light only to get the image of high-quality is arrived.It, will in vehicular motion The image of acquisition is input to Misty Image classifier, if it is determined that foggy image, is input to image defogging model to there is mist figure As carrying out image defogging, if it is determined that fog free images, then carry out in next step；
2. by after defogging image or real-time fog free images be input to sign board character recognition system, using having trained Road and traffic sign plates gauss hybrid models GMM classifier classification, using the road and traffic sign plates image of acquisition as input, It is transferred to sign board character recognition module 9, sign board image is carried out to the pretreatment of image segmentation: color image in the module Gray processing, the denoising of 5*5 Gaussian Blur, Hough transform segmentation, Oust thresholded image, morphology opening and closing operation, connected component point The operations such as analysis, illumination histogram equalization obtain that treated road and traffic sign plates region, and the region is positioned, it will obtain The road and traffic sign plates region obtained carries out character as input, using trained neuroid classifier is had been off Identification；Road signs information after identification, input warning module 11 issue related sign board letter by vehicle-mounted loudspeaker 14 The audible alert of breath and the flag information alerted in vehicle-carrying display screen simultaneous display；
3. infrared camera II 15 acquires driver's sight blinkpunkt, fixation time, frequency of wink and PERCLOS and is input to Fatigue pattern classifier, driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS within the scope of given threshold value, Then it is not applied to driver's warning information；Driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS are wherein any one A parameter value is not within the scope of given threshold value, then warning module 11 makes a sound warning by vehicle-mounted loudspeaker 14 and shows vehicle-mounted Show the screen display road signs information.
In the above specific implementation example, the number for acquiring different road images is 3000, including 1000 containing not With the road image of road sign and 2000 any non-rice habitats images (including building, meadow, sky etc.), but the present invention couple The range of road image acquisition number is not limited to the present embodiment, and is based on common knowledge, Primary Stage Data collection capacity is bigger, later data The accuracy of processing is higher, therefore end value is only provided in the present embodiment, i.e. the example of minimum value；Similarly, in this specific implementation example The number for acquiring different driver's status images is 100, including the driving behavior status image of 50 fatigue driving people With the driving behavior status image of 50 normal driving people, of road image under different weather is acquired in this specific implementation example Number is 3000, has mist road image and 2000 fogless road images including 1000.It is adopted in this specific implementation example The number for collecting different mist figures is 3000, including 1000 foggy images and 2000 fog free images.This specific implementation model The number L that different road signs are acquired in example is 1000.Of different road signs is acquired in this specific implementation example Number K is 2000.End value, the i.e. example of minimum value are also only provided in a particular embodiment.

Claims (7)

1. a kind of early warning system for missing road sign based on machine vision travelling in fog day, it is characterised in that: including power supply (1), transformation plug I (2), infrared camera I (3), transformation plug II (4), onboard electronic control unit module (5), image classification mould Block (6), image defogging module (7), sign board identification module (8), sign board character recognition module (9), driving behavior classification mould Block (10), warning module (11), vehicle-carrying display screen (12), hoot device (13), vehicle-mounted loudspeaker (14) and infrared photography First II (15), the power supply (1) are connect by transformation plug I (2) with infrared camera I (3), and power supply (1) passes through transformation plug II (4) are connect with infrared camera II (15)；
The onboard electronic control unit module (5) includes image classification module (6), image defogging module (7), sign board identification module (8), sign board character recognition module (9), driving behavior categorization module (10), warning module (11), described image categorization module (6) one end is connect by conducting wire with infrared camera I (3), and the other end of image classification module (6) is gone by conducting wire and image Mist module (7) connection；Described image defogging module (7) is connect by conducting wire with sign board identification module (8)；The sign board is known Other module (8) is connect by conducting wire with sign board character recognition module (9)；The sign board character recognition module (9) is by leading Line is connect with warning module (11)；One end of the driving behavior categorization module (10) passes through conducting wire and infrared camera II (15) The other end of connection, driving behavior categorization module (10) is connect by conducting wire with warning module (11)；
The vehicle-carrying display screen (12) is connect by conducting wire with warning module (11)；One end of the hoot device (13) is logical It crosses conducting wire to connect with warning module (11), the other end of hoot device (13) is connected by conducting wire and vehicle-mounted loudspeaker (14) It connects.
2. the method for early warning of road sign is missed based on machine vision travelling in fog day, using early warning as described in claim 1 System, it is characterised in that:
Include the following steps
Step 1: establishing road and traffic sign plates identifying system
A, road and traffic sign plates gauss hybrid models GMM classifier is established
1. infrared camera acquisition N has the infrared road image of road and traffic sign plates and without the infrared road of road and traffic sign plates Image, and acquired image is transferred to sign board identification module (8), including N1Opening has road and traffic sign plates infrared Road image and N2It opens without the infrared road image of road and traffic sign plates, N, N1、N2It is natural number, in sign board identification module (8) training set of images of road and traffic sign plates classifier is established in；
2. carrying out binaryzation, slant correction and pantograph to the image in the training set of images of road and traffic sign plates classifier Very little normalized obtains the image training library of the road and traffic sign plates classifier of uniform sizes；
3. using the image training library off-line training gauss hybrid models GMM classifier of road and traffic sign plates classifier, Gauss Mixed model GMM classifier according to the textural characteristics, area features, profile shape characteristic of road and traffic sign plates metal material with And road and traffic sign plates histogram feature, according to whether there is or not infrared road image of the road and traffic sign plates to acquisition to divide Class obtains the characteristics of image for having road and traffic sign plates, and gauss hybrid models GMM classifier, which is established, to be completed；
B, the neuroid classifier for the identification of road and traffic sign plates character classification is established
1. infrared camera acquires L road and traffic sign plates images, L is natural number, to the road and traffic sign plates figure of acquisition Character therein is split as carrying out histogram equalization processing, threshold filter, searching profile algorithm, to the word of segmentation Size normalized is accorded with, uniformly reduces its size to 32*16 pixel；
2. establishing Chinese character network, alphabetical network, digital network, graphical symbol network to traffic mark according to the information on sign board Will board information is classified；
Using all 6763 Chinese characters and 571 characters of the GB2312-80 in Chinese character base CCLIB, traffic mark board is established Chinese character, letter and digital dot array character repertoire；
Figure sample database is established, K traffic mark board images of acquisition, K is natural number, logotype therein is extracted, according to friendship The graphic characteristics of logical sign board, extract its graphical symbol and establish graphical symbol dot chart and be stored in figure sample database；
Chinese character, letter and digital dot array character repertoire and figure sample database are merged into sign board information identification library；
3. constructing Back propagation neural metanetwork classifier structure
A) input layer number
To the character for being normalized to 32*16 dot matrix size, with each pixel for a grid, input layer number is taken 512；
B) output layer neuron number
Being respectively as follows: Chinese character according to heterogeneous networks output layer neuron number is 1000；Letter 26；Number 10；Graphical symbol is 500；
C) hidden layers numbers are one layer, One hidden layer neuron number calculation formula are as follows:
In formula, h_num is hidden neuron number, and i_num is input layer number, and o_num is output layer neuron Number；
D) activation primitive selects logistic functional form as follows:
In formula, vjFor the local field of neuron j,For the output function of hidden layer；
4. artificial neural network's training
Sign board information is identified that four kinds of Chinese character, letter, number and figure samples in library are separately input to corresponding neuron In network, and respectively four networks are carried out with the setting of Studying factors, factor of momentum, error target value and threshold range, so The recognition training of progress artificial neural network afterwards, input layer to hidden layer weight or hidden layer are to output layer weight not in the threshold of setting It is worth in range, then shows artificial neural network's failure to train, exit training；Input layer is to hidden layer weight and hidden layer to output layer Weight then saves weight in the threshold range of setting, and artificial neural network trains successfully；
5. sign board character recognition
The weight matrix that training obtains is input in the weight matrix of corresponding neuroid structure, by character sample to be identified This is saved in file in the matrix form, the classification of the character sample is identified using neuroid after training, and export knot Fruit；
6. artificial neural network tests, the accuracy rate of identification is calculated,
Manual record exports result for 100 times or more, and after being trained neuroid recognition accuracy, if accuracy rate is low In 95%, then repeat step 4. artificial neural network's training and step 5. sign board character recognition, if accuracy rate is 95% or 95% or more, then the neuroid classifier training with robustness is completed；
Step 2: establishing Fatigue pattern classifier
1. acquiring driving behavior data of the M1 drivers under normal driving state, M2 drivers are acquired in fatigue driving shape Driving behavior data under state, wherein M1, M2 are natural number；
2. extracting fatigue driving characteristic parameter using feature extracting method, normal driving and tired is obtained using the method for statistical analysis Please the characteristic parameter sailed determines that driver's sight track, frequency of wink and PERCLOS are fatigue driving characteristic parameter；
3. infrared camera II (15) tracks driver's sight track, the blinkpunkt position of human eye and the traffic identified are driven Sign board position is inconsistent, then sends the signal that driver misses road signs；Drive human eye blinkpunkt position with The traffic mark board position consistency identified then records the time that driver watches attentively, and the time is no more than 1 second, then sends driver Miss the signal of road signs；Time is more than 1 second, then carries out in next step；
4. infrared camera II (15) records the frequency of wink and wink time of driver, frequency of wink threshold range is set as 2 Second/time~4 seconds/time, each wink time threshold range is 0.25 second~0.3 second, and the driver of record is frequency of wink or blinks Between at the moment the signal that driver misses road signs is not sent then in given threshold range；The driver of record is to blink Eye frequency and wink time then carry out in next step in given threshold range；
5. according to measurement fatigue/drowsiness physical quantity PERCLOS,
The eyelid size under 100 groups of normal driving states is taken to obtain driver's left eyelid average-size L driverELMAnd driving People's right eyelid average-size RELM,
Eyes closed degree is more than 80% frequency n in unit timepCalculation formula be
In formula, LELJFor the left eyelid size of jth frame image, RELJFor the right eyelid size of jth frame image；
In unit time eyes open degree less than 20% shared by ratio P80Evaluation criterion as PERCLOS
P80Calculation formula be
In formula, f0For sample frequency, TP80For computation window size；
The calculation formula of measurement fatigue/drowsiness physical quantity PERCLOS is as follows:
Eyes closed is set to 80% and 80% or more shared time threshold as 30 seconds/point, infrared camera II (15) detects Eyes closed is more than given threshold value, then sends driver and miss to 80% and 80% or more shared time in the unit time The signal of road signs；
6. establishing the number of characteristic ginseng value under the conditions of fatigue driving according to the driving behavior data statistics under normal and fatigue state According to library, Fatigue pattern classifier is constructed using the method for machine learning；
Step 3: being built with fog free images classifier
A, road image support vector machines classifier is established
1. infrared camera acquires X infrared road images and infrared non-rice habitats images, and acquired image has been transferred to Fog free images categorization module, including X1Open infrared road image and X2Open infrared non-rice habitats image, X, X1、X2It is nature Number establishes the image training library of road image classifier in having fogless image classification module；
2. by the image training library off-line training support vector machines classifier of road image classifier, support vector machines Classifier obtains road image feature according to road texture, to road image and non-rice habitats image classification, and obtains infrared road Characteristics of image, road image support vector machines classifier, which is established, to be completed；
B, establishing has fogless Image Classifier
1. infrared camera acquires the infrared foggy image and infrared fog free images of Y different mistiness degree, including Y1Opening has mist Infrared image and Y2Open fogless infrared image, Y, Y1、Y2It is natural number, foundation has fogless in having fogless image classification module The image training library of Image Classifier；
2. by having the image of fogless Image Classifier training library off-line training gauss hybrid models GMM classifier, Gaussian Mixture Model GM M classifier extracts the one-dimensional intensity histogram of foggy image and fog free images according to foggy image gray value frequecy characteristic Figure characteristics of image, and classified according to the histogram of image to foggy image and fog free images, there is fog free images Gaussian Mixture Model GM M classifier, which is established, to be completed；
3. by having the image of fogless Image Classifier training library off-line training support vector machines classifier, support vector machines SVM classifier carries out foggy image and fog free images according to the Fourier transformation frequecy characteristic of foggy image and fog free images Classification, and the Fourier transformation frequecy characteristic of foggy image and fog free images is extracted, there is the classification of fog free images support vector machines Device, which is established, to be completed；
By the foggy image for thering is fog free images gauss hybrid models GMM classifier to sort out and there is fog free images support vector machines The foggy image that SVM classifier sorts out takes union, obtains foggy image sample database；
Step 4: establishing image defogging model
1. atmospheric optics model is I (x)=J (x) e-βd+A(1-e-βd), I (x) is sorted foggy image, and J (x) is defogging Clear image later, A are global atmosphere light ingredient, e-βdIt is atmospheric extinction coefficient for atmospheric transmissivity value t, β, d is visibility Value；
2. smallest passage gray level image of the mist road image in RGB RGB triple channel image is taken, then again to acquisition Gray level image does mini-value filtering, obtains foggy image dark:Wherein, JdarkRefer to that J's is dark Channel, JCIndicate each channel of color image, C is RGB triple channel；Ω is the whole image window for having all pixels,For Minimum value pixel in whole image window all pixels,For whole image window tri- channel components of each pixel RGB Minimum pixel value；
3. preceding 0.1% pixel is taken according to brightness size from dark channel image, then it is original have found in mist figure it is corresponding The value of point with maximum brightness, the signal component value A as atmosphere light；
4. handling atmospheric optics model, form is as follows:
I (x)=J (x) e-βd+A(1-e-βd)
Wherein C is RGB triple channel, asks dark to seek minimum operation twice to above formula both sides above formula both ends, recycles dark Channel gray value is close to zero, therefore, can derive:
Wherein t is atmospheric transmissivity value, since the presence of mist makes one to feel the presence of the depth of field, The middle factor ω introduced between one [0,1], obtains atmospheric transmissivity figure,
5. there is loss at the edge and grain details that obtain transmittance figure, edge guarantor is carried out to transmittance figure using median filter The filter optimization held obtains refinement transmittance figure；
6. threshold value t is arranged0, when t value is less than t0When, enable t=t0, with t0=0.1 is standard, and by treated, triple channel image is closed At, clear image J (x) after recovering defogging,
Step 5: building road sign identifies prediction policy
1. infrared camera I (3) acquires realtime graphic
In vehicular motion, infrared camera I (3) acquires realtime graphic, and acquired image has been sequentially inputted to Fog free images classifier and image defogging model carry out image defogging to foggy image；
2. by after defogging image or real-time fog free images be input to road and traffic sign plates identifying system, by the road of acquisition Traffic mark board image is transferred to sign board character recognition module (9), in sign board character recognition module (9) as input By road and traffic sign plates image carry out image segmentation pretreatment after, using off-line training complete neuroid classifier into Line character identification；Road signs information after identification inputs warning module (11) and is issued by vehicle-mounted loudspeaker (14) Audible alert simultaneously shows the road signs information on vehicle-carrying display screen (12)；
3. infrared camera II (15) acquisition driver's sight blinkpunkt, fixation time, frequency of wink and PERCLOS are input to tired Labor pattern classifier, driver's sight blinkpunkt, fixation time, frequency of wink, PERCLOS are within the scope of given threshold value, then It is not applied to driver's warning information；Driver's sight blinkpunkt, fixation time, frequency of wink or PERCLOS be not in given threshold It is worth in range, then warning module (11) is made a sound by vehicle-mounted loudspeaker (14) and alerted and in the car-mounted display screen display road Road road signs information.
3. the method for early warning according to claim 2 for missing road sign based on machine vision travelling in fog day, feature It is: N >=3000 in the step 1, N1>=1000, N2≥2000。
4. the method for early warning according to claim 2 for missing road sign based on machine vision travelling in fog day, feature It is: L >=1000 in the step 1, K >=2000.
5. the method for early warning according to claim 2 for missing road sign based on machine vision travelling in fog day, feature It is: the M in the step 21>=50, M2≥50。
6. the method for early warning according to claim 2 for missing road sign based on machine vision travelling in fog day, feature It is: X >=3000 in the step 3, X1>=1000, X2≥2000。
7. the method for early warning according to claim 2 for missing road sign based on machine vision travelling in fog day, feature It is: Y >=3000 in the step 3, Y1>=1000, Y2≥2000。
CN201610218031.3A 2016-04-11 2016-04-11 The early warning system and method for road sign are missed based on machine vision travelling in fog day Expired - Fee Related CN105844257B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610218031.3A CN105844257B (en) 2016-04-11 2016-04-11 The early warning system and method for road sign are missed based on machine vision travelling in fog day

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610218031.3A CN105844257B (en) 2016-04-11 2016-04-11 The early warning system and method for road sign are missed based on machine vision travelling in fog day

Publications (2)

Publication Number Publication Date
CN105844257A CN105844257A (en) 2016-08-10
CN105844257B true CN105844257B (en) 2019-03-01

Family

ID=56597063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610218031.3A Expired - Fee Related CN105844257B (en) 2016-04-11 2016-04-11 The early warning system and method for road sign are missed based on machine vision travelling in fog day

Country Status (1)

Country Link
CN (1) CN105844257B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110852972A (en) * 2019-11-11 2020-02-28 苏州科技大学 Single image rain removing method based on convolutional neural network

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10528055B2 (en) * 2016-11-03 2020-01-07 Ford Global Technologies, Llc Road sign recognition
CN106548211A (en) * 2016-11-08 2017-03-29 中国科学院自动化研究所 Transformer station's point for measuring temperature automatic identifying method and device based on infrared image
WO2018132961A1 (en) * 2017-01-18 2018-07-26 Nokia Technologies Oy Apparatus, method and computer program product for object detection
CN106992011A (en) * 2017-01-25 2017-07-28 杭州电子科技大学 Engineering machinery sound identification method based on MF PLPCC features
CN107025796A (en) * 2017-04-28 2017-08-08 北京理工大学珠海学院 Automobile assistant driving vision early warning system and its method for early warning
CN107194396A (en) * 2017-05-08 2017-09-22 武汉大学 Method for early warning is recognized based on the specific architecture against regulations in land resources video monitoring system
JP6658674B2 (en) * 2017-06-08 2020-03-04 トヨタ自動車株式会社 Driving support system
CN107480214A (en) * 2017-07-27 2017-12-15 广东欧珀移动通信有限公司 Information displaying method, device, computer-readable recording medium and mobile terminal
CN107403010A (en) * 2017-07-27 2017-11-28 广东欧珀移动通信有限公司 Information displaying method, device, computer-readable recording medium and mobile terminal
CN109383216A (en) * 2017-08-10 2019-02-26 郑州宇通客车股份有限公司 A kind of vehicle windscreen hazes identifying system, method and vehicle glass
CN107545304A (en) * 2017-09-16 2018-01-05 胡明建 A kind of design method for changing activation primitive artificial neuron according to network demand
CN107844761B (en) * 2017-10-25 2021-08-10 海信集团有限公司 Traffic sign detection method and device
KR102333033B1 (en) * 2017-12-07 2021-12-01 삼성전자주식회사 Vehicle and control method thereof
CN108304066A (en) * 2018-01-24 2018-07-20 三峡大学 Security identifier visual communication effect quantization method
WO2020000253A1 (en) * 2018-06-27 2020-01-02 潍坊学院 Traffic sign recognizing method in rain and snow
JP7176398B2 (en) * 2018-12-21 2022-11-22 トヨタ自動車株式会社 CONTROL DEVICE, VEHICLE, IMAGE DISPLAY SYSTEM, AND IMAGE DISPLAY METHOD
CN110171360A (en) * 2019-06-17 2019-08-27 大陆汽车车身电子系统（芜湖）有限公司 A kind of alarm method for electronic device
CN112874527A (en) * 2019-11-29 2021-06-01 奥迪股份公司 Driving assistance device, vehicle comprising same, and corresponding method and medium
CN113126296B (en) * 2020-01-15 2023-04-07 未来（北京）黑科技有限公司 Head-up display equipment capable of improving light utilization rate
CN112329622A (en) * 2020-11-05 2021-02-05 山东交通学院 Night visibility detection and visibility grade division early warning method
CN112330558A (en) * 2020-11-05 2021-02-05 山东交通学院 Road image recovery early warning system and method based on foggy weather environment perception

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1263329A (en) * 2000-02-18 2000-08-16 李世杰 Traffic sign warning system and method for identifying direction in said system
CN103489010A (en) * 2013-09-25 2014-01-01 吉林大学 Fatigue driving detecting method based on driving behaviors
CN103578227A (en) * 2013-09-23 2014-02-12 吉林大学 Fatigue driving detection method based on GPS positioning information
CN104240239A (en) * 2014-09-01 2014-12-24 西南交通大学 Method for detecting local road segment hazy weather based on road image

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6739741B2 (en) * 2002-09-26 2004-05-25 Apt Co., Ltd. Display device by columnar light

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1263329A (en) * 2000-02-18 2000-08-16 李世杰 Traffic sign warning system and method for identifying direction in said system
CN103578227A (en) * 2013-09-23 2014-02-12 吉林大学 Fatigue driving detection method based on GPS positioning information
CN103489010A (en) * 2013-09-25 2014-01-01 吉林大学 Fatigue driving detecting method based on driving behaviors
CN104240239A (en) * 2014-09-01 2014-12-24 西南交通大学 Method for detecting local road segment hazy weather based on road image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于机器视觉的驾驶员嘴部状态检测方法;施树明，金立生;《吉林大学学报（工学版）》;20040430;第34卷(第2期);第232-236页

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110852972A (en) * 2019-11-11 2020-02-28 苏州科技大学 Single image rain removing method based on convolutional neural network

Also Published As

Publication number Publication date
CN105844257A (en) 2016-08-10

Similar Documents

Publication Publication Date Title
CN105844257B (en) The early warning system and method for road sign are missed based on machine vision travelling in fog day
CN110197589B (en) Deep learning-based red light violation detection method
CN105512623B (en) Based on multisensor travelling in fog day vision enhancement and visibility early warning system and method
CN110188807B (en) Tunnel pedestrian target detection method based on cascading super-resolution network and improved Faster R-CNN
CN110097109A (en) A kind of road environment obstacle detection system and method based on deep learning
CN111898523A (en) Remote sensing image special vehicle target detection method based on transfer learning
CN107633220A (en) A kind of vehicle front target identification method based on convolutional neural networks
CN101430195B (en) Method for computing electric power line ice-covering thickness by using video image processing technology
CN110226170A (en) A kind of traffic sign recognition method in rain and snow weather
CN107067002A (en) Road licence plate recognition method in a kind of dynamic video
CN103824081B (en) Method for detecting rapid robustness traffic signs on outdoor bad illumination condition
CN102509098B (en) Fisheye image vehicle identification method
CN104751600B (en) Anti-fatigue-driving safety means and its application method based on iris recognition
CN108734189A (en) Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather
CN106909937A (en) Traffic lights recognition methods, control method for vehicle, device and vehicle
CN104463241A (en) Vehicle type recognition method in intelligent transportation monitoring system
CN108491797A (en) A kind of vehicle image precise search method based on big data
CN104050450A (en) Vehicle license plate recognition method based on video
CN108918532A (en) A kind of through street traffic sign breakage detection system and its detection method
CN105844245A (en) Fake face detecting method and system for realizing same
CN106934374A (en) The recognition methods of traffic signboard and system in a kind of haze scene
CN106096504A (en) A kind of model recognizing method based on unmanned aerial vehicle onboard platform
CN108091141A (en) Vehicle License Plate Recognition System
CN106919939B (en) A kind of traffic signboard tracks and identifies method and system
CN107886034A (en) Driving based reminding method, device and vehicle

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20190301