CN105809121A - Multi-characteristic synergic traffic sign detection and identification method - Google Patents

Multi-characteristic synergic traffic sign detection and identification method Download PDF

Info

Publication number
CN105809121A
CN105809121A CN201610121846.XA CN201610121846A CN105809121A CN 105809121 A CN105809121 A CN 105809121A CN 201610121846 A CN201610121846 A CN 201610121846A CN 105809121 A CN105809121 A CN 105809121A
Authority
CN
China
Prior art keywords
traffic signs
color
traffic
local
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610121846.XA
Other languages
Chinese (zh)
Inventor
康波
蔡会祥
王琳
赵辉
李云霞
敬斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201610121846.XA priority Critical patent/CN105809121A/en
Publication of CN105809121A publication Critical patent/CN105809121A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a multi-characteristic synergic traffic sign detection and identification method which is performed according to the following steps. A color probability model is established for traffic signs with different colors through the images of traffic sign samples and representative colors are determined out of the traffic signs with different colors so as to obtain probability check lists for the representative colors, train and obtain shape classifying devices for traffic signs belonging to different categories and identifying models. For traffic images to be detected, each probability check list for the representative colors is used first to get the probability images of the traffic images, which are then converted to grey scale maps. An MSER algorithm is used to detect the areas in the grey scale maps which change stably and the areas are regarded as potential windows to be picked up that meet the pre-set height-width ratio. The shape classifying devices then determines whether the potential windows to be picked up contain traffic signs or not, and if they do, the identifying models will identify these corresponding shapes. The method can achieve a better detection and identification effect by combining the characteristics of colors and shapes of traffic signs.

Description

Road traffic sign detection that multiple features is collaborative and recognition methods
Technical field
The invention belongs to road traffic sign detection and identify technical field, more specifically saying, relating to the collaborative road traffic sign detection of a kind of multiple features and recognition methods.
Background technology
Along with the development of economy Yu technology, intelligent transport technology obtains development energetically, the detection of a traffic signs and identification important component part as intelligent transport technology, obtains and increasingly payes attention to widely.The detection of traffic signs is usually according to captured road image with identifying, first road image is carried out pretreatment, then detects traffic signs from road image, finally carries out Classification and Identification again.
The task of road traffic sign detection is to detect the position of traffic signs in the image of input, should have the feature of low loss, low false drop rate.Wherein low loss is the index of most critical, because road traffic sign detection is the basis of follow-up Traffic Sign Recognition, the former is the object latter providing identification.Once the process of detection misses certain mark, it will directly result in whole road traffic sign detection and miss this traffic signs with identification system.For the feature of traffic signs regular shape, color clear, scholars propose a lot based on shape, based on color, based on the detection method of template.From the result that the German road traffic sign detection contests of 2013 are announced, the method based on template has very big advantage in accuracy rate, to illumination, the multiple unfavorable conditions such as blocks and well adapts to ability.But the operand of this kind of method is general bigger, it is difficult to meet the requirement of real-time.
For the identification of traffic signs, it it is a multi-class classification problem, therefore a lot of algorithm for pattern recognitions are introduced into the identification of traffic signs, including template matching, sparse coding, SupportVectorMachine (SVM), DeepNeuralNetworks (DNN), Adaboost algorithm etc..CNN (ConvolutionalNeuralNetwork, convolutional neural networks) is relative to other machines learning algorithm, owing to it has the characteristic automatically extracting feature and by extensive concern.But ordinary straight line style CNN network is in Traffic Sign Recognition process, owing to the input of later layer is only relevant with the output of preceding layer, too small for those and broad image identification ability is also limited in.
Summary of the invention
It is an object of the invention to overcome the deficiencies in the prior art, the collaborative road traffic sign detection of a kind of multiple features and recognition methods are provided, the various features adopting traffic signs has worked in coordination with detection and the identification of traffic signs, reduce loss and the false drop rate of traffic signs, improve the recognition effect of traffic signs.
For achieving the above object, multiple features of the present invention is collaborative road traffic sign detection and recognition methods, it is characterised in that including:
S1: traffic signs is classified according to the color feature of traffic signs, each colour type obtains several traffic signs sample images respectively；For every traffic signs sample image, extract the color characteristic of each pixel, according to color characteristic, all pixels of this traffic signs sample image are clustered, number of clusters is N+1, N is the primary color quantity of traffic signs, the pixel of cluster corresponding in each sample image in every class traffic signs sample image is merged, obtains N+1 sample set of this colour type, each sample set is set up corresponding Gauss model；
S2: for the traffic signs of each colour type, calculate each R, G, B value according to the color probability model of the N+1 of its correspondence and belong to the Probability p (c of each colori| x), x represents pixel R, G, B value, ciExpression color, i=1,2 ..., N+1；From the N kind primary color of this colour type, choose a kind of color representatively color, be designated as ci', then the probability representing color is normalized and obtains its normalization probabilitySet up each R, G, B value and belong to the probability search table representing color；
S3: traffic signs is divided into M class according to shape, each shape classification sets up a Shape Classification device based on HOG feature, and its training method is: for each Shape Classification device, obtains two class sample images, one class is the traffic signs sample image of correspondingly-shaped, another kind of for other images；Unified samples picture size, extract the HOG feature of every sample image, using the input as Shape Classification device of the HOG feature, if for the decision content of affiliated shape classification as the output of Shape Classification device, training obtains the Shape Classification device of correspondingly-shaped classification traffic signs；
S4: according to the shape classification number M of traffic signs, the traffic signs of each shape classification being respectively provided with a model of cognition, each shape classification obtains several traffic signs sample images respectively；First all sample images are carried out pretreatment, strengthen including unified image size and contrast；Adopt each pretreated sample image of shape classification that its model of cognition is trained, then it is grouped randomly ordered for pretreated traffic signs sample image after training every time, Q kind distortion mode is set, to often organize sample image randomly choose in Q kind distortion mode q kind distortion mode according to random order, sample image is carried out distortion process, its model of cognition is trained by the new samples image after adopting distortion to process, until reaching training termination condition；
S5: travel through each pixel in traffic image to be detected, according to each probability search table representing color, calculates each pixel and belongs to the probability of such color, obtains traffic image to be detected at each probability graph represented under color, is then converted into gray-scale map；Utilizing MSER algorithm to detect the grey scale change stability region in each gray-scale map, remove depth-width ratio region beyond default depth-width ratio scope in stability region, remaining stability region is as candidate window；
S6: candidate window size adjusting to Shape Classification device is inputted size, extract the HOG feature of each candidate window correspondence image block, input each traffic signs Shape Classification device, judge that whether this candidate window is the shape classification of traffic signs, if, then illustrating that this candidate window exists traffic signs, the shape of traffic signs is the shape that correspondingly-shaped grader is carried out judging, is otherwise absent from traffic signs；
S7: be the candidate window that there is traffic signs for step S6 judged result, extract correspondence image, adjust the input image size to model of cognition, and according to the traffic signs shape that step S6 judges, Traffic Sign Images after size adjusting is inputted the model of cognition of correspondingly-shaped, is identified result.
nullRoad traffic sign detection that multiple features of the present invention is collaborative and recognition methods，First pass through traffic signs sample image and set up the color probability model of each colour type traffic signs，Select the representative color of each colour type traffic signs，Calculate and obtain each probability search table representing color，Training simultaneously obtains Shape Classification device and the model of cognition of each shape classification traffic signs，For traffic image to be detected，Each probability search table representing color is first used to obtain the probability graph of traffic image to be detected，It is then converted into gray-scale map，MSER algorithm is utilized to detect the grey scale change stability region in each gray-scale map，Using the region that meets default depth-width ratio scope as candidate window，Each candidate window adopt each traffic signs Shape Classification device determine whether there is traffic signs，For there is the candidate window of traffic signs，The model of cognition adopting correspondingly-shaped carries out Traffic Sign Recognition.
The present invention, in the detection and identification process of traffic signs, have employed the combination of two kinds of features of CF feature and worked in coordination with detection, thus improve detection efficiency, improve the recognition effect of traffic signs.
Accompanying drawing explanation
Fig. 1 is the detailed description of the invention flow chart of the collaborative road traffic sign detection of multiple features of the present invention and recognition methods；
Fig. 2 is the exemplary plot of prohibitory sign；
Fig. 3 is traffic signs sample image cluster result exemplary plot；
Fig. 4 is distortion effect exemplary plot；
Fig. 5 is convolutional neural networks structure chart；
Fig. 6 is convolution kernel and the convolution results of first volume lamination；
Fig. 7 is convolution kernel and the convolution results of volume Two lamination；
Fig. 8 is traffic image to be detected；
Fig. 9 is the probability graph that traffic image to be detected obtains according to red probability search table；
Figure 10 is the gray-scale map that probability graph shown in Fig. 9 is corresponding；
Figure 11 is that MESE algorithm detects the grey scale change stability region result figure obtained；
Figure 12 is the candidate window figure of traffic image to be detected；
Figure 13 is the road traffic sign detection result figure of traffic image to be detected；
Figure 14 is the false recognition rate comparison diagram of linear type convolutional neural networks (network 1) and multiple row type convolutional neural networks (network 2).
Detailed description of the invention
Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described, in order to those skilled in the art is more fully understood that the present invention.Requiring particular attention is that, in the following description, when perhaps the detailed description of known function and design can desalinate the main contents of the present invention, these descriptions here will be left in the basket.
Embodiment
Fig. 1 is the detailed description of the invention flow chart of the collaborative road traffic sign detection of multiple features of the present invention and recognition methods.As it is shown in figure 1, the road traffic sign detection that multiple features of the present invention is worked in coordination with comprises the following steps with recognition methods:
S101: set up color probability model:
Traffic signs is classified by the color feature according to traffic signs, and each class obtains several traffic signs sample images respectively.For every traffic signs sample image, extract the color characteristic of each pixel, according to color characteristic, all pixels of this traffic signs sample image are clustered, number of clusters is N+1, N is the primary color quantity of traffic signs, that is each primary color respectively class of traffic signs, a remaining class is background colour, and clustering method can select as required.The pixel of cluster corresponding in each sample image in every class traffic signs sample image is merged, obtains N+1 sample set of this classification, each sample set is set up corresponding Gauss model, and calculates the prior probability of each sample set under each colour type.
For Traffic Sign Images, its color be mainly made up of color in N+1 (front N kind be constitute traffic signs primary color, N+1 kind is background), such as except after background colour, the primary color of caution sign is yellow and black, the primary color of prohibitory sign is red, black and white, and the primary color of Warning Mark is blue and white.Fig. 2 is the exemplary plot of prohibitory sign.Prohibitory sign in Fig. 2 is speed(-)limit sign.Therefore the color feature of traffic signs is still comparatively obvious.
When setting up color probability model, it is possible to select color characteristic as required.Owing to the traffic signs under natural scene is subject to the illumination effect of different condition (backlight, strong illumination etc.), in order to make color probability model more accurate, the present embodiment proposes a kind of employing Color invariants (colorinvariance) as color characteristic.
According to Geosebmek et al. Gauss light spectrum model set up, parameter (E, the E of each pixel value of coloured image (R, G, B) and Gauss color modelλ,Eλλ) relation be:
E E λ E λ λ = H R G B
Wherein H is the coefficient matrix of 3 × 3, is approximately
Then calculate and obtainCλAnd CλλIt is that color is weighed, unrelated towards, illumination direction with visual angle, intensity of illumination, surface.Therefore in the present embodiment, preferred color weighs CλAnd CλλColor characteristic as each pixel.
Under each colour type, the computing formula of the prior probability of each sample set is:
p ( c i ) = n c i n
Wherein, ciExpression i-th sample set, i=1,2 ..., N+1,Represent and belong to ciPixel number, n represents the number of all sampled pixel points under colour type.
Fig. 3 is traffic signs sample image cluster result exemplary plot.As it is shown on figure 3, this traffic signs sample image is prohibitory sign, its primary color has three kinds, and therefore making number of clusters is 4, adopts kmeans clustering method to cluster.Visible, the pixel in this traffic signs sample image has been divided into 4 classes well.All of red ban traffic signs sample is adopted and clusters in the same way, the pixel that correspondence clusters is merged, obtains the sample set of 4 kinds of colors.Then each sample set is set up Gauss model.By the analysis to sample set element, in order to improve the robustness of model, in the present embodiment, redness, black, background sample set being set up mixed Gauss model respectively, single Gauss model set up by dialogue colo(u)r atlas collection.
S102: calculate traffic signs color probability look-up table:
For the traffic signs of each colour type, calculate each R, G, B value according to the color probability model of the N+1 of its correspondence and belong to the Probability p (c of each colori| x), x represents pixel R, G, B value, ciExpression color, i=1,2 ..., N+1.Traffic signs due to each colour type, one can be selected and represent color, namely the color this colour type and another colour type can distinguished completely, the representative color of such as caution sign is yellow, the representative color of prohibitory sign is red, the representative color of Warning Mark is blue, therefore from its N kind primary color, choose a kind of color representatively color, then it is normalized and obtains its normalization probability representing color, set up each R, G, B value belongs to the probability search table representing color.
For pixel x=, (r, g, b), it belongs to color ciProbability be:
p ( c i | x ) = p ( x | c i ) p ( c i ) p ( x )
In a concrete picture, p (x) determines that, it is possible to think:
p(ci|x)≈p(x|ci)p(ci)
Again by color probability model obtained in the previous step, it is possible to directly obtain p (x | ci):
N ( x ; u c i j , Σ c i j ) = 1 2 π Σ c i j exp - 1 2 ( x - u c i j ) Σ c i j - 1 ( x - u c i j ) T
p ( x | c i ) = Σ j = 1 K λ k N ( x , u c i j , Σ c i j )
In formula,Represent color ciGauss model in the average of jth Gauss model,Represent color ciThe covariance of jth Gauss model in Gauss model, K represents color ciGauss model quantity.
Thus may determine that p (ci|x)。
p ( c i | x ) = p ( x | c i ) p ( c i ) = p ( c i ) Σ j = 1 K λ k N ( x , u c i j , Σ c i j )
It is c that note represents colori', the probability after its normalization is:
p ~ ( c i ′ | x ) = p ( c i ′ | x ) Σ i ′ = 1 N + 1 p ( c i ′ | x )
Thus, it is possible to calculate each pixel and belong to and represent color ci' probability.For accelerating detection speed, the present invention, according to color probability model, calculates the off-line probability search table of the representative color of each colour type.Representing color relation is the color that can this colour type and other colour types be distinguished.For red ban traffic signs, for its red obvious feature of aspect ratio, calculate red probability search table, containing 256^3 element, so for any one pixel, its probability belonging to redness can be obtained according to its R, G, B value table look at.
S103: training traffic signs Shape Classification device:
Traffic signs is divided into M class according to shape, each shape classification sets up one based on HOG ((HistogramofOrientedGradient, histograms of oriented gradients) the Shape Classification device of feature, its training method is: for each Shape Classification device, gather two class sample images, one class is the traffic signs sample image of correspondingly-shaped, another kind of for other images, including the sample image of the traffic signs sample image of other shapes or other non-traffic signss.Unified samples picture size, all of sample image is normalized to the size of 20*20 by the present embodiment, extract the HOG feature of every sample image, using the input as Shape Classification device of the HOG feature, whether is the decision content output as Shape Classification device of generic, training obtains the Shape Classification device of correspondingly-shaped classification traffic signs.Traffic signs is divided in the present embodiment circular and triangle two class according to shape, and the grader adopted is SVM (SupportVectorMachine, support vector machine) grader.
S104: training Traffic Sign Recognition model:
In order to complete the identification of traffic signs, it is necessary to the good model of cognition of training in advance.In order to improve the recognition accuracy of model of cognition, the shape classification number M of present invention traffic signs, the traffic signs of each shape classification is respectively provided with a model of cognition, each shape classification obtains several traffic signs sample images respectively.First all sample images are carried out pretreatment, strengthen including unified image size and contrast.Unified image size is in order to avoid the training effect of model of cognition is produced to affect by sample-size size.The present embodiment adopt restriction contrast self-adapting histogram equilibrium (CLAHE) algorithm carry out contrast enhancing.The contrast self-adapting histogram equilibrium (AHE) that restriction contrast self-adapting histogram equilibrium algorithm compares traditional strengthens image, will not excessively amplify noise, it is possible to reach ideal treatment effect in enhancing process.
In addition, in order to ensure the training effect of model of cognition and increase the robustness of identification, for each shape classification, then it is grouped randomly ordered for pretreated traffic signs sample image after training every time, Q kind distortion mode is set, randomly choosing q kind distortion mode in Q kind distortion mode, according to random order, sample image carries out distortion process to often organizing sample image, its model of cognition is trained by the new samples image after adopting distortion to process, until reaching training termination condition.Training termination condition is the requirement that the model of cognition pre-set to reach, and is usually output error, requires that namely training terminates if met.
In the present embodiment, distortion mode includes adding white noise, translation, affine transformation, rotation four kinds.Fig. 4 is distortion effect exemplary plot.When distortion, it is possible to randomly choose several distortion mode, distort with random order, for instance select translation and rotate two ways, first sample image being rotated, be then balanced again.Processed by distortion, it is possible to substantial increase sample size and type, it is possible to make finally to train the Traffic Sign Recognition model obtained more accurate, thus promoting Traffic Sign Recognition rate.
Existing model of cognition has multiple, adopts convolutional neural networks in the present invention.In order to improve the discrimination of traffic signs, the present embodiment redesigns a kind of convolutional neural networks.Fig. 5 is convolutional neural networks structure chart.As shown in Figure 5, the ground floor of the convolutional neural networks in the present embodiment is input layer, as the first stage--the input of low-level feature abstract, including first volume lamination, the first maximum pond layer and the first local normalization layer, extract followed by second stage high-level characteristic: volume Two lamination, the second maximum pond layer and the second local normalization layer；Again the output result of the second local normalization layer is input in the convolutional network of the first local convolutional layer and the second local convolutional layer composition；At full articulamentum, second local convolutional layer is not only trained Traffic Sign Recognition model as the input of full articulamentum by the present invention, also combine with the output (i.e. the first local convolutional layer) of more front layer, first local convolutional layer and the second local convolutional layer have the feature representation of different levels, so both is combined, it is simultaneously entered to full articulamentum, as the feature description that network is final；Finally, belong to the probability data of certain classification to obtain object to be identified, use SoftMax to return as output layer.
The work process of convolutional neural networks is as follows:
1) first volume lamination:
Input picture is carried out convolution by first volume lamination.In the present embodiment, the size of traffic signs sample image is all unified to be 32 × 32, and using 3 Channel Color Traffic Sign Images as input picture, the dimension of input training data is 3072 dimensions.First volume lamination adopts 64 different convolution kernels being sized to 5 × 5 to carry out convolution, obtains convolutional layer, and its output is characteristic pattern, exports to the first maximum pond layer.Fig. 6 is convolution kernel and the convolution results of first volume lamination.As shown in Figure 6, the characteristic pattern that in the present embodiment, first volume lamination obtains 64 altogether, it is sized to 28 × 28, the computing formula adopted in the present embodiment is:
y 2 r = f ( Σ y 1 * k i j r + b j r )
Wherein, y1For input traffic signs sample image,R for obtaining after convolution opens characteristic pattern,It is the r convolution kernel,For corresponding biasing,WithBeing the parameter needing training, f () is sigmoid activation primitive.
2) the first maximum pond layer
First maximum pond layer is to step 1) 64 characteristic patterns of gained carry out maximum pond.In the present embodiment, pond mode selects maximum pond, and in the present embodiment, pond core is sized to 2 × 2, and the pond layer obtained is sized to 14 × 14, and computing formula is as follows:
y 3 r = f ( β 2 r d o w n ( y 2 r ) + b 2 r )
Wherein,It is gained image behind pond,For the r characteristic pattern of gained after convolutional layer 1, down () is a down-sampling function,WithRespectively the property taken advantage of biasing and additivity biasing, be the parameter needing training.
3) the first local normalization layer
First local normalization layer is by 2) the pond layer that obtains carries out locally normalization.First local acknowledgement's normalization layer completes one " closing on suppression " operation, and local input area is normalized.Normalization formula is as follows:
y 4 = ( 1 + ( α / λ ) Σ x i 2 ) β
Wherein, y4For gained image after normalization, λ is local size size, and α is normalization zoom factor, and β is exponential term, and in the present embodiment, α and β selects 1 and 5 respectively.
4) second stage high-level characteristic extracts.
Volume Two lamination involved by second stage high level, the second local normalization layer, the second maximum pond layer formula as previously described, be no longer now simply that such as first volume lamination but carries out based on input picture on the basis that the first stage exports.Fig. 7 is convolution kernel and the convolution results of volume Two lamination.
5) the first local convolutional layer and the second local convolutional layer
First local convolutional layer reception tank layer carries out local convolution, and the local convolutional layer obtained is sent respectively to the second local convolutional layer and full articulamentum；Second convolutional layer reception local, local convolutional layer proceeds local convolution, exports the local convolutional layer obtained to meeting articulamentum.Feature is extracted further by the first local convolutional layer and the second local convolutional layer.
6) full articulamentum
In multiple row type convolutional neural networks, the input of full articulamentum had both included the output of the second local convolutional layer, also included the output of the first local convolutional layer.Full articulamentum merges the first local convolutional layer and the local convolutional layer of the second local convolutional layer, by fusion results output to Softmax layer.In this way, the information of full articulamentum input is multi-level, has both included this higher level abstract characteristics, also includes the feature input of lower level, it is thus achieved that better Traffic Sign Recognition effect.
6) Softmax layer.
Use convolution, the data obtained by sample mode, the feature description under a convolutional neural networks can be obtained by full articulamentum, then pass through the fusion results (i.e. the feature of network extraction) that full articulamentum exports by Softmax layer and adopt SoftMax to return to classify.The subclass numbers determined when output size is according to training determines, such as triangle model of cognition just has 15 kinds of types, then the maximum output of this layer is just 15.
Above four steps complete the preparation of road traffic sign detection and identification, next by entering actual detection and identification.
S105: traffic image pretreatment:
Travel through each pixel in traffic image to be detected, the probability search table of the representative color according to each colour type, calculate each pixel and belong to the probability of such color, obtain traffic image to be detected at each probability graph represented under color, be then converted into gray-scale map.It is to say, traffic image to be detected can obtain many secondary gray-scale maps, each to represent color corresponding secondary.Fig. 8 is traffic image to be detected.Fig. 9 is the probability graph that traffic image to be detected obtains according to red probability search table.Figure 10 is the gray-scale map that probability graph shown in Fig. 9 is corresponding.According to Fig. 8 to Figure 10 it can be seen that area probability red in figure is higher, its region corresponding in gray-scale map is brighter, and other area probability are relatively low, and its region corresponding in gray-scale map is dark.
Utilize the grey scale change stability region that MSER (MaximallyStableExtremalRegions, maximum stable extremal region) algorithm detects in each gray-scale map.Through repeatedly testing, the region at traffic signs place must be the region that grey scale change is stable in gray-scale map.Therefore after obtaining grey scale change stability region, arranging the depth-width ratio scope of traffic signs according to the features of shape of traffic signs, remove depth-width ratio region beyond set depth-width ratio scope in stability region, remaining stability region is as candidate window.In the present embodiment, the depth-width ratio of traffic signs ranges for [0.6,1.4].Figure 11 is that MESE algorithm detects the grey scale change stability region result figure obtained.Figure 12 is the candidate window figure of traffic image to be detected.Adopt above method to obtain candidate window, compare traditional sliding window detection, it is possible to greatly reduce candidate window quantity, accelerate detection speed.
S106: road traffic sign detection:
Candidate window size adjusting to Shape Classification device is inputted size, extract the HOG feature of each candidate window correspondence image block, each traffic signs Shape Classification device that input step S103 obtains, judge that whether this candidate window is the shape classification of traffic signs, if, then illustrate that this candidate window exists traffic signs, be otherwise absent from.Visible, adopt the shape that can simultaneously obtain this traffic signs in this way.Figure 13 is the road traffic sign detection result figure of traffic image to be detected.
S107: Traffic Sign Recognition:
The Traffic Sign Images that step S106 extracts is adjusted the input image size to model of cognition, and according to the traffic signs shape that step S106 judges, the Traffic Sign Images after size adjusting is inputted model of cognition, is identified result.
In the present embodiment, model of cognition adopts row convolutional neural networks.Figure 14 is the false recognition rate comparison diagram of linear type convolutional neural networks (network 1) and multiple row type convolutional neural networks (network 2).As shown in figure 14, the convergence of multiple row type convolutional neural networks faster, will reach same false recognition rate, and required training sample is less, thus reducing the training complexity of model of cognition.
From above step it can be seen that the present invention is at detection-phase, utilize the color characteristic of traffic signs to set up color probability model and carry out coarse sizing, utilize the features of shape search grey scale change stability region of traffic signs to carry out fine screening；At cognitive phase, the HOG feature of traffic signs and feature of image is utilized to be identified.The visible CF feature that present invention incorporates traffic signs, the mode adopting feature collaborative carries out detection and the identification of traffic signs, thus better being detected and recognition effect.
Although above the illustrative detailed description of the invention of the present invention being described; so that those skilled in the art understand the present invention; it is to be understood that; the invention is not restricted to the scope of detailed description of the invention; to those skilled in the art; as long as various changes limit and in the spirit and scope of the present invention determined, these changes are apparent from, and all utilize the innovation and creation of present inventive concept all at the row of protection in appended claim.

Claims (6)

1. a multiple features is collaborative road traffic sign detection and recognition methods, it is characterised in that comprise the following steps:
S1: traffic signs is classified according to the color feature of traffic signs, each colour type obtains several traffic signs sample images respectively；For every traffic signs sample image, extract the color characteristic of each pixel, according to color characteristic, all pixels of this traffic signs sample image are clustered, number of clusters is N+1, N is the primary color quantity of traffic signs, the pixel of cluster corresponding in each sample image in every class traffic signs sample image is merged, obtains N+1 sample set of this colour type, each sample set is set up corresponding Gauss model；
S2: for the traffic signs of each colour type, calculate each R, G, B value according to the color probability model of the N+1 of its correspondence and belong to the Probability p (c of each colori| x), x represents pixel R, G, B value, ciExpression color, i=1,2 ..., N+1；From the N kind primary color of this colour type, choose a kind of color representatively color, be designated as c 'i, then the probability representing color is normalized and obtains its normalization probabilitySet up each R, G, B value and belong to the probability search table representing color；
S3: traffic signs is divided into M class according to shape, each shape classification sets up a Shape Classification device based on HOG feature, and its training method is: for each Shape Classification device, obtains two class sample images, one class is the traffic signs sample image of correspondingly-shaped, another kind of for other images；Unified samples picture size, extract the HOG feature of every sample image, using the input as Shape Classification device of the HOG feature, if for the decision content of affiliated shape classification as the output of Shape Classification device, training obtains the Shape Classification device of correspondingly-shaped classification traffic signs；
S4: according to the shape classification number M of traffic signs, the traffic signs of each shape classification being respectively provided with a model of cognition, each shape classification obtains several traffic signs sample images respectively；First all sample images are carried out pretreatment, strengthen including unified image size and contrast；Adopt each pretreated sample image of shape classification that its model of cognition is trained, then it is grouped randomly ordered for pretreated traffic signs sample image after training every time, Q kind distortion mode is set, to often organize sample image randomly choose in Q kind distortion mode q kind distortion mode according to random order, sample image is carried out distortion process, its model of cognition is trained by the new samples image after adopting distortion to process, until reaching training termination condition；
S5: for traffic image to be detected, travel through each pixel in traffic image to be detected, according to each probability search table representing color, calculate each pixel and belong to the probability of such color, obtain traffic image to be detected at each probability graph represented under color, be then converted into gray-scale map；Utilizing MSER algorithm to detect the grey scale change stability region in each gray-scale map, remove depth-width ratio region beyond default depth-width ratio scope in stability region, remaining stability region is as candidate window；
S6: candidate window size adjusting to Shape Classification device is inputted size, extract the HOG feature of each candidate window correspondence image block, input each traffic signs Shape Classification device, judge that whether this candidate window is the shape classification of traffic signs, if, then illustrating that this candidate window exists traffic signs, the shape of traffic signs is the shape that correspondingly-shaped grader is carried out judging, is otherwise absent from traffic signs；
S7: be the candidate window that there is traffic signs for step S6 judged result, extract correspondence image, adjust the input image size to model of cognition, and according to the traffic signs shape that step S6 judges, Traffic Sign Images after size adjusting is inputted the model of cognition of correspondingly-shaped, is identified result.
2. road traffic sign detection according to claim 1 and recognition methods, it is characterised in that the color characteristic in described step S1 adopts color to weigh CλAnd Cλλ, its computational methods are:
Certain pixel value of note traffic signs sample image is (R, G, B), adopts Gauss light spectrum model to calculate and obtains this pixel correspondence parameter (E, Eλ,Eλλ), computing formula is:
E E λ E λ λ = H R G B
Wherein H is the coefficient matrix of 3 × 3；
The color measurement C that this pixel is correspondingλAnd CλλComputing formula be:
C λ = E λ E , C λ λ = E λ λ E .
3. road traffic sign detection according to claim 1 and recognition methods, it is characterised in that the grader in described step S3 adopts SVM classifier.
4. road traffic sign detection according to claim 1 and recognition methods, it is characterised in that the contrast in described step S4 strengthens employing restriction contrast self-adapting histogram equilibrium algorithm.
5. road traffic sign detection according to claim 1 and detection recognition methods, it is characterised in that the model of cognition in described step S4 adopts row convolutional neural networks, including first volume lamination, the first maximum pond layer, the first local normalization layer, volume Two lamination, second local normalization layer, the second maximum pond layer, the first local convolutional layer, second local convolutional layer, full articulamentum, Softmax layer, wherein:
First volume lamination receives image and carries out convolution, exports the convolutional layer obtained to the first maximum pond layer；Convolutional layer is carried out pond by the first maximum pond layer, is exported by the pond layer obtained to the first local normalization layer；Pond layer is carried out local normalization by the first local normalization layer, exports the local acknowledgement's normalization layer obtained to volume Two lamination；Volume Two lamination receives local acknowledgement's normalization layer and carries out convolution, exports the convolutional layer obtained to the second maximum pond layer；Convolutional layer is carried out pond by the second maximum pond layer, is exported by the pond layer obtained to the first local convolutional layer；First local convolutional layer reception tank layer carries out local convolution, and the local convolutional layer obtained is sent respectively to the second local convolutional layer and full articulamentum；Second convolutional layer reception local, local convolutional layer proceeds local convolution, exports the local convolutional layer obtained to meeting articulamentum；Full articulamentum merges the first local convolutional layer and the local convolutional layer of the second local convolutional layer, by fusion results output to Softmax layer；Softmax layer adopts Softmax recurrence that fusion results is classified, output category result.
6. road traffic sign detection according to claim 1 and recognition methods, it is characterised in that in described step S5, depth-width ratio ranges for [0.6,1.4].
CN201610121846.XA 2016-03-03 2016-03-03 Multi-characteristic synergic traffic sign detection and identification method Pending CN105809121A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610121846.XA CN105809121A (en) 2016-03-03 2016-03-03 Multi-characteristic synergic traffic sign detection and identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610121846.XA CN105809121A (en) 2016-03-03 2016-03-03 Multi-characteristic synergic traffic sign detection and identification method

Publications (1)

Publication Number Publication Date
CN105809121A true CN105809121A (en) 2016-07-27

Family

ID=56466069

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610121846.XA Pending CN105809121A (en) 2016-03-03 2016-03-03 Multi-characteristic synergic traffic sign detection and identification method

Country Status (1)

Country Link
CN (1) CN105809121A (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845528A (en) * 2016-12-30 2017-06-13 湖北工业大学 A kind of image classification algorithms based on K means Yu deep learning
CN106960219A (en) * 2017-03-10 2017-07-18 百度在线网络技术（北京）有限公司 Image identification method and device, computer equipment and computer-readable medium
CN107122737A (en) * 2017-04-26 2017-09-01 聊城大学 A kind of road signs automatic detection recognition methods
CN107316004A (en) * 2017-06-06 2017-11-03 西北工业大学 Space Target Recognition based on deep learning
CN107330463A (en) * 2017-06-29 2017-11-07 南京信息工程大学 Model recognizing method based on CNN multiple features combinings and many nuclear sparse expressions
CN107704509A (en) * 2017-08-31 2018-02-16 北京联合大学 A kind of method for reordering for combining stability region and deep learning
CN108304785A (en) * 2018-01-16 2018-07-20 桂林电子科技大学 Road traffic sign detection based on self-built neural network and recognition methods
CN108520212A (en) * 2018-03-27 2018-09-11 东华大学 Method for traffic sign detection based on improved convolutional neural networks
CN109255279A (en) * 2017-07-13 2019-01-22 深圳市凯立德科技股份有限公司 A kind of method and system of road traffic sign detection identification
CN109325438A (en) * 2018-09-18 2019-02-12 桂林电子科技大学 The real-time identification method of live panorama traffic sign
CN109409409A (en) * 2018-09-21 2019-03-01 长沙理工大学 HOG + CNN-based real-time detection method for traffic sign
CN109559536A (en) * 2018-12-10 2019-04-02 百度在线网络技术（北京）有限公司 Traffic lights, traffic light recognition method, device, equipment and storage medium
CN109614843A (en) * 2017-09-19 2019-04-12 福特全球技术公司 Color study
CN110097600A (en) * 2019-05-17 2019-08-06 百度在线网络技术（北京）有限公司 The method and device of traffic mark board for identification
CN110909674A (en) * 2019-11-21 2020-03-24 清华大学苏州汽车研究院（吴江） Traffic sign identification method, device, equipment and storage medium
CN111461039A (en) * 2020-04-07 2020-07-28 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111507282A (en) * 2020-04-21 2020-08-07 名创优品（横琴）企业管理有限公司 Target detection early warning analysis system, method, equipment and medium
CN111539470A (en) * 2020-04-20 2020-08-14 重庆第二师范学院 Image processing method, image processing device, computer equipment and storage medium
CN111989915A (en) * 2018-04-02 2020-11-24 幻影人工智能公司 Dynamic image region selection for visual inference
CN113392930A (en) * 2021-07-02 2021-09-14 西安电子科技大学 Traffic sign target detection method based on multi-level divide-and-conquer network
CN113469644A (en) * 2021-06-18 2021-10-01 深圳市点购电子商务控股股份有限公司 Shelf life reminding method, automatic vending device, computer equipment and storage medium
CN113506400A (en) * 2021-07-05 2021-10-15 深圳市点购电子商务控股股份有限公司 Automatic vending method, automatic vending device, computer equipment and storage medium
CN113536943A (en) * 2021-06-21 2021-10-22 上海赫千电子科技有限公司 Road traffic sign identification method based on image enhancement

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104616021A (en) * 2014-12-24 2015-05-13 深圳市腾讯计算机系统有限公司 Method and device for processing traffic sign symbols
US9092696B2 (en) * 2013-03-26 2015-07-28 Hewlett-Packard Development Company, L.P. Image sign classifier

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9092696B2 (en) * 2013-03-26 2015-07-28 Hewlett-Packard Development Company, L.P. Image sign classifier
CN104616021A (en) * 2014-12-24 2015-05-13 深圳市腾讯计算机系统有限公司 Method and device for processing traffic sign symbols

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YI YANG ET AL: "Real-Time Traffic Sign Detection via Color Probability Model and Integral Channel Features", 《CCPR 2014，PART II, CCIS 484》 *
周广波: "基于颜色和形状特征的交通标志检测", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
敬斌: "基于机器视觉的交通标志检测与识别算法研究", 《万方数据知识服务平台 学位论文》 *

Cited By (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845528A (en) * 2016-12-30 2017-06-13 湖北工业大学 A kind of image classification algorithms based on K means Yu deep learning
CN106960219A (en) * 2017-03-10 2017-07-18 百度在线网络技术（北京）有限公司 Image identification method and device, computer equipment and computer-readable medium
CN106960219B (en) * 2017-03-10 2021-04-16 百度在线网络技术（北京）有限公司 Picture identification method and device, computer equipment and computer readable medium
CN107122737A (en) * 2017-04-26 2017-09-01 聊城大学 A kind of road signs automatic detection recognition methods
CN107122737B (en) * 2017-04-26 2020-07-31 聊城大学 Automatic detection and identification method for road traffic signs
CN107316004A (en) * 2017-06-06 2017-11-03 西北工业大学 Space Target Recognition based on deep learning
CN107330463A (en) * 2017-06-29 2017-11-07 南京信息工程大学 Model recognizing method based on CNN multiple features combinings and many nuclear sparse expressions
CN109255279A (en) * 2017-07-13 2019-01-22 深圳市凯立德科技股份有限公司 A kind of method and system of road traffic sign detection identification
CN107704509A (en) * 2017-08-31 2018-02-16 北京联合大学 A kind of method for reordering for combining stability region and deep learning
CN109614843A (en) * 2017-09-19 2019-04-12 福特全球技术公司 Color study
CN108304785A (en) * 2018-01-16 2018-07-20 桂林电子科技大学 Road traffic sign detection based on self-built neural network and recognition methods
CN108520212A (en) * 2018-03-27 2018-09-11 东华大学 Method for traffic sign detection based on improved convolutional neural networks
CN111989915A (en) * 2018-04-02 2020-11-24 幻影人工智能公司 Dynamic image region selection for visual inference
CN109325438A (en) * 2018-09-18 2019-02-12 桂林电子科技大学 The real-time identification method of live panorama traffic sign
CN109325438B (en) * 2018-09-18 2021-06-15 桂林电子科技大学 Real-time identification method of live panoramic traffic sign
CN109409409A (en) * 2018-09-21 2019-03-01 长沙理工大学 HOG + CNN-based real-time detection method for traffic sign
CN109559536A (en) * 2018-12-10 2019-04-02 百度在线网络技术（北京）有限公司 Traffic lights, traffic light recognition method, device, equipment and storage medium
CN110097600A (en) * 2019-05-17 2019-08-06 百度在线网络技术（北京）有限公司 The method and device of traffic mark board for identification
CN110097600B (en) * 2019-05-17 2021-08-06 百度在线网络技术（北京）有限公司 Method and device for identifying traffic sign
CN110909674A (en) * 2019-11-21 2020-03-24 清华大学苏州汽车研究院（吴江） Traffic sign identification method, device, equipment and storage medium
CN110909674B (en) * 2019-11-21 2024-01-05 清华大学苏州汽车研究院（吴江） Traffic sign recognition method, device, equipment and storage medium
CN111461039A (en) * 2020-04-07 2020-07-28 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111461039B (en) * 2020-04-07 2022-03-25 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111539470A (en) * 2020-04-20 2020-08-14 重庆第二师范学院 Image processing method, image processing device, computer equipment and storage medium
CN111507282A (en) * 2020-04-21 2020-08-07 名创优品（横琴）企业管理有限公司 Target detection early warning analysis system, method, equipment and medium
CN111507282B (en) * 2020-04-21 2023-06-30 创优数字科技(广东)有限公司 Target detection early warning analysis system, method, equipment and medium
CN113469644A (en) * 2021-06-18 2021-10-01 深圳市点购电子商务控股股份有限公司 Shelf life reminding method, automatic vending device, computer equipment and storage medium
CN113536943A (en) * 2021-06-21 2021-10-22 上海赫千电子科技有限公司 Road traffic sign identification method based on image enhancement
CN113536943B (en) * 2021-06-21 2024-04-12 上海赫千电子科技有限公司 Road traffic sign recognition method based on image enhancement
CN113392930A (en) * 2021-07-02 2021-09-14 西安电子科技大学 Traffic sign target detection method based on multi-level divide-and-conquer network
CN113392930B (en) * 2021-07-02 2024-05-14 西安电子科技大学 Traffic sign target detection method based on multi-level division control network
CN113506400A (en) * 2021-07-05 2021-10-15 深圳市点购电子商务控股股份有限公司 Automatic vending method, automatic vending device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN105809121A (en) Multi-characteristic synergic traffic sign detection and identification method
CN107729801B (en) Vehicle color recognition system based on multitask deep convolution neural network
CN107133974B (en) Gaussian Background models the vehicle type classification method combined with Recognition with Recurrent Neural Network
CN110363122A (en) A kind of cross-domain object detection method based on multilayer feature alignment
CN109284669A (en) Pedestrian detection method based on Mask RCNN
CN107122776A (en) A kind of road traffic sign detection and recognition methods based on convolutional neural networks
CN109816024A (en) A kind of real-time automobile logo detection method based on multi-scale feature fusion and DCNN
CN109359684A (en) Fine granularity model recognizing method based on Weakly supervised positioning and subclass similarity measurement
CN109684922B (en) Multi-model finished dish identification method based on convolutional neural network
CN109902806A (en) Method is determined based on the noise image object boundary frame of convolutional neural networks
CN108171136A (en) A kind of multitask bayonet vehicle is to scheme to search the system and method for figure
CN108108761A (en) A kind of rapid transit signal lamp detection method based on depth characteristic study
CN107885764A (en) Based on the quick Hash vehicle retrieval method of multitask deep learning
CN108509954A (en) A kind of more car plate dynamic identifying methods of real-time traffic scene
CN106650786A (en) Image recognition method based on multi-column convolutional neural network fuzzy evaluation
CN108009518A (en) A kind of stratification traffic mark recognition methods based on quick two points of convolutional neural networks
CN105574550A (en) Vehicle identification method and device
CN106650731A (en) Robust license plate and logo recognition method
CN106529532A (en) License plate identification system based on integral feature channels and gray projection
CN109635784A (en) Traffic sign recognition method based on improved convolutional neural networks
CN108647700A (en) Multitask vehicle part identification model based on deep learning, method and system
CN109886147A (en) A kind of more attribute detection methods of vehicle based on the study of single network multiple-task
CN110458082A (en) A kind of city management case classification recognition methods
CN108268865A (en) Licence plate recognition method and system under a kind of natural scene based on concatenated convolutional network
CN110807485B (en) Method for fusing two-classification semantic segmentation maps into multi-classification semantic map based on high-resolution remote sensing image

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20160727