CN105809138B - A kind of road warning markers detection and recognition methods based on piecemeal identification - Google Patents

A kind of road warning markers detection and recognition methods based on piecemeal identification Download PDF

Info

Publication number
CN105809138B
CN105809138B CN201610145850.XA CN201610145850A CN105809138B CN 105809138 B CN105809138 B CN 105809138B CN 201610145850 A CN201610145850 A CN 201610145850A CN 105809138 B CN105809138 B CN 105809138B
Authority
CN
China
Prior art keywords
image
roi
interest
triangle
mark
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610145850.XA
Other languages
Chinese (zh)
Other versions
CN105809138A (en
Inventor
贾永红
胡志雄
周明婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing tuxun Fengda Information Technology Co., Ltd
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201610145850.XA priority Critical patent/CN105809138B/en
Publication of CN105809138A publication Critical patent/CN105809138A/en
Application granted granted Critical
Publication of CN105809138B publication Critical patent/CN105809138B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
    • G06K9/00791Recognising scenes perceived from the perspective of a land vehicle, e.g. recognising lanes, obstacles or traffic signs on road scenes
    • G06K9/00818Recognising traffic signs
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/32Aligning or centering of the image pick-up or image-field
    • G06K9/3233Determination of region of interest
    • G06K9/325Detection of text region in scene imagery, real life image or Web pages, e.g. licenses plates, captions on TV images
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
    • G06K9/54Combinations of preprocessing functions

Abstract

The invention discloses a kind of road warning markers detections and recognition methods based on piecemeal identification, carry out HSV colour space transformation to Vehicular video data and carry out binary conversion treatment;It extracts the profile information of binary image and its shape is judged by the geometrical characteristic of profile, to detect the region of warning mark;The blocking characteristic and HOG feature in warning mark region are extracted, and goes out the type of mark to be identified using self-built flag library as matches criteria, obtains the result of Mark Detection.This method can guarantee that landmark identification accuracy is high and is effectively reduced missing inspection, to obtain better Mark Detection result.There is good application value in intelligent transportation field.

Description

A kind of road warning markers detection and recognition methods based on piecemeal identification
Technical field
The present invention relates to the recognition methods of traffic sign, especially a kind of road warning markers detection based on piecemeal identification With recognition methods.
Background technique
Traffic sign on road is a kind of public sign for having significant color and shape feature, for manage traffic, It indicates direction of traffic and guarantees the facility of the coast is clear and traffic safety.Traffic sign is the important carrier of traffic information, can With give vehicle, the accurate traffic guiding of pedestrian, therefore timely and accurately identification road signs information for traffic safety to close weight It wants.The road signs in China can be divided into prohibitory sign, caution sign and Warning Mark three classes, wherein 40 kinds of prohibitory sign, The red frame of white background, it is mostly round;45 kinds of caution sign, yellow bottom black surround, mostly triangle;29 kinds of Warning Mark, blue bottom white edge, mostly It is round.Therefore, pass through the detection to color and shape feature, it is easy to be divided three kinds of marks.
The identification process of road signs can be summarized as two step such as road traffic sign detection and Classification and Identification.
The detection algorithm of traffic sign is varied, is broadly divided into following four: detection based on color characteristic is based on shape The detection of shape feature, the detection based on template matching and detection based on color Geometrical mergence feature etc. are based on color characteristic Detection method processing speed it is most fast, therefore be most widely used.In the detection method based on color characteristic, first to image The segmentation of color space is carried out, the bianry image vector quantization for then obtaining segmentation finally filters out the area where traffic sign Domain.In the selection of color space, most commonly RGB color space and the color space HIS.The former detection in traffic sign Do not need to carry out colour space transformation in the process, real-time is good, but the disadvantage is that cannot very well the simulation mankind to the vision of color Perception, and the influence vulnerable to illumination.The latter needs that RGB image is first transformed into HIS (Hue: color by colour space transformation Adjust, Intensity: brightness, Saturation: saturation degree) under color model, then carry out color segmentation.HIS color model pair The description of color more meets the mankind to the visual analysis of color, and three components are independent, therefore are more advantageous to image procossing, The influence that traffic sign is imaged in illumination can also be weakened well simultaneously.But since the transformation from RGB to HIS color space belongs to Nonlinear transformation is related to anti-triangulation calculation, takes a long time, and hinders the real-time detection of traffic sign.In addition, in rgb space Signal when small variation occurs, tonal signaling can also have a greater change when being transformed into HIS space, that is, show unstable Property.
The Classification and Identification of traffic sign is the final goal of Traffic Sign Recognition problem.Currently, having more traffic mark Will sorting algorithm is broadly divided into statistical classification algorithm, neural network classification algorithm, syntax classification and Ensemble classifier method etc.. During classification, used feature is mostly color or Shape expression etc., and traffic sign at this stage is known on the whole Other method there are recognition accuracies not high, long operational time is not able to satisfy the disadvantages of demand of vehicle-mounted real-time.
Summary of the invention
In order to solve the above-mentioned technical problems, the present invention provides it is a kind of based on piecemeal identification road warning markers detection with Recognition methods.
The technical scheme adopted by the invention is that: a kind of road warning markers detection and identification side based on piecemeal identification Method, which comprises the following steps:
1. a kind of road warning markers detection and recognition methods based on piecemeal identification, which is characterized in that including following step It is rapid:
Step 1: obtaining Vehicular video data, disassembled into chronological sequence tactic image, transfer out and correspond to GPS, IMU data of time of exposure vehicle are to obtain the speed and posture of vehicle;
Step 2: Vehicular video being decomposed into frame image, and each frame is pre-processed;The pretreatment includes filtering, goes Mist removes haze, color balance;
Step 3: treated in step 2 frame image is transformed into HSV color space by rgb color space, with obtain with The color space that human visual system meets the most;
Step 4: face is carried out using frame image of the HSV threshold segmentation method after optimization to the HSV space converted in step 3 Color segmentation, and binary conversion treatment is carried out to segmentation result, obtain bianry image;
Step 5: morphology opening operation and closed operation being carried out to remove isolated point to bianry image, and calculated by edge extracting Method obtains the profile information of connected region, determines the shape of connected region according to the geometric parameter of profile on this basis, thus Detect the delta-shaped region in image, region, that is, coarse area-of-interest Rough_ where obtaining triangle mark substantially ROI;
Step 6: carrying out the profile of traffic sign to coarse area-of-interest Rough_ROI according to the geometrical characteristic of triangle Fitting, and dimension normalization and geometric correction processing are carried out to it, obtain accurate area-of-interest Exact_ROI;
Step 7: extracting the blocking characteristic of accurate area-of-interest Exact_ROI;
Step 8: extracting the HOG feature of accurate area-of-interest Exact_ROI;
Step 9: in conjunction with blocking characteristic and HOG feature, using flag library as sample, using SVM method to the mark detected Carry out Classification and Identification.
Preferably, using the HSV threshold segmentation method after optimization to the frame image converted in step 3 described in step 4 Color segmentation is done based on hsv color space, specific implementation process includes following sub-step:
Step 4.1: for the frame image after being converted from RGB color to hsv color space, calculating each in frame image The yellow degree of pixel obtains the gray level image of characterization yellow degree;
Step 4.2: for the gray level image of yellow degree, carrying out mean filter to it with square window, remove noise;
Step 4.3: the yellow bitmap of yellow degree gray level image is extracted using OTSU automatic threshold method, using square Window carries out mean filter to it, obtains the ratio of total pixel shared by yellow dots number in each square window;
Step 4.4: setting dual threshold, the image yellow bitmap after being optimized.
Preferably, the specific implementation of step 6 includes following sub-step:
Step 6.1: to the coarse area-of-interest Rough_ROI extracted in step 5, counterclockwise by harsh feeling The profile turning point of interest region Rough_ROI is numbered, and is denoted as P1、P2、P3、…、Pn, n >=3, and the contour line that will be fitted Section is denoted as L respectively1、L2、L3、…、Ln
Step 6.2: taking L1、L2、L3、…、LnLength arranges three of the line segment of first three as triangle mark to be fitted in sequence Three sides are determined three frontier juncture systems according to angular relationship, relative positional relationship, are denoted as L respectively by sideIt is left、LIt is rightAnd LBottom
Step 6.3: respectively in LIt is left、LIt is rightAnd LBottomPlace rectilinear direction, while three line segments are extended, finally obtain two Three sides of two intersections;Remember LIt is left、LIt is rightExtended line meet at PTop;LIt is right、LBottomExtended line intersects at point PIt is right;LIt is left、LBottomExtension meets at PIt is left
Step 6.4:PIt is left、PTop、PIt is rightThree vertex of the triangle detected, connect P two-by-twoIt is left、PTop、PIt is right3 points, institute's group At delta-shaped region be the accurate area-of-interest Exact_ROI of traffic sign that is partitioned into;
Step 6.5: being normalized to all accurate area-of-interest Exact_ROI sizes using bilinearity difference arithmetic 40*40 pixel;
Step 6.6: geometric correction being carried out to accurate area-of-interest Exact_ROI using affine transformation, makes it in mark Positive triangle shape in library is identical.
Preferably, the specific implementation of step 7 includes following sub-step:
Step 7.1: triangle being divided into 14 pieces, does not overlap each other and field separates each other, each block size is 10 × 10 Pixel, No. 1 block upper left starting point coordinate is (30,35), is in turn divided into 14 fritters, No. 14 fritter is in the most right of all piecemeals Lower section, inside 14 pieces of fritter whole bitmap triangle black outline borders;
Step 7.2: utilizing OTSU binarization method, grayscale image is handled, two-value kernel pattern is obtained;
Step 7.3: piecemeal window is accounted for the sum of all prospect color pixels of kernel two-value pattern extracted in each fritter The ratio of size is as characteristic value, and wherein foreground pixel value is 1, background 0, and window area size is 10 × 10;And by this The characteristic value of 14 fritters is merged into a vector, thus obtains the 1 of the accurate area-of-interest Exact_ROI of each traffic sign × 14 dimensional feature vectors, as its blocking characteristic.
Preferably, the detailed process of step 8 are as follows: angular region is averagely divided into 8 channels, and is existed with each pixel The gradient amplitude in 8 directions is weight, is weighted ballot to each channel using three linear difference methods.
Preferably, the specific implementation of step 9 includes following sub-step:
Step 9.1: using HOG feature to image rough sort, obtaining mark major class;
Step 9.2: according to specific major class, merging blocking characteristic using HOG, smart classification is carried out to mark using SVM.
Compared with the existing technology, the beneficial effects of the present invention are:
1) image restoration technology based on dark subjunctive is proposed, traffic sign under haze weather can be significantly improved Saturation degree improves detection effect.
2) the threshold method segmentation under HSV space is optimized, can be more clearly partitioned into traffic sign.
3) indicate incomplete problem for segmentation, propose the method for Accurate Curve-fitting segmentation figure spot profile, it can be to presence The traffic sign block, divided in infull situation carries out complete extraction.
4) HOG feature and blocking characteristic are merged, and is constructed suitable for SVM by slightly to smart hierarchical classification frame, significantly Ground improves the precision of label category.
Detailed description of the invention
Fig. 1 is the flow chart of the embodiment of the present invention;
The effect pair of threshold segmentation method after Fig. 2 is the conventional threshold value threshold segmentation method of the embodiment of the present invention and optimizes Than figure, wherein first is classified as colored former image, second is classified as conventional segmentation as a result, third is classified as Optimized Segmentation result;
Fig. 3 is piecemeal schematic diagram in the blocking characteristic extracting method of the embodiment of the present invention;
Fig. 4 is the cross-correlation coefficient figure between 45 kinds of the embodiment of the present invention blocking characteristics with reference to caution sign, in figure Correlation coefficient value by 0 to 1, it is more next greatly to be worth bigger expressions degree of correlation, on the contrary then more uncorrelated;
Fig. 5 is the traffic sign hierarchical classification algorithm flow chart of the embodiment of the present invention.
Specific embodiment
Understand for the ease of those of ordinary skill in the art and implement the present invention, with reference to the accompanying drawings and embodiments to this hair It is bright to be described in further detail, it should be understood that implementation example described herein is merely to illustrate and explain the present invention, not For limiting the present invention.
Referring to Fig.1, a kind of road warning markers detection and recognition methods based on piecemeal identification provided by the invention, including Following steps:
Step 1: obtaining Vehicular video, disassembled into chronological sequence tactic image, transfer out corresponding exposure GPS, IMU data of moment vehicle;
Step 2: Vehicular video is sampled and pre-processed, pretreated concrete mode is as follows:
Firstly, using Support Libraries such as GDAL, OpenCV, passing through convolution algorithm under 2010 platform of Visual Studio The filtering of video data is completed in equal operations.
Secondly as the adverse weathers situation such as overcast and rainy haze can have an impact Vehicular video imaging process, lead to traffic Blurring, it is therefore desirable to defogging processing be carried out to enhance image information to video image, algorithm is assumed using dark herein Carry out defogging enhancing processing.So-called dark assumes algorithm, refers to the image under the natural scene of sunny no haze, in big portion Divide in the subrange of non-sky, certain some pixel has at least one Color Channel with very low value, i.e. the region light intensity The minimum value of degree is the value of a very little, for any such image J, can express image dark channel with formula (1).
Wherein Ω (x) is the neighborhood centered on x, a window being normally defined centered on x, Jdark(x) it is known as The dark of image, c indicate each color in rgb space, JCFor each channel of color image.
Step 3: to treated in step 2, frame image carries out rgb color space to HSV colour space transformation, to obtain The color space met the most with human visual system;
HSV (Hue-Saturation-Value, coloration-saturation degree-brightness) color space is by A.R.Smith 1978 Year is created, alternatively referred to as hexagonal vertebral model.Coloration (H) is the attribute value of color, and value range is 0 ° -360 °, wherein 0 ° For red, 240 ° are blue, each color and its complementary color differ 180 °.Saturation degree (S) is the purity of color, and S gets over great Yan Color is purer, and the smaller color of S is lighter, gradually approximate obscure, value range 0.0-1.0, and coordinate is from the center of circle to circle in a model It is all excessive.Lightness (V) is the brightness value of color, and value range 0-255, wherein 0 is black, 255 be white.By rgb space To HSV space conversion such as formula (2).
V=max (R, G, B)
Wherein, max=max (R, G, B), min=min (R, G, B)
Step 4: the frame image for the HSV space being converted in step 3 being done using the HSV threshold segmentation method after optimization Color segmentation, and binary conversion treatment is carried out to segmentation result, obtain two-value frame image;
Since three components are mutually indepedent two-by-two in HSV color space, i.e. tone H is not influenced by brightness value, but conventional HSV color space dividing method be easy to produce the yellow pixels of many "false", the present invention divides traditional HSV color space Method has made optimization, even if concretism is that triangle alerts the yellow of traffic sign kernel in this camera colour cast, inverse Photo-beat takes the photograph, environment light is partially dark or frontlighting shooting etc. it is extreme under, color value still seems more biasing Huang than ambient enviroment, optimization point Cut that steps are as follows:
1) for the frame image of the HSV space after converting from RGB color to hsv color space, HSV space is calculated The yellow degree of each pixel in frame image obtains the gray level image f of characterization yellow degreeR(c), calculation formula such as formula (3).
Wherein, c indicates each color;
2) for the gray level image f of yellow degreeRIt (c), is r with radius1Square window carry out mean filter, removal Unnecessary noise obtains image fR‘(c)。
It 3) for extracting yellow bitmap R (x) by OTSU automatic threshold method, and is r with radius2Square window into Row mean filter, obtains imageIts meaning can be regarded as each r2Yellow dots number accounts for r in size windows2*r2Ratio.
4) the image yellow bitmap after calculation optimizationComputation rule such as formula (4).
Fig. 2 is the Contrast on effect of threshold segmentation method after conventional threshold value threshold segmentation method and optimization, wherein first row For colored former image, second is classified as conventional segmentation as a result, third is classified as Optimized Segmentation result.
Step 5: morphology opening operation and closed operation being carried out to remove isolated point to bianry image, and calculated by edge extracting Method obtains the profile information of connected region, determines the shape of connected region according to the geometric parameter of profile on this basis, thus Detect the delta-shaped region in image, the area-of-interest Rough_ in the region where obtaining triangle mark substantially, that is, rough ROI;
Shape feature includes: contour area SWheel, profile perimeter CWheel, profile minimum circumscribed circle area SCircle, profile minimum it is external Circumference CCircle, profile minimum circumscribed rectangle area SSquare, profile minimum circumscribed rectangle perimeter CSquare, profile width WWheel, profile elevations h HWheel With profile radius RWheel.The geometric properties and shape of the present invention detection threshold value table 1 of standard triangle.
1 standard triangular nature of table and shape of the present invention detection threshold value
The step of geometry extracting method based on profile, is as follows:
1) edge extracting is carried out to the two-value image obtained after segmentation, and obtains the profile of each connected region;
2) all profiles are traversed, convex surface detection is carried out, remove concave profile rather than closed outline (triangle traffic sign Shape is convex surface geometry);
3) extract the area of profile, perimeter, minimum circumscribed circle area, minimum circumscribed circle perimeter, minimum circumscribed rectangle area, The information such as minimum circumscribed rectangle perimeter, width, height filter out non-targeted figure spot.
Step 6: carrying out the contour fitting of traffic sign to coarse area-of-interest according to the geometrical characteristic of triangle, and right It carries out dimension normalization and geometric correction processing, obtains accurate area-of-interest Exact_ROI;
Due to tilting, block etc. natural scene mesoclimate complicated condition, imaging angle, by color segmentation and shape Obtained traffic sign target area and imperfect is detected, accurate profile information cannot be obtained, it is therefore desirable to completion profile.Wheel Specific step is as follows for exterior feature fitting:
Step 6.1: to the coarse area-of-interest Rough_ROI extracted in step 5, counterclockwise by harsh feeling The profile turning point of interest region Rough_ROI is numbered, and is denoted as P1、P2、P3、…、Pn, n >=3, and the contour line that will be fitted Section is denoted as L respectively1、L2、L3、…、Ln
Step 6.2: taking L1、L2、L3、…、LnLength arranges three of the line segment of first three as triangle mark to be fitted in sequence Three sides are determined three frontier juncture systems according to angular relationship, relative positional relationship, are denoted as L respectively by sideIt is left、LIt is rightAnd LBottom
Step 6.3: respectively in LIt is left、LIt is rightAnd LBottomPlace rectilinear direction, while three line segments are extended, finally obtain two Three sides of two intersections;Remember LIt is left、LIt is rightExtended line meet at PTop;LIt is right、LBottomExtended line intersects at point PIt is right;LIt is left、LBottomExtension meets at PIt is left
Step 6.4:PIt is left、PTop、PIt is rightThree vertex of the triangle detected, connect P two-by-twoIt is left、PTop、PIt is right3 points, institute's group At delta-shaped region be the accurate area-of-interest Exact_ROI of traffic sign that is partitioned into;
Step 6.5: being normalized to all accurate area-of-interest Exact_ROI sizes using bilinearity difference arithmetic 40*40 pixel;
Step 6.6: geometric correction being carried out to accurate area-of-interest Exact_ROI using affine transformation, makes it in mark Positive triangle shape in library is identical.
Due to imaging angle, weather illumination, the result that influences cause Traffic Sign Images to be partitioned into such as block cannot be fine Be identified.Although having obtained region of interest ROI in road traffic sign detection step, feature extraction operation it Before, in order to improve the robustness of feature, while in order to meet training aids input/output format requirement, it is necessary to carry out ruler to ROI Degree normalization, so that final intrinsic dimensionality, dimension are in same range, in space.
1) dimension normalization
The different sizes of traffic sign, but since feature does not have scale robustness, if the size of traffic sign is not It is same then be difficult to carry out Classification and Identification, it is therefore desirable to these various sizes of marks are subjected to dimension normalization, used in the present invention The ROI size that all image detections go out is normalized to 40*40 pixel by bilinear interpolation algorithm.
2) geometric correction
Since there are the deviation of angle, not usually positive triangle shapes, but flag library during shooting for traffic sign In mark be equilateral triangle, it is therefore desirable to the ROI that will acquire carries out geometric correction, makes itself and the shape one in flag library It causes, to reduce matching error.The correction of geometry, formula such as formula (5) are carried out in the present invention using affine transformation.
[X in formula1,X2,X3]TTo correct any point on preceding image, [X1’,X2’,X3’]TFor on image after geometric correction Corresponding same place, H-matrix are 3 × 3 transformation matrixs.In road traffic sign detection and identification, due in actual scene The distance of triangle traffic sign board and in-vehicle camera is much larger than the size of the imaging plane of camera sensor, therefore imaging plane can To be approximately considered a point, i.e. geometric correction need not be walked back and forth using perspective and penetrate transformation, transformation for mula be penetrated using walking back and forth, such as formula (6).
Transformation matrix H can be obtained from above formula, and there are six unknown numbers altogether, therefore, as can be established by obtaining 3 pairs of coordinates by 6 Equation solution transformation matrix H.In this way, equation problem is converted into the coordinate for obtaining three points of the triangle in original image, and entangle The coordinate of three points of standard triangle after just.
Step 7: extracting the blocking characteristic of Exact_ROI;
The basic thought of blocking characteristic is the kernel (removing mark dark border part) triangle warning signal image Region is divided into different fritters, then respectively extracts the feature of these pockets, finally by the feature of all fritters A feature vector is connected and composed, carries out identification classification in this, as the final feature of sign image.
In conjunction with the features such as triangle itself is symmetrical, kernel pattern is concentrated mainly on yellow area centre, the present invention is set A kind of blocking characteristic extracting method similar to pyramid symmetrical structure is counted, the regional scope of piecemeal covers kernel figure substantially Case, and can be very good the kernel pattern of 45 kinds of description different triangle warning traffic signs.As shown in Figure 3 by triangle quilt It is divided into 14 pieces, does not overlap each other and field separates each other, each block size is 10*10 pixel, and No. 1 block upper left starting point coordinate is (30,35) are in turn divided into 14 fritters, and No. 14 fritter is in the most lower right of all piecemeals, 14 pieces of fritter whole bitmap triangles Inside shape black outline border.Since kernel pattern is made of gray level image, and color usually there are two types of triangle shaped kernels patterns, OTSU binaryzation grayscale image is thus carried out again obtains two-value kernel pattern.Therefore the feature that piecemeal extracts is two-value pattern, The sum that all prospect color pixels of kernel two-value pattern are extracted in each fritter accounts for the ratio of piecemeal window size as feature Value, (foreground pixel value is 1, background 0, and window area size is 10*10), that is, the feature for seeking each piecemeal is before block is interior Scenery summation is than all pixels number in upper piece.Similarly, it calculates each piecemeal and obtains a value as small block eigenvalue, finally by 14 The characteristic value of a fritter connects into big feature, the feature vector that each triangle is flagged with 1*14 dimension is obtained, as three The feature of angle caution sign classification.
Fig. 4 gives the cross-correlation coefficient between 45 kinds of blocking characteristics with reference to caution sign, the correlation coefficient value in figure By 0 to 1, the bigger expression degree of correlation of value is more next big, on the contrary then more uncorrelated.Correlation coefficient value is expressed in a manner of color, more inclined Then indicate that degree of correlation is higher to red, color, which is biased to indigo plant, then indicates that cross-correlation coefficient i.e. easier is distinguished the two closer to 0 It comes.It is can be found that under this blocking characteristic from Fig. 4, the mutual relation numerical value of most traffic signs between any two is big Mostly 0.15 hereinafter, still can directly ignore their degree of correlation, it is believed that most caution sign can fine area Point.
Step 8: extracting the HOG feature of Exact_ROI;
From fig. 4, it can be seen that the related coefficient between the blocking characteristic of small part caution sign be it is bigger, be difficult to lead to The method for crossing classification is distinguished, and just needs to introduce the feature subsidiary classification of more higher-dimension at this time.The high dimensional feature that the present invention uses It is HOG feature, extraction step is as follows:
1) image gradient is calculated
It utilizes [- 1,0,1] gradient operator to carry out convolution algorithm to original image, obtains the component gradscalx of horizontal direction, so Use [- 1,0,1] afterwardsTGradient operator does convolution operation to original image again, obtains the gradient component gradscaly on vertical direction.So The gradient magnitude of the point, direction value are calculated using formula (7) afterwards.
2) gradient orientation histogram is constructed in unit
0~360 degree (oriented) or 0~180 degree (undirected) angular region is averagely divided into N number of channel, then by unit In channel weighting ballot where from each pixel to its gradient direction α, weight is gradient amplitude G (x, y).In order to improve HOG spy The rotational invariance of sign, channel number N usually take small value.In order to make voting results to space and angle smoothing transition, at this Invention is voted using Tri linear interpolation method, 4 units that each pixel is closed on to it, 2 angles closed in each unit Degree, 8 channel ballots are cumulative altogether.
3) unit Synthetic block, the interior normalization of block
Due to raying differentia influence, so that the fluctuation of gradient magnitude is very big, it is therefore necessary to strong to obtained gradient Angle value is normalized, and improves the robustness changed to illumination variation and contrast.
Each unit is combined into big, spatially continuous block, the feature of all units in a block is connected Get up the total characteristic of block can be obtained.Due to block divide when have overlapping, the feature of each unit after normalization operation, It can repeatedly be appeared in last total characteristic with different value, thus obtain HOG descriptor.
4) HOG feature is collected
Similar and 3) step operation, the characteristic set in block is included in window, is included by it in detection window Feature is connected in block, obtains the HOG feature eventually for classification.
Step 9: in conjunction with blocking characteristic and HOG feature, using flag library as sample, using SVM method to the mark detected Carry out Classification and Identification.
The present invention is using a kind of by slightly to smart hierarchical classification algorithm, flow chart element is as shown in Figure 5.The process of classification is divided into Two steps.First with HOG feature to image rough sort, mark major class is obtained;Then HOG fusion point is recycled according to specific major class Block feature and SVM obtain final result to image disaggregated classification.Due to carry out geometric correction before subdivision, by geometric correction ROI afterwards can be very good eliminate visual angle inclination caused by influence so that classifier can preferably find mark between it is thin Elementary errors is different, improves classifying quality.
The present invention analyzes the detection for counting final triangle mark and recognition result.Statistic mixed-state recognition correct rate etc. is common Evaluation index.
In conjunction with the area of GPS, IMU data and the ROI region extracted that are obtained in step 1, estimates identification and obtain Traffic sign GPS coordinate, to realize the positioning of traffic sign.
It should be understood that the part that this specification does not elaborate belongs to the prior art.
It should be understood that the above-mentioned description for preferred embodiment is more detailed, can not therefore be considered to this The limitation of invention patent protection range, those skilled in the art under the inspiration of the present invention, are not departing from power of the present invention Benefit requires to make replacement or deformation under protected ambit, fall within the scope of protection of the present invention, this hair It is bright range is claimed to be determined by the appended claims.

Claims (4)

1. a kind of road warning markers detection and recognition methods based on piecemeal identification, which comprises the following steps:
Step 1: obtaining Vehicular video data, disassembled into chronological sequence tactic image, transfer out corresponding exposure GPS, IMU data of moment vehicle are to obtain the speed and posture of vehicle;
Step 2: Vehicular video being decomposed into frame image, and each frame is pre-processed;It is described pretreatment include filtering, defogging, Remove haze, color balance;
Step 3: treated in step 2 frame image being transformed into HSV color space by rgb color space, to obtain and human eye The color space that vision system meets the most;
Step 4: color point is carried out using frame image of the HSV threshold segmentation method after optimization to the HSV space converted in step 3 It cuts, and binary conversion treatment is carried out to segmentation result, obtain bianry image;
Step 5: morphology opening operation and closed operation being carried out to remove isolated point to bianry image, and obtained by Boundary extracting algorithm The profile information of connected region is taken, the shape of connected region is determined according to the geometric parameter of profile on this basis, to detect Delta-shaped region in image out, region, that is, coarse area-of-interest Rough_ROI where obtaining triangle mark substantially;
Step 6: the profile for carrying out traffic sign to coarse area-of-interest Rough_ROI according to the geometrical characteristic of triangle is quasi- It closes, and carries out dimension normalization and geometric correction processing to it, obtain accurate area-of-interest Exact_ROI;
Specific implementation includes following sub-step:
Step 6.1:, counterclockwise will be coarse interested to the coarse area-of-interest Rough_ROI extracted in step 5 The profile turning point of region Rough_ROI is numbered, and is denoted as P1、P2、P3、…、Pn, n >=3, and the profile line segment fitted is divided L is not denoted as it1、L2、L3、…、Ln
Step 6.2: taking L1、L2、L3、…、LnLength arranges three sides of the line segment of first three as triangle mark to be fitted in sequence, will Three sides determine three frontier juncture systems according to angular relationship, relative positional relationship, are denoted as L respectivelyIt is left、LIt is rightAnd LBottom
Step 6.3: respectively in LIt is left、LIt is rightAnd LBottomPlace rectilinear direction, while three line segments are extended, finally obtain two two-phases Three sides handed over;Remember LIt is left、LIt is rightExtended line meet at PTop;LIt is right、LBottomExtended line intersects at point PIt is right;LIt is left、LBottomExtension meets at PIt is left
Step 6.4:PIt is left、PTop、PIt is rightThree vertex of the triangle detected, connect P two-by-twoIt is left、PTop、PIt is rightIt is 3 points, composed Delta-shaped region is the accurate area-of-interest Exact_ROI of traffic sign being partitioned into;
Step 6.5: all accurate area-of-interest Exact_ROI sizes being normalized to 40 × 40 using bilinearity difference arithmetic Pixel;
Step 6.6: geometric correction being carried out to accurate area-of-interest Exact_ROI using affine transformation, makes it in flag library Positive triangle shape it is identical;
Step 7: extracting the blocking characteristic of accurate area-of-interest Exact_ROI;
Specific implementation includes following sub-step:
Step 7.1: triangle is divided into 14 pieces, does not overlap each other and field separates each other, each block size is 10*10 pixel, No. 1 block upper left starting point coordinate is (30,35), is in turn divided into 14 fritters, No. 14 fritter in the most lower right of all piecemeals, Inside 14 pieces of fritter whole bitmap triangle black outline borders;
Step 7.2: utilizing OTSU binarization method, grayscale image is handled, two-value kernel pattern is obtained;
Step 7.3: piecemeal window size is accounted for the sum of all prospect color pixels of kernel two-value pattern extracted in each fritter Ratio as characteristic value, wherein foreground pixel value is 1, background 0, and window area size is 10 × 10;And by this 14 The characteristic value of fritter is merged into a vector, thus obtains the 1 × 14 of the accurate area-of-interest Exact_ROI of each traffic sign Dimensional feature vector, as its blocking characteristic;
Step 8: extracting the HOG feature of accurate area-of-interest Exact_ROI;
Step 9: in conjunction with blocking characteristic and HOG feature, using flag library as sample, the mark detected being carried out using SVM method Classification and Identification.
2. the road warning markers detection and recognition methods according to claim 1 based on piecemeal identification, it is characterised in that: Hsv color space is based on to the frame image converted in step 3 using the HSV threshold segmentation method after optimization described in step 4 to do Color segmentation, specific implementation process include following sub-step:
Step 4.1: for the frame image after converting from RGB color to hsv color space, calculating each pixel in frame image Yellow degree, obtain characterization yellow degree gray level image;
Step 4.2: for the gray level image of yellow degree, carrying out mean filter to it with square window, remove noise;
Step 4.3: the yellow bitmap of yellow degree gray level image is extracted using OTSU automatic threshold method, using square window Mean filter is carried out to it, obtains the ratio of total pixel shared by yellow dots number in each square window;
Step 4.4: setting dual threshold, the image yellow bitmap after being optimized.
3. the road warning markers detection and recognition methods according to claim 1 based on piecemeal identification, which is characterized in that The detailed process of step 8 are as follows: angular region is averagely divided into 8 channels, and the gradient amplitude with each pixel in 8 directions For weight, ballot is weighted to each channel using three linear difference methods.
4. the road warning markers detection and recognition methods according to claim 1 based on piecemeal identification, which is characterized in that The specific implementation of step 9 includes following sub-step:
Step 9.1: using HOG feature to image rough sort, obtaining mark major class;
Step 9.2: according to specific major class, merging blocking characteristic using HOG, smart classification is carried out to mark using SVM.
CN201610145850.XA 2016-03-15 2016-03-15 A kind of road warning markers detection and recognition methods based on piecemeal identification Active CN105809138B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610145850.XA CN105809138B (en) 2016-03-15 2016-03-15 A kind of road warning markers detection and recognition methods based on piecemeal identification

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610145850.XA CN105809138B (en) 2016-03-15 2016-03-15 A kind of road warning markers detection and recognition methods based on piecemeal identification

Publications (2)

Publication Number Publication Date
CN105809138A CN105809138A (en) 2016-07-27
CN105809138B true CN105809138B (en) 2019-01-04

Family

ID=56467390

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610145850.XA Active CN105809138B (en) 2016-03-15 2016-03-15 A kind of road warning markers detection and recognition methods based on piecemeal identification

Country Status (1)

Country Link
CN (1) CN105809138B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10063917B2 (en) * 2016-03-16 2018-08-28 Sorenson Media Inc. Fingerprint layouts for content fingerprinting
CN106023623A (en) * 2016-07-28 2016-10-12 南京理工大学 Recognition and early warning method of vehicle-borne traffic signal and symbol based on machine vision
CN106649851A (en) * 2016-12-30 2017-05-10 徐庆 Similar trademark query result ordering method, device and trademark server thereof
CN106781521B (en) * 2016-12-30 2020-12-25 东软集团股份有限公司 Traffic signal lamp identification method and device
CN107066933B (en) 2017-01-25 2020-06-05 武汉极目智能技术有限公司 Road sign identification method and system
CN107025796A (en) * 2017-04-28 2017-08-08 北京理工大学珠海学院 Automobile assistant driving vision early warning system and its method for early warning
CN109842764A (en) * 2017-11-27 2019-06-04 韩劝劝 Image shot by cell phone compound platform
CN109214434A (en) * 2018-08-20 2019-01-15 上海萃舟智能科技有限公司 A kind of method for traffic sign detection and device
CN111433779A (en) * 2018-11-09 2020-07-17 北京嘀嘀无限科技发展有限公司 System and method for identifying road characteristics
CN109215364B (en) * 2018-11-19 2020-08-18 长沙智能驾驶研究院有限公司 Traffic signal recognition method, system, device and storage medium
CN109543691A (en) * 2018-12-27 2019-03-29 斑马网络技术有限公司 Ponding recognition methods, device and storage medium
CN109782214B (en) * 2019-01-26 2021-02-26 哈尔滨汇鑫仪器仪表有限责任公司 Electric energy meter state remote sending mechanism
CN110414362A (en) * 2019-07-02 2019-11-05 安徽继远软件有限公司 Electric power image data augmentation method based on production confrontation network
CN111080595A (en) * 2019-12-09 2020-04-28 北京字节跳动网络技术有限公司 Image processing method, image processing device, electronic equipment and computer readable medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542260A (en) * 2011-12-30 2012-07-04 中南大学 Method for recognizing road traffic sign for unmanned vehicle
CN103971126B (en) * 2014-05-12 2017-08-08 百度在线网络技术(北京)有限公司 A kind of traffic sign recognition method and device
CN104517103A (en) * 2014-12-26 2015-04-15 广州中国科学院先进技术研究所 Traffic sign classification method based on deep neural network

Also Published As

Publication number Publication date
CN105809138A (en) 2016-07-27

Similar Documents

Publication Publication Date Title
CN105809138B (en) A kind of road warning markers detection and recognition methods based on piecemeal identification
CN105260699B (en) A kind of processing method and processing device of lane line data
CN105488454B (en) Front vehicles detection and ranging based on monocular vision
CN108108761B (en) Rapid traffic signal lamp detection method based on deep feature learning
CN103971126A (en) Method and device for identifying traffic signs
CN106682601B (en) A kind of driver's violation call detection method based on multidimensional information Fusion Features
CN106651872A (en) Prewitt operator-based pavement crack recognition method and system
CN103218831B (en) A kind of video frequency motion target classifying identification method based on profile constraint
CN107273896A (en) A kind of car plate detection recognition methods based on image recognition
CN102096823A (en) Face detection method based on Gaussian model and minimum mean-square deviation
CN103035013A (en) Accurate moving shadow detection method based on multi-feature fusion
CN105678318B (en) The matching process and device of traffic sign
Khalid et al. Automatic measurement of the traffic sign with digital segmentation and recognition<? show [AQ="" ID=" Q1]"?
CN104217196B (en) A kind of remote sensing image circle oil tank automatic testing method
CN102938057B (en) A kind of method for eliminating vehicle shadow and device
CN107066933B (en) Road sign identification method and system
CN106503748A (en) A kind of based on S SIFT features and the vehicle targets of SVM training aids
CN106529532A (en) License plate identification system based on integral feature channels and gray projection
CN104143077B (en) Pedestrian target search method and system based on image
CN106529461A (en) Vehicle model identifying algorithm based on integral characteristic channel and SVM training device
CN107704853A (en) A kind of recognition methods of the traffic lights based on multi-categorizer
CN103680145A (en) Automatic pedestrian and vehicle recognition method based on local image characteristics
CN107154044B (en) Chinese food image segmentation method
CN105069816A (en) Method and system for counting inflow and outflow people
CN106407951A (en) Monocular vision-based nighttime front vehicle detection method

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20200525

Address after: Room 901, 9 / F, building 12, Taiyanggong Middle Road, Chaoyang District, Beijing 100020

Patentee after: Beijing tuxun Fengda Information Technology Co., Ltd

Address before: 430072 Hubei Province, Wuhan city Wuchang District of Wuhan University Luojiashan

Patentee before: WUHAN University

TR01 Transfer of patent right