CN105825203A - Ground arrowhead sign detection and identification method based on dotted pair matching and geometric structure matching - Google Patents

Abstract

The invention belongs to the field of computer vision, relates to image processing correlation knowledge and especially relates to a shape matching method. The method is characterized in that ground arrowhead signs are extracted for identification from a video to be detected. First of all, a top view of each frame of an image is obtained by use of inverse projection mapping; secondly, image segmentation is carried out on an HSV space by use of a K mean value clustering method, communication areas whose brightness and colors accord with conditions are separated, and geometric shape screening is performed on these communication areas; then, edge extraction is performed on candidate areas, and local multi-dimensional HOG features are extracted from each edge point; and finally, by use of the features, dotted pair matching is performed between templates and the candidate areas, geometric structure matching is carried out on matching results, and categories of the areas are identified. The method provided by the invention has the following advantages: such conditions as shielding, wear, deformation, rotation and other sign interference which occur in detection of the ground arrowhead signs are overcome, and in case that the aforementioned conditions are not ideal, the method still has a quite good identification rate.

Description

Based on point, the ground arrow mark of coupling and geometry coupling is detected and recognition methods

Technical field

The invention belongs to computer vision field, relate to the relevant knowledge in image procossing, based on point, the ground arrow mark of coupling and geometry coupling is detected and recognition methods particularly to a kind of.

Background technology

In two ten years in past, traffic above-ground landmark identification, as unmanned and intelligent transportation important component part, has attracted the researcher of many computer vision fields.Therefore, the most efficient and practical technology and method are occurred in that.In these indicate, ground arrow mark contains important Traffic Information, so the detection to this kind of mark is particularly important with identification.Below the representative article starting to deliver successively from 2004 is described as follows.

Rebut, et al. J. in " Imagesegmentationandpatternrecognitionforroadmarkinganal ysis.InInternationalSymposiumonIndustrialElectronics; 2004 " literary composition, describe, by Fourier's operator, the candidate region extracted, and combine KNN grader candidate region is identified.But, the integrality of outline of candidate region is required the highest by Fourier's operator, is not suitable in this way having and blocks and the situation of candidate region well damage.Suchitra, et al. S. first candidate region is resolved into several pieces at article " Apracticalsystemforroadmarkingdetectionandrecognition.In TIP; 2009 ", utilize Grad size in the x and y direction and positive and negative by edge segments.Then every section of edge is carried out respectively Hough transformation, and the hough space obtained is carried out peakology, obtain the angle information at edge, it is judged that every section of edge is tilted to the left or is tilted to the right.Next again by template piecemeal, the left hand edge of each piece and the incline direction of right hand edge are summed up, and according to the result summed up, candidate region is detected, found out the image block meeting condition, and these image blocks are combined, it may be judged whether it is arrow mark and classifies.But the method depends on the integrity at arrow mark edge, thus bad for the arrow mark recognition effect of abrasion occurs.

YuhangHe et al. proposes a node diagnostic in article " Usingeditdistanceandjunctionfeaturetodetectandrecognizea rrowroadmarking.inIntelligentTransportationSystems (ITSC); 2014 ", each candidate region is expressed as a node string, and the position and angle according to each node encodes.Then coding result is utilized to calculate the similarity of candidate region and template image.The partial structurtes of arrow mark are combined by the method with overall structure, higher to abrasion and the robustness blocked, but relatively low for there is the arrow mark discrimination of overall deformation.

Summary of the invention

For the deficiencies in the prior art, the present invention provides a kind of and detects the ground arrow mark of coupling and geometry coupling and recognition methods based on point, the method has other vehicles situation to ground arrow mark partial occlusion at the video that vehicle-mounted camera shoots, in the case of situation that arrow mark distance automobile in ground deforms upon too far and the change of vehicle traveling direction make the arrow mark run-off the straight extracted, containing (lane line during other traffic above-ground marks in video, zebra crossing etc.), can correctly detect ground arrow mark, and it accurately and is quickly classified.

In order to achieve the above object, the technical scheme is that

A kind of ground arrow mark detection mated coupling and geometry based on point and recognition methods, the method utilizes Inverse projection method to process each frame of Vehicular video, obtains the top view of road scene.According to the luminance difference of traffic above-ground mark with ground, K means clustering method is utilized to carry out cluster to extract brightness and the qualified connected region of saturation in the HSV space of top view.Connected region is screened by the stock size utilizing arrow mark, it is thus achieved that arrow mark candidate region, completes arrow mark detection.At cognitive phase, the present invention proposes a kind of point and coupling and geometry is mated the recognition methods combined, and takes full advantage of the local of arrow mark and overall shape information.On real road, craspedodrome left/right rotation mark and mark of keeping straight on seldom exist, and turn left mark and right-hand rotation mark be symmetrical, only one of them need to be detected and identify, therefore the present invention is only to craspedodrome mark (S), turns left to indicate that (L) and right-hand rotation mark (SR) of keeping straight on detect and identify.Fig. 1 is the system block diagram of the present invention.Implement step to include:

The first step, inverse projection maps

Due to the viewing angle problem of vehicle mounted camera shooting video, there is serious perspective distortion in the traffic above-ground mark of acquisition, this can affect the recognition effect to arrow mark.In order to eliminate this impact, the most each frame road image is carried out inverse projection mapping process, obtain the top view of road scene, it is to avoid traffic above-ground mark generation serious deformation.We use trilinear method to realize inverse projection mapping, initially set up bodywork reference frame and camera coordinate system.In world coordinate system, X_vPoint to the front of automotive ordinate axis, Y_vPoint to the right being perpendicular to automotive ordinate axis, Z_vPoint to the top being perpendicular to automotive ordinate axis.If ground even, camera coordinate system initial point is at the photocentre of video camera, successively around X_v、Y_v、Z_vThe anglec of rotation of axle be ψ,And θ.Photocentre coordinate in bodywork reference frame be t=(l, d, h).If bodywork reference frame having a little for p_v(x_v,y_v,z_v), its coordinate in camera coordinate system is p_c(x_c,y_c,z_c), relation therebetween is:

$p_v^T=R\·p_c^T+t^T---\left(1\right)$

Wherein,

If it can be seen that inverse projection to be realized maps, need to calculate ψ,θ and t=(l, d, h) six outer parameters.

Taken up an official post to anticipate one in smooth ground and be parallel to X_vAxle, and be a straight line L to its distance, its parametric equation in bodywork reference frame is x_v=s, y_v=a, z_v=0, wherein s is any real number.According to pin-hole imaging model, in conjunction with formula (1), straight line L parametric equation in plane of delineation coordinate system is:

$\{\begin{array}{c}u=f_i{d}_x{y}_c/x_c\\ =f_i{d}_x({\mathrm{sr}}_{12}+{\mathrm{ar}}_{22}-{\mathrm{lr}}_{12}-{\mathrm{dr}}_{22}-{\mathrm{hr}}_{32})/({\mathrm{sr}}_{11}+{\mathrm{ar}}_{21}-{\mathrm{lr}}_{11}-{\mathrm{dr}}_{21}-{\mathrm{hr}}_{31})\\ v=-f_j{d}_y{z}_c/x_c\\ =-f_j{d}_y({\mathrm{sr}}_{13}+{\mathrm{ar}}_{23}-{\mathrm{lr}}_{13}-{\mathrm{dr}}_{23}-{\mathrm{hr}}_{33})/({\mathrm{sr}}_{11}+{\mathrm{ar}}_{21}-{\mathrm{lr}}_{11}-{\mathrm{dr}}_{21}-{\mathrm{hr}}_{31})\end{array}---\left(3\right)$

Wherein, d_x, d_yIt is respectively horizontal and vertical proportionality coefficient (camera intrinsic parameter)；U and v is the coordinate of plane of delineation coordinate system；I and j is the coordinate of pixel coordinate system；f_iAnd f_jIt is respectively the focal length (camera intrinsic parameter) in i and j direction.Straight line is fastened in image coordinate and be there is end point (u_h,v_h)

If there being three straight lines being parallel to L on ground, then these three straight lines have identical end point.We utilize this relation of equal quantity, in the case of known camera intrinsic parameter, it is possible to obtain outer parameter ψ,θ and t=(l, d, h).By outer parameter ψ,(l, d, h) substitute into formula (3) to θ and t=, find the corresponding point in image coordinate of the point on car body coordinate plane, realize the conversion to car body coordinate plane of the image coordinate plane, complete inverse projection and map, obtain the top view of road scene.

Second step, image is split

Ideally, ground arrow mark for white connected region and has obvious luminance difference with ground.But, owing to blocking the interference with other regions, it is difficult with and set the binarization method of threshold value by these extracted region out.In order to avoid the loss of ground arrow mark monochrome information, present invention employs and carry out the method for K mean cluster in particular color space image is split.Cluster is a kind of by the method for targeted packets.K mean cluster thinks that each target has the locus of oneself, to the principle of these target partitions is: the Target space position in place clusters is near as far as possible, and the Target space position in other clusters is as far as possible.K mean cluster needs specify number of clusters in advance and weigh the distance measure that two Target space positions are far and near.

Owing to hsv color space is the homogeneous color space of approximation, relative to rgb space, it more meets human visual system, and the Euclidean distance that hsv color space is 2 is approximated to direct ratio with the perception degree of people.In hsv color space, saturation component S and luminance component V can be respectively described color of image feature and shape facility, and two components are independent, and luminance component V is unrelated with the color information of image.Therefore, in the present invention RGB color image is transformed into hsv color space, saturation component S and luminance component V is recombinated, again reconstructed picture is carried out K mean cluster image is split, utilizing Euclidean distance to estimate, to divide the image into be three layers, meeting most that layer of pixel required is final segmentation result, and using the connected region that wherein comprises as candidate region, and wherein one layer contains all colours saturation and brightness meets the connected region of condition.

3rd step, candidate region is screened

In China, the size of traffic above-ground mark needs to meet unification of the motherland standard.In the present invention, we utilize physical dimension parameter to screen candidate region.Being produced distortion owing to arrow length can affect by distance, we only have chosen arrow width, the ratio of width to height and area and get rid of non-arrow region.But in actual road conditions, arrow mark can be owing to the factor such as blocking and imperfect, thus we do not screen in strict accordance with national standard size, but certain interval as filter criteria near selection standard size.

4th step, rim detection

The edge of connected region contains its major part geological information, and therefore we represent, with edge, candidate's connected region that previous step filters out.Follow-up matching result can be impacted owing to the edge of connected region there will be inevitable noise in burr and image, so we have carried out expansive working to candidate region, reduce burrs on edges, and choose the Canny edge detection algorithm that can effectively suppress noise and edge registration, candidate region is carried out Canny rim detection, obtains more smooth edge.Signal to noise ratio is estimated by this algorithm with location product, can obtain optimization Approaching Results.Therefore, the problem that this edge detection method can well solve the candidate region marginal existence burr extracted.

5th step, feature extraction and construction feature collection

The local shape information that marginal point comprises can be described by feature.Have employed a local multiple dimensioned HOG feature connected region edge is described.The division of rectangle HOG block a: image block (Block) is made up of some unit (Cell), and a unit is made up of several pixels.Do gradient direction statistics the most independently, the rectangular histogram of gained with gradient direction as transverse axis, desirable 0～180 degree or 0～360 degree of gradient direction, ground arrow mark in the present invention is detected and chooses 0～180 degree and can obtain more preferable result.This gradient scope being divided into several director more interval (orientationbin), each Direction interval can a corresponding Nogata post.In the present invention, we use 9 subintervals.

Centered by the most multiple dimensioned HOG feature refers to the marginal point of above onestep extraction, the rectangular block choosing several yardstick carries out gradient direction statistics, obtain the HOG characteristic vector of the different scale of local, and by these combination of eigenvectors together, the characteristic vector of this combination contains local feature abundant at marginal point.Feature extraction and construction feature collection concretely comprise the following steps:

5.1) carrying out the HOG feature extraction of local at marginal point, all marginal points are calculated gradient direction, choose any edge point A, intercept centered by marginal point A, size is the image block of a × a, and this image block is divided into 4 unit；Gradient direction span is divided into k subinterval, obtains 4 × k=4k dimension local HOG characteristic vector；

5.2) take centered by marginal point A, size is the image block of 2a × 2a, image block is equally divided into the sub-block of 4 a × a, respectively each sub-block is calculated 4k dimensional feature vector according to method described in 5.1, four 4k dimensional feature vectors are connected in series and obtain 4k × 4=16k dimension local HOG characteristic vector；

5.3) intercept centered by marginal point A, size is the image block of 4a × 4a, image block is equally divided into the sub-block of 4 a × a, respectively each sub-block is calculated 16k dimensional feature vector according to method described in 5.2, four 16k dimensional feature vectors are connected in series and obtain 16k × 4=64k dimension local HOG characteristic vector；

5.4) the local HOG characteristic vector under above three yardstick is connected in series, forms the 4k+16k+64k=84k dimensional feature vector of marginal point A, and represent this connected region with this 84k dimensional feature vector；

5.5) structure template base comprises craspedodrome mark S, and turn left mark L and the mark SR that turns right that keeps straight on；For the candidate region in the arrow image in template base and test image, all obtain marginal point and 84k dimensional feature vector corresponding to each marginal point according to above-mentioned steps；Candidate region all marginal points characteristic of correspondence vector constitutes the set of eigenvectors of this candidate region；All marginal point characteristic of correspondence vectors of template image constitute the set of eigenvectors of this template.

In order to realize arrow mark coupling, we construct a template base.This template base contains six kinds of ground arrow marks, including left-hand rotation of turning left, turn right, keep straight on, keep straight on, keeps straight on and turns right and craspedodrome left/right rotation.All arrow marks in template base are required for carrying out above-mentioned rim detection and two steps of feature extraction, and the characteristic vector composition characteristic collection extracted, for follow-up matching process.

6th step, point is to coupling

Image is tested for each, has all carried out above-mentioned steps to obtain candidate region and edge thereof, and each edge, candidate region is carried out feature extraction, obtain characteristic of correspondence vector.Each candidate region in one width test image needs template all of with template base to mate.In the present invention, we carry out a little to coupling first with the characteristic vector of marginal point, filter out the marginal point with identical partial structurtes, get rid of abnormity point, improve the efficiency of coupling further.

First, assume that candidate region and a certain template have extracted M and N number of marginal point respectively, the corresponding characteristic vector of each marginal point, we construct a M × N matrix D and store the Euclidean distance between two stack features vectors, and Euclidean distance represents the difference between two characteristic vectors.Wherein, the element d of D matrix_i,jEuclidean distance between the jth marginal point characteristic of correspondence vector of the i-th marginal point characteristic of correspondence vector sum template image of in store candidate region.It follows that utilize Euclidean distance matrix D to carry out a little to coupling.Assume D_iIt is the i-th row of matrix D, D_iIn element according to ascending order arrangement obtain vectorD’_iIn adjacent two elements ratio constitute vector R=[r₁,…,r_j,…,r_N-1], i.e.If r in R_kIt is first value more than predetermined threshold value α, then D '_iBefore in, the template image marginal point of k value correspondence is the point mated with candidate region i-th marginal point.By that analogy, each provisional capital of D is carried out this operation, it is possible to find, on test image, the point mated with each candidate region marginal point, formed from candidate region to the matching double points in template image direction.This matching process is two-way, and therefore our string every to D is also carried out processing equally, obtains from template image to the matching double points in direction, candidate region, and the common factor of these two groups of matching double points defines matching double points collection, here it is the final result that point is to coupling.This matching double points collection will be used for geometry coupling.

7th step, geometry mates

The point of previous step improves the effectiveness of coupling further to coupling, but candidate region multiple templates may all have identical partial structurtes with in template base, so point can accurately not provide recognition result to coupling.In the present invention, we will carry out geometry coupling on the basis of point is to coupling, be analyzed these to the whole geometry structure constituted, identify the classification of candidate region accurately.

Matching double points collection obtained in the previous step contains the matching double points between candidate region and all template images, and the some half in matching double points is present in candidate region, and half is present in template image.These two groups of points define two scatterplot respectively in test image and template image.The geometric center of scatterplot is c, the mean value computation of point coordinates get.Assume p_iAnd p_jIt is any two points on a scatterplot, d (p_i, c) with d (p_j, c) difference representation vectorWithLength, θ_ijIt is two vectorial angle (θ_ij∈[0,π]).We utilize two K₀×K₀Lower triangular matrix represents and comprises K₀The scatterplot of individual point, the two matrix is defined as:

G={g_ij|i∈[1,K₀-1]；j∈[0,i-1]}(6)

Θ={ θ_ij|i∈[1,K₀-1]；j∈[0,i-1]}(7)

Wherein, g_ij=min (d (p_i,c)/d(p_j,c),d(p_j,c)/d(p_i,c)).Obviously, two matrixes have invariance for rotating with yardstick, are only affected by the geometry of scatterplot.

G_cAnd G_tIt is respectively candidate region and the G matrix of template image, Θ_cAnd Θ_tIt is respectively candidate region and the Θ matrix of template image.We utilize Θ_cAnd Θ_tThe difference of middle element is come G_cAnd G_tIt is filtered so that the matching double points of some exceptions can be excluded.Filtering principle is as follows:

${\tilde{g}}_{c_{ij}}=\left\{\begin{array}{cc}{g}_{c_{ij}}& \left|{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}\right|\≤\mathrm{\γ}\\ 0& \left|{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}\right|>\mathrm{\γ}\end{array}\right\}---\left(8\right)$

${\tilde{g}}_{t_{ij}}=\left\{\begin{array}{cc}{g}_{t_{ij}}& \left|{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}\right|\≤\mathrm{\γ}\\ 0& \left|{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}\right|>\mathrm{\γ}\end{array}\right\}---\left(9\right)$

${\tilde{G}}_c=\left\{{\tilde{g}}_{c_{ij}}\right|i\∈\[1,K_0-1\];j\∈\[0,i-1\]\}---\left(10\right)$

${\tilde{G}}_t=\left\{{\tilde{g}}_{t_{ij}}\right|i\∈\[1,K_0-1\];j\∈\[0,i-1\]\}---\left(11\right)$

Wherein, γ is the threshold value set by experiment.Have employed Euclidean distance in the present invention to weighWithDifference:

$e=\frac{1}{s}\underset{i,j}{\Σ}{({\tilde{g}}_{c_{ij}}-{\tilde{g}}_{t_{ij}})}^2---\left(12\right)$

Wherein, s is the number of matrix non-zero element.Assume in template base, have K template image, arbitrary candidate region is all calculated an e-value with this K template image, minima in these values is if less than default threshold value, then it is identical category that the template image that this minima is corresponding is considered as with candidate region.The setting of threshold value is different to different templates, is all obtained by experiment.

The invention have the benefit that overcome the detection of ground arrow mark with identify during often the blocking of appearance, the situation such as deformation and rotation, and discrimination is higher.

Accompanying drawing explanation

Fig. 1 is system block diagram；

Fig. 2 (a) is initial pictures；Fig. 2 (b) is the top view after Inverse projection；Bianry image after Fig. 2 (c) K mean cluster；

Detection and recognition result figure when the multiple arrow of Fig. 3 (a) occurs simultaneously；Detection and recognition result figure during the abrasion of Fig. 3 (b) arrow；Detection and recognition result figure during Fig. 3 (c) arrow serious deformation；Detection and recognition result figure when Fig. 3 (d) arrow tilts；Detection and recognition result figure during the interference of other surface marks of Fig. 3 (e).

Detailed description of the invention

Step one: on real road, craspedodrome left/right rotation mark and mark of keeping straight on seldom exist, and turn left mark and right-hand rotation mark be symmetrical, only one of them need to be detected and identify, therefore the present invention is only to craspedodrome mark (S), turns left to indicate that (L) and right-hand rotation mark (SR) of keeping straight on detect and identify.

Step 2: the road assuming video camera front is smooth, and I={ (u, v) } ∈ E²Represent the initial pictures obtained, V={ (x_v,y_v,z_v)}∈E³Represent the image after Inverse projection.We want the scene top view obtained to be W={ (x, y, 0) } ∈ V.The process that inverse projection maps can be regarded as from image coordinate plane to the conversion of car body coordinate plane, and the most different coordinate positions represents Same Scene.In the present invention, we are provided with an area-of-interest (bodywork reference frame nearly car 1/2 region), formula (1) and (2) the image W after can being converted.In W, the pixel value of pixel is equal to pixel value related like vegetarian refreshments in I, if in W pixel in I correspondence position beyond the image range obtained, then in W, this pixel will be set to black.As shown in Fig. 2 (b), the top view of original image after Inverse projection, can be obtained.

Step 3: the Euclidean distance that hsv color space is 2 is approximated to direct ratio with the perception degree of people, and it has an important feature: luminance component V is unrelated with the color information of image, i.e. between saturation and the brightness of the image in hsv color space, there is relatively independent feature, first RGB color image is transformed into hsv color spatially, take out V layer therein and S layer reassembles into two channel image, this reconstructed picture is carried out K mean cluster.In the present invention, the segmentation number of plies is set to 3 by us, uses Euclidean distance to estimate and carries out 3 times clustering having obtained optimum Clustering Effect.Finally have chosen pixel value most that layer of pixel more than 200 is final segmentation result, and using the connected region that wherein comprises as candidate region.The method takes full advantage of ground arrow mark and the luminance difference on ground and colouring information to detect the candidate region in each two field picture of video.

Step 3: containing many connected regions in the image after segmentation, in order to screen these connected regions, we have asked for the width of each connected region, the ratio of width to height and area, then by the step of table 1, these three amount are carried out preliminary screening.As shown in Fig. 2 (c), after K mean cluster and physical dimension choice of parameters, obtain comprising the bianry image of all candidate regions.

Table 1 candidate region is screened

Step 4: in order to make the edge extracted more smooth, we have carried out expansive working to candidate region, decreases the interference such as burr.Then candidate region is carried out Canny rim detection, obtain more smooth edge.

Step 5: all marginal points detecting previous step calculate gradient direction, choose any edge point A, first intercept centered by marginal point A, size is the image block of 16 × 16, this image block is divided into 4 unit, gradient direction is averagely divided into 9 subintervals, thus can obtain 4 × 9=36 and tie up HOG characteristic vector.Intercept the most again centered by marginal point A, size is the image block of 32 × 32, image block is equally divided into the sub-block of 4 16 × 16 again, each sub-block is further divided into 4 unit, 4 × the 9=36 calculating each unit ties up local feature vectors, four characteristic vectors are connected in series, so can obtain 36 × 4=144 dimensional feature vector.The image block of 64 × 64 is chosen centered by marginal point A, image block is equally divided into the sub-block of 4 32 × 32, respectively each sub-block is calculated 36 × 4=144 dimensional feature vector according to the method described above, four 144 dimensional feature vectors are connected in series and obtain 36 × 4 × 4=576.Finally the local HOG feature under these three yardstick is connected in series, forms 756 dimensional feature vectors of this marginal point.

Step 6: for the candidate region in the arrow image in template base and test image, is required to obtain marginal point and 756 dimensional feature vectors corresponding to each marginal point according to above-mentioned steps.Candidate region all marginal points characteristic of correspondence vector constitutes the set of eigenvectors of this candidate region, and similarly, all marginal point characteristic of correspondence vectors of a template image constitute the set of eigenvectors of this template.It follows that each candidate region is required for carrying out with all templates in template base, a little to mating, choosing the marginal point with candidate region in template set with identical partial structurtes.We are carried out a little to coupling according to the step of table 2.

2, table is to coupling

Step 7: match point obtained in the previous step has half to belong to candidate region, and forms scatterplot at candidate region, and similarly, second half marginal point defines scatterplot on template image.It follows that we will carry out geometry coupling to the scatterplot of the scatterplot at candidate region and each template.Calculate the angle theta of any two points on scatterplot_ijG is compared with the distance to center_ij, and according to formula (6) and (7) structural matrix G and Θ.So we can respectively obtain two matrix G of candidate region_cAnd Θ_c, and the G of template image_tAnd Θ_t.Then, we utilize Θ_cAnd Θ_tThe difference of middle element according to formula (8) and (9) to G_cAnd G_tIn element screen, if the difference of angle more than 90 degree, is considered as this two groups of matching double points exceptions.Finally to the G through screening_cAnd G_tMatrix asks for Euclidean distance, and this single stepping is carried out between candidate region and each template image, if Euclidean distance minima thinks identical with candidate region classification less than threshold value, the template of this distance correspondence, otherwise candidate region is non-arrow mark.

Fig. 3 is in varied situations to the detection of ground arrow mark and the result of identification in the present invention, our method can process situation that many arrows occur simultaneously, arrow mark abrasion, arrow mark entirety deformation, arrow mark rotate, and the situation of other surface marks interference.Meanwhile, under the conditions of the most undesirable, the present invention still has preferable discrimination.

Claims (1)

1. the ground arrow mark of coupling and geometry coupling is detected and recognition methods by one kind based on point, it is characterized in that, this ground arrow mark detection only indicates S to keeping straight on recognition methods, and the mark L and the right-hand rotation mark SR that keeps straight on that turns left detects and identifies, comprises the steps:

The first step, inverse projection maps

1.1) in world coordinate system, X_vPoint to the front of automotive ordinate axis, Y_vPoint to the right being perpendicular to automotive ordinate axis, Z_vPoint to the top being perpendicular to automotive ordinate axis；If ground even, camera coordinate system initial point is at the photocentre of video camera, successively around X_v、Y_v、Z_vThe anglec of rotation of axle be ψ,And θ；Photocentre coordinate in bodywork reference frame be t=(l, d, h)；If bodywork reference frame having a little for p_v(x_v,y_v,z_v), its coordinate in camera coordinate system is p_c(x_c,y_c,z_c), relation therebetween is:

$p_v^T=R\·p_c^T+t^T---\left(1\right)$

Wherein,

1.2) for smooth ground take up an official post meaning one be parallel to X_vAxle, and arrive X_vWheelbase is from the straight line L for a, and its parametric equation in bodywork reference frame is x_v=s, y_v=a, z_v=0, wherein s is any real number；In the case of known camera intrinsic parameter, straight line L parametric equation in plane of delineation coordinate system is:

Wherein, d_x、d_y、f_iAnd f_jFor camera intrinsic parameter, d_xFor grid scale coefficient, d_yLongitudinal proportionality coefficient, f_iFor i direction focal length, f_jFor j direction focal length；；U and v is the coordinate of plane of delineation coordinate system；I and j is the coordinate of pixel coordinate system；

1.3) on road surface, two straight lines are chosen again so that it is be parallel to straight line L, calculate, according to formula (4) (5), the end point (u that straight line L fastens in image coordinate with these two straight lines respectively_h,v_h):

These three straight lines have identical end point, utilize relation of equal quantity, calculate outer parameter ψ,θ and t=(l, d, h)；

1.4) by outer parameter ψ,(l, d, h) substitute into formula (3) to θ and t=, obtain the corresponding point in image coordinate of the point on car body coordinate plane, realize the conversion to car body coordinate plane of the image coordinate plane, complete inverse projection and map, obtain the top view of road scene；

Second step, image is split

RGB color image is transformed into hsv color space, saturation component S and luminance component V is recombinated and carries out K mean cluster, utilizing Euclidean distance to estimate, to divide the image into be three layers, meeting most that layer of pixel required is final segmentation result, and using the connected region that wherein comprises as candidate region；

3rd step, screens candidate region

Use arrow mark standard size that candidate region is screened, use arrow width, the ratio of width to height and area to get rid of non-arrow region；

4th step, rim detection

Candidate region is carried out expansive working, reduces burrs on edges；Candidate region is carried out Canny rim detection, obtains more smooth edge；

5th step, feature extraction and construction feature collection

5.1) carrying out the HOG feature extraction of local at marginal point, all marginal points are calculated gradient direction, choose any edge point A, intercept centered by marginal point A, size is the image block of a × a, and this image block is divided into 4 unit；Gradient direction span is divided into k subinterval, obtains 4 × k=4k dimension local HOG characteristic vector；

5.2) take centered by marginal point A, size is the image block of 2a × 2a, image block is equally divided into the sub-block of 4 a × a, respectively each sub-block is calculated 4k dimensional feature vector according to method described in 5.1, four 4k dimensional feature vectors are connected in series and obtain 4k × 4=16k dimension local HOG characteristic vector；

5.3) intercept centered by marginal point A, size is the image block of 4a × 4a, image block is equally divided into the sub-block of 4 a × a, respectively each sub-block is calculated 16k dimensional feature vector according to method described in 5.2, four 16k dimensional feature vectors are connected in series and obtain 16k × 4=64k dimension local HOG characteristic vector；

5.4) the local HOG characteristic vector under above three yardstick is connected in series, forms the 4k+16k+64k=84k dimensional feature vector of marginal point A, and represent this connected region with this 84k dimensional feature vector；

5.5) structure template base comprises craspedodrome mark S, and turn left mark L and the mark SR that turns right that keeps straight on；For the candidate region in the arrow image in template base and test image, all obtain marginal point and 84k dimensional feature vector corresponding to each marginal point according to above-mentioned steps；Candidate region all marginal points characteristic of correspondence vector constitutes the set of eigenvectors of this candidate region；All marginal point characteristic of correspondence vectors of template image constitute the set of eigenvectors of this template；

6th step, point is to coupling

6.1) candidate region and a certain template have extracted M and N number of marginal point respectively, construct a M × N matrix D, the Euclidean distance between storage candidate region and template image characteristic vector；D_iIt it is the i-th row of matrix D；D_iIn element according to ascending order arrangement obtainD’_iThe ratio of middle adjacent element constitutes R=[r₁,…,r_j,…,r_N-1], i.e.If r in R_kIt is first value more than predetermined threshold value α, then D '_iBefore in, the template image marginal point of k value correspondence is the point mated with candidate region i-th marginal point；

6.2) each provisional capital of matrix D is carried out above-mentioned process, obtain the point mated with each candidate region marginal point on template image, formed from candidate region to the matching double points in template image direction；

6.3) row of matrix D are carried out above-mentioned process, obtain from template image to the matching double points in direction, candidate region；The matching result of row and column is taken common factor, completes the template image point with candidate region to mating；

7th step, geometry mates

Point forms scatterplot respectively to matching result on template image and test image, calculates the angle theta of any two points on scatterplot_ijG is compared with the distance to center_ij, construct two K₀×K₀Lower triangular matrix represents and comprises K₀The scatterplot of individual point:

G={g_ij|i∈[1,K₀-1]；j∈[0,i-1]}(6)

Θ={ θ_ij|i∈[1,K₀-1]；j∈[0,i-1]}(7)

G_cAnd G_tIt is respectively candidate region and the G matrix of template image, Θ_cAnd Θ_tIt is respectively candidate region and the Θ matrix of template image；We utilize Θ_cAnd Θ_tThe difference of middle element is come G_cAnd G_tIt is filtered, gets rid of abnormity point, obtainWith

${\tilde{g}}_{c_{ij}}=\left\{\begin{array}{cc}{g}_{c_{ij}}& |{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}|\≤\mathrm{\γ}\\ 0& |{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}|>\mathrm{\γ}\end{array}\right\}---\left(8\right)$

${\tilde{g}}_{t_{ij}}=\left\{\begin{array}{cc}{g}_{t_{ij}}& |{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}|\≤\mathrm{\γ}\\ 0& |{\mathrm{\θ}}_{c_{ij}}-{\mathrm{\θ}}_{t_{ij}}|>\mathrm{\γ}\end{array}\right\}---\left(9\right)$

${\tilde{G}}_c=\left\{{\tilde{g}}_{c_{ij}}\right|i\∈\[1,K_0-1\];j\∈\[0,i-1\]\}---\left(10\right)$

${\tilde{G}}_t=\left\{{\tilde{g}}_{t_{ij}}\right|i\∈\[1,K_0-1\];j\∈\[0,i-1\]\}---\left(11\right)$

Euclidean distance is used to weigh the matrix of candidate regionMatrix with all template imagesDiversity, if wherein Euclidean distance minima is considered as with candidate region being identical category less than the threshold value preset, then corresponding template image.