CN110263635B - Marker detection and identification method based on structural forest and PCANet - Google Patents

Abstract

The invention belongs to the technical field of automatic target identification, and discloses a marker detection and identification method based on a structural forest and PCANet, which comprises the steps of firstly detecting the edge structure of a pavement marker based on the structural forest; then, aiming at auxiliary lines and typical markers in the scene, extracting auxiliary line and corner characteristic regions by adopting a dynamic clustering algorithm based on skeleton extraction, and determining candidate regions of the typical markers by a maximum stable extremum region characteristic detection algorithm based on image enhancement processing; finally, the PCANet structure is adopted to carry out marker identification on the candidate area. The invention carries out structured mapping on the markers with unobvious edge structure characteristics, and adopts dynamic clustering and enhanced MSER characteristics for extraction for PCANet identification. The method can overcome the problems of unobvious marker contrast and small training data set, and has important significance in providing auxiliary information for drivers in real time.

Description

Marker detection and recognition method based on structural forest and PCANet

technical field

The invention belongs to the technical field of automatic target recognition, and in particular relates to a marker detection and recognition method based on structural forest and PCANet.

Background technique

The detection and recognition of road markers is an important part of machine vision in the application field. It can provide reliable target position information and realize the positioning function of vehicles in specific scenarios. It is widely used in automatic driving, driver assistance systems and visual navigation, etc. Usually the detection and recognition of markers can be described as: in a specific scene, according to the obtained scene marker information, a specific image preprocessing method is used to extract the feature structure of the markers in the image, and send them to the classifier for processing. Calculate and finally get the recognition result. The specific working process is:

(1) First obtain the image information in the known scene, combine the structural features of the markers in the image, design different image processing methods, reduce the interference of irrelevant areas in the image, and at the same time enhance the feature structure of the markers in the image, which is convenient for subsequent features. Descriptor calculation;

(2) Then, in the process of feature extraction, according to certain rules, for the preprocessed image sequence, design a method for extracting candidate regions of markers;

(3) Finally, the candidate regions to be identified are subjected to template matching or sent to the classifier for calculation, so as to achieve the purpose of marker identification.

At present, there are two methods for marker detection and recognition: methods based on traditional feature extraction and methods based on machine learning. Based on the traditional feature extraction method, according to the structural features of the markers, a series of complex processing is used to obtain image gradient features, which are used for the production of marker templates. This is mainly due to the use of feature points and feature descriptors, the performance of the algorithm is limited by the selection of thresholds under different lighting conditions, and the versatility is poor. In the actual operation process, it is often necessary to adjust the threshold of the algorithm according to the environment. more complicated. At the same time, the accuracy of algorithm recognition is related to the number of template matching. In order to improve the accuracy of feature extraction, it is often at the expense of increasing feature dimensions or using multi-scale templates, which results in high computational complexity and low recognition accuracy. The method based on machine learning, by constructing a large number of data sets and optimizing the training strategy, is used for the detection and recognition of markers, which greatly improves the accuracy of the recognition, but also brings about large training data sets and unsatisfactory real-time performance. Happening. In order to achieve a better recognition and classification effect, it is necessary to collect a large number of marker images and create a training set and a test set of the markers, resulting in a large training data set, and different training strategies need to be formulated to optimize the network parameters. At the same time, since the feature extraction adopts a multi-layer convolution architecture, the dimension of feature extraction is high, the computational complexity is large, and the real-time performance is not ideal. Therefore, it is necessary to study a lightweight network architecture for the identification of markers, in terms of reducing the training data set and improving the real-time performance, to realize the detection and identification of markers concisely and efficiently, and to provide corresponding auxiliary information for vehicles .

To sum up, the problems existing in the prior art are: the existing feature extraction and identification methods are overly dependent on the selection of the feature descriptor sub-parameters, the preparation of the data set, the complex computation, and the low identification efficiency.

The difficulty of solving the above technical problems: When the vehicle arrives at the predetermined site, it is necessary to identify the road markers and assist the personnel to make decisions. The detection and recognition of road markers is mainly aimed at the markers on the road surface, including: lane lines, arrows, linear markers, regional markers, optical characters, etc. For complex and special application scenarios, considering external factors such as changes in illumination, occlusion of shadows and markers, and image distortion, it is still difficult to accurately detect and identify auxiliary lines and regional markers in real time.

The significance of solving the above technical problems: In order to overcome the problems of the existing methods, such as the selection of sub-parameters based on the feature description, the large training data set, the complex calculation, and the poor real-time performance. In the aspect of feature extraction, it is necessary to adopt a processing method that enhances the feature structure comparison of markers to improve the stability of feature points and feature descriptors. At the same time, in the marker recognition stage, a lightweight network architecture is adopted to improve the real-time performance of marker recognition on the basis of ensuring the accuracy of marker recognition.

SUMMARY OF THE INVENTION

In view of the problems existing in the prior art, the present invention provides a marker detection and identification method based on structural forest and PCANet.

The present invention is achieved in this way, a marker detection and identification method based on structural forest and PCANet, the marker detection and identification method based on structural forest and PCANet comprises the following steps:

The first step is image preprocessing; obtain a video image sequence, and combine the internal parameters of the camera to perform distortion correction on the image; use the edge detection algorithm based on the structure forest to obtain a map of the image edge structure;

The second step is the extraction of candidate regions; for the extraction of auxiliary lines and corner regions, on the basis of the obtained edge structure mapping image, the skeleton of the auxiliary line is extracted by the dynamic clustering algorithm based on skeleton extraction, and the K3M sequential iteration method is used. , and perform cluster analysis on the line in Hough space. If the line is judged to be the inner point of a certain type of line cluster, it will be updated in the category of the existing line cluster; if the line is judged to be the outer point of the line cluster, it will be updated at the same time. The category and number of straight line clusters, the least squares algorithm is used to fit auxiliary lines, and the intersections between the straight lines are solved as corner points. For the extraction of typical marker regions, in order to enhance the edge structure difference between the marker region and the background, the edge structure of the background and the marker in the edge structure map image is enhanced according to the formula; Extract the maximum stable extremum region of , if it meets the set conditions, it is used as the candidate region of the marker; otherwise, as the interference region, delete the region; the background and the edge structure of the marker in the edge structure mapping image are enhanced. The formula is:

I _bd =I _gray -I _edge ;

I _db =(1-I _edge )-I _gray ;

In the formula, I _gray is the grayscale image of the input image, I _bd is the result of enhancing the edge of the image whose brightness is higher than that of other image regions, and I _db is the result of enhancing the edge of the image whose brightness is lower than that of other image regions;

The third step is the identification of markers; according to the generated corner points and the candidate regions of the markers, the binarized hash codes of the candidate regions are calculated respectively to obtain the extended histogram feature; the classifier of the pre-trained PCANet structure is used for classification and identification .

Further, the step 1 specifically includes:

根据节点分离信息增益I_j最大的原则，可确定分离函数为：Random forest is composed of N independent decision trees T _i (x), each decision tree is composed of hierarchical nodes. For each decision tree T _i (x), given its corresponding training set

According to the principle that the node separation information gain I _j is the largest, the separation function can be determined as:

其中θ_j为使信息增益最大化的参数；θ_j＝{k,γ}，k为x的某一量化的特征，γ为该特征对应的阈值；通过递归训练节点分离函数

和

直到达到了预定的决策树深度或者信息增益的阈值；信息增益定义为：In the formula,

where θ _j is the parameter that maximizes the information gain; θ _j ={k,γ}, k is a quantized feature of x, and γ is the threshold corresponding to the feature; the node separation function is trained recursively

and

Until a predetermined decision tree depth or information gain threshold is reached; information gain is defined as:

In the formula, H(S _j ) is the Shannon entropy, and py is the probability of the output label _y corresponding to the training data set S _j ;

Calculate I _j , for each node j, map all labels y in the node to discretized labels c, and use c instead of y to calculate I _j :

The output of the structured random forest is to map the high-dimensional output label y to a binary vector. In order to reduce the amount of calculation, K-means clustering or the dimensionality reduction quantification method of principal component analysis is used to judge whether the nodes with similar output labels y belong to the same marker, and the specific label number C(1,2,…, k); In the process of training, BSDS500 is used as the training set of structured random forest, and the edge detection model based on structured forest is obtained for edge detection.

Further, the second step specifically includes: adopting the K3M sequential iterative algorithm, which is divided into two steps. First, the pseudo-skeleton is extracted, and the neighborhood of the pixel points is analyzed according to the fixed rotation direction. Remove the outer contour of the marker, and extract a pseudo-skeleton with a pixel width of 2; then, perform weight coding on the 8 neighborhoods of each pixel on the pseudo-skeleton to extract the real skeleton of the image;

The coordinates of the straight line cluster l _j obtained from the skeleton extraction are converted to Hough space as (ρ _j , θ _j ), and the straight line clustering is performed on it, where θ∈[0,180°); θ is equally spaced into 180 small intervals , vote for all the angles, and then calculate the sum of the straight line lengths for each class:

In the formula, n is the number of straight line clusters, m is the number of straight line clusters of each type, and length _j represents the length of the detected straight line. When length _j > λ, all straight line clusters in the interval are clustered as new straight lines. , λ is the minimum threshold of auxiliary line length;

After Gaussian filtering is performed on the image, the training model of the structural forest can be used to obtain the map of the structural edge; the MSER feature has a better extraction effect for areas with higher or lower brightness than the surrounding brightness; After the enhancement processing, the MSER feature is used to extract the maximum stable extreme value region, and the edge-enhanced image is expressed as:

I _bd =I _gray -I _edge ;

I _db =(1-I _edge )-I _gray ;

In the formula, I _gray is the grayscale image of the input image, I _bd is the result of enhancing the edge of the image whose brightness is higher than other image areas, and I _db is the result of enhancing the edge of the image whose brightness is lower than other image areas.

Further, after the image enhancement processing, there will be obvious differences between the image markers and the edge of the background area; the MSER feature detector can be used to extract pixels with similar color information in the area, and extract the extreme values in the image. Stable region; at the same time, the number of candidate regions can be reduced by setting the relevant constraints of the markers; the constraints are:

(a) Area ratio constraint: in the MSER area, the ratio of the filling area of all pixels to the minimum enclosing rectangle of the candidate area is used to remove the curved lane markers in the image;

(b) Aspect ratio constraint: the ratio of the width and height of the smallest circumscribed rectangle of the candidate region, which is used to remove small crack regions in the image;

(c) Width constraint: the width limit range of the marker candidate area, set as a percentage of the image width;

(d) Height constraint: the height limit range of the landmark candidate area, set as a percentage of the image height.

Further, the specific implementation steps of the third step are as follows: for the input image, the PCANet structure includes the operation of zero averaging and PCA filter, and the mapping matrix of image features is formed by solving the first k feature vectors, and the PCA filter of the first layer is filtered. is expressed as:

In the formula, k ₁ ×k ₂ is the size of the sliding window, q _l (XX ^T ) is the first n eigenvectors to obtain image features, and W _l ¹ is the feature corresponding to the first L ₁ largest eigenvalues of the covariance matrix Mapping matrix; after two layers of PCA filters, the output value is subjected to binary hash coding, and the number of output coding bits is the same as the number of filters in the second layer L ₂ , which is expressed as:

For the matrix output by the first layer, it is divided into B blocks, the hash code value is counted, and its histogram features are cascaded to form extended histogram features, which represent the features of image extraction:

表示各子直方图的特征，其输出的是维度为

的向量。In the formula,

Represents the characteristics of each sub-histogram, and its output is the dimension of

vector.

Another object of the present invention is to provide an automatic driving control system applying the method for marker detection and recognition based on structural forest and PCANet.

Another object of the present invention is to provide a driver assistance system applying the method for marker detection and recognition based on structural forest and PCANet.

Another object of the present invention is to provide a visual navigation control system using the method for marker detection and recognition based on structural forest and PCANet.

To sum up, the advantages and positive effects of the present invention are as follows: the present invention designs a method for the problems of inconspicuous feature structure comparison, small training data set, and unsatisfactory real-time performance in the detection and recognition of road markers in special scenarios. A structured forest and PCANet-based marker detection and recognition method. The edge detection algorithm based on structured random forest is used to obtain the edge structure features in the image, the auxiliary line and corner candidate regions are obtained by the dynamic clustering algorithm, and the maximum stable extreme value region is obtained by the image enhancement algorithm. The network structure PCANet identifies candidate regions. The present invention can overcome the problems of inconspicuous marker comparison and small training data set, and it is of great significance to provide auxiliary information for drivers in real time.

Description of drawings

FIG. 1 is a flowchart of a method for detecting and identifying markers based on structural forest and PCANet provided by an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of marker detection and identification based on structural forest and PCANet provided by an embodiment of the present invention.

FIG. 3 is a flowchart for implementing a method for detecting and identifying markers based on structural forest and PCANet provided by an embodiment of the present invention.

Fig. 4 is the processing result diagram of each stage provided by the embodiment of the present invention;

(a) Original image (b) Edge detection based on structural forest (c) Corner and candidate region (d) Marker recognition results.

Fig. 5 is the contrast result diagram that the embodiment of the present invention provides;

In the figure: (a) the experimental results of the comparison algorithm 1; (b) the experimental results of the comparison algorithm 2; (c) the experimental results of the comparison algorithm 3; (d) the experimental results of the algorithm of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In order to overcome the shortcomings of feature extraction and identification methods that rely too much on the preparation of data sets, complex calculation, and low identification efficiency, the invention proposes a marker based on structural forest and PCANet structure for the detection and identification of road markers in special scenarios. detection and identification methods. First, the edge structure of pavement landmarks is detected based on the structural forest; then, for the auxiliary lines and typical landmarks in the scene, the dynamic clustering algorithm based on skeleton extraction is used to extract the auxiliary lines and corner feature regions, and the auxiliary lines and corner feature regions are extracted based on image enhancement. The processed maximum stable extreme value region feature detection algorithm determines the candidate regions of typical markers; finally, the candidate regions are identified by the PCANet structure.

The application principle of the present invention will be described in detail below with reference to the accompanying drawings.

As shown in FIG. 1 , the method for detecting and identifying markers based on structural forest and PCANet provided by an embodiment of the present invention includes the following steps:

S101: Through the image preprocessing stage, for the problem that the edge structure features of markers are not obvious in complex scenes, a structured random forest is used to detect the edge features of the target;

S102: In the generation stage of the candidate region, according to the obtained edge structure map, the dynamic clustering algorithm and the improved maximum stable extreme value region feature extraction algorithm are respectively used to extract the candidate region of the marker in the image;

S103: According to the actual scene target, make a target training set to train PCANet, and identify candidate marker regions.

The method for detecting and identifying markers based on structural forest and PCANet provided by the embodiment of the present invention specifically includes the following steps:

(1) Image preprocessing; first, obtain a video image sequence, and combine the internal parameters of the camera to perform distortion correction on the image, and then use the edge detection algorithm based on the structure forest to obtain a map of the image edge structure;

(2) Extraction of candidate regions. For the extraction of auxiliary lines and corner areas, based on the obtained edge structure mapping image, the skeleton of the auxiliary lines is extracted by the K3M sequential iteration method through the dynamic clustering algorithm based on skeleton extraction, and the lines are clustered in Hough space. Class analysis, if the line is judged to be the inner point of a certain type of line cluster, it will be updated in the category of the existing line cluster; if the line is judged to be the outer point of the line cluster, the category and number of the line cluster will be updated at the same time. The quadratic algorithm fits the auxiliary lines and solves the intersection between the lines as the corner area. For typical marker region extraction, in order to enhance the edge structure difference between the marker region and the background, the edge structure of the background and the marker in the edge structure map image is enhanced according to formulas (7) and (8), and then, The MSER feature detector is used to extract the maximum stable extreme value area in the image. If the set conditions are met, it will be used as the candidate area of the marker; otherwise, as the interference area, the area will be deleted;

(3) Marker identification; according to the generated corner points and marker candidate regions, the binarized hash codes of the candidate regions are calculated respectively to obtain the extended histogram feature, and then the pre-trained PCANet structure classifier is used for classification and identification .

The application principle of the present invention will be further described below with reference to the accompanying drawings.

The method for detecting and identifying markers based on structural forest and PCANet provided by the embodiment of the present invention, the implementation process of the algorithm is shown in Figure 3, and includes the following steps:

Step 1: Obtain a video image sequence, and combine the internal parameters of the camera to perform distortion correction on the image, and then use the edge detection algorithm based on the structure forest to obtain a map of the image edge structure;

Step 2: Aiming at the auxiliary lines and typical markers in the scene, on the basis of the image edge structure map, the dynamic clustering algorithm based on skeleton extraction is used to extract the auxiliary lines and corner feature candidate areas, and the image enhancement processing-based Maximum stable extreme value region feature detection algorithm to determine candidate regions of typical markers;

Step 3: According to the generated marker candidate regions, the binary hash codes of the candidate regions are calculated respectively to obtain the extended histogram feature, and then the pre-trained PCANet structure classifier is used for classification and identification.

A further improvement of the present invention is that the specific implementation steps of step 1 are as follows:

根据节点分离信息增益I_j最大的原则，可确定分离函数为：Random forest is composed of N independent decision trees T _i (x), each decision tree is composed of hierarchical nodes. For each decision tree T _i (x), given its corresponding training set

According to the principle that the node separation information gain I _j is the largest, the separation function can be determined as:

其中θ_j为使信息增益最大化的参数；θ_j＝{k,γ}，k为x的某一量化的特征，γ为该特征对应的阈值。通过递归训练节点分离函数

和

直到达到了预定的决策树深度或者信息增益的阈值。信息增益可以定义为：In the formula,

where θ _j is a parameter that maximizes the information gain; θ _j ={k,γ}, k is a quantized feature of x, and γ is a threshold corresponding to the feature. Train node separation function recursively

and

Until a predetermined decision tree depth or information gain threshold is reached. Information gain can be defined as:

In the formula, H(S _j ) is the Shannon entropy, and py is the probability of the training data set S _j corresponding to the output label _y .

In order to facilitate the calculation of I _j , for each node j, all labels y in the node are mapped to the discretized label c, and c can be used instead of y to calculate I _j :

The output of the structured random forest is to map the high-dimensional output label y to a binary vector. In order to reduce the amount of calculation, K-means clustering or the dimensionality reduction quantification method of principal component analysis is used to judge whether the nodes with similar output labels y belong to the same marker, and the specific label number C(1,2,…, k). In the process of training, BSDS500 is used as the training set of structured random forest, and the edge detection model based on structured forest is obtained for edge detection.

A further improvement of the present invention is that the specific implementation steps of step 2 are as follows:

After the binarization of the image edge detection result, it will bring about the influence of noise, discontinuous edge of marker contour and other problems. It is necessary to use the method of morphological filtering to expand, denoise and smooth the image. Complete marker contour area. In order to express the position and direction of the auxiliary lines, the skeleton is extracted by thinning operation on the morphologically filtered image. The invention adopts the K3M sequential iterative algorithm, which is divided into two steps. First, the pseudo-skeleton is extracted, and the neighborhood of the pixel point is analyzed for connectivity according to the fixed rotation direction. A pseudo-skeleton with a pixel width of 2 is extracted; then, weight coding is performed on the 8 neighborhoods of each pixel on the pseudo-skeleton to extract the real skeleton of the image.

On the basis of skeleton extraction, since there are few points in the image, HoughP transform can efficiently detect the straight line where the skeleton is located. However, because the points on the skeleton extracted by the same auxiliary line are not strictly on the same straight line, it is necessary to cluster the line clusters corresponding to the same auxiliary line. Due to the uncertainty of the number and direction of straight lines in the scene, a dynamic clustering algorithm is required. The present invention converts the straight line cluster l _j obtained by skeleton extraction into the Hough space, and the coordinate is (ρ _j , θ _j ), and performs line clustering on it, where θ∈[0,180°). Here, divide θ into 180 cells at equal intervals, vote for all angles, and then calculate the sum of the straight line lengths for each class:

In the formula, n is the number of straight line clusters, m is the number of straight line clusters of each type, and length _j represents the length of the detected straight line. When length _j > λ, all straight line clusters in the interval are clustered as new straight lines. , λ is the minimum threshold of auxiliary line length.

Improved MSER feature extraction. Under different lighting conditions, the brightness of the markers will change. After the Gaussian filter is performed on the image, the training model of the structural forest can be used in the present invention to obtain the map of the structural edge. The MSER feature has a better extraction effect for areas with higher or lower brightness than the surrounding brightness. Therefore, in the present invention, after the edge structure in the image is enhanced, the MSER feature is used to extract the maximum stable extreme value region. The edge-enhanced image can be expressed as:

I _bd =I _gray -I _edge (7)

I _db =(1-I _edge )-I _gray (8)

In the formula, I _gray is the grayscale image of the input image, I _bd is the result of enhancing the edge of the image whose brightness is higher than other image areas, and I _db is the result of enhancing the edge of the image whose brightness is lower than other image areas.

After image enhancement processing, there will be obvious differences between the landmarks of the image and the area edges of the background. Using the MSER feature detector, the pixels with similar color information in the region can be extracted, and the extreme stable region in the image can be extracted. At the same time, the number of candidate regions can be reduced by setting the relevant constraints of the markers. As shown in Table 1, the constraints are:

(a) Area ratio constraint: in the MSER area, the ratio of the filling area of all pixels to the minimum enclosing rectangle of the candidate area is mainly used to remove the curved lane markers in the image;

(b) Aspect ratio constraint: the ratio of the width and height of the smallest circumscribed rectangle of the candidate area, which is mainly used to remove small crack areas in the image;

(c) Width constraint: the width limit range of the marker candidate area, set as a percentage of the image width;

(d) Height constraint: the height limit range of the landmark candidate area, set as a percentage of the image height.

A further improvement of the present invention is that the specific implementation steps of step 3 are as follows:

The PCANet classifier is composed of a PCANet structure and a one-to-many SVM classifier. Different from the general deep learning network structure, PCANet uses a principal component analysis (PCA) network structure to replace the convolutional layer of the deep learning network, the nonlinear layer uses binary hash coding, and the pooling layer uses Block histogram, by concatenating block histograms, constitutes the extended histogram feature of the image. For the input image, the PCANet structure generally includes the operation of zero-average and PCA filter. By solving the first k feature vectors to form the mapping matrix of image features, the PCA filter of the first layer can be expressed as:

In the formula, k ₁ ×k ₂ is the size of the sliding window, q _l (XX ^T ) is the first n eigenvectors to obtain image features, and W _l ¹ is the feature corresponding to the first L ₁ largest eigenvalues of the covariance matrix Mapping matrix. After two layers of PCA filters, the output value is subjected to binary hash encoding, and the number of encoded bits of the output is the same as the number of filters in the second layer L ₂ , which can be expressed as:

For the matrix output by the first layer, it is divided into B blocks, the hash code value is counted, and its histogram features are cascaded to form extended histogram features, which represent the features of image extraction:

表示各子直方图的特征，其输出的是维度为

的向量。In the formula,

Represents the characteristics of each sub-histogram, and its output is the dimension of

vector.

Parameter optimization. For the PCANet structure, the selection of parameters needs to be based on the specific target task, including parameters such as the number of convolutional layers, the size and number of filters, and the step size of the sliding window. In the experiment, it is found that the recognition effect of the two-layer PCA filter structure is higher than that of the single-layer PCA filter structure, but continuing to increase the number of layers of the PCA filter cannot greatly improve the recognition accuracy. Therefore, the present invention selects the structure of the two-layer PCA filter. At the same time, the more the number of filters, the better the recognition effect of PCANet. Considering the types of markers in practical application scenarios, in the present invention, 8 PCA filters are selected for each layer to meet the identification requirements. Moreover, according to the area ratio of the overlapping area of the sliding window, the applicable scenarios of PCANet are different, and the area overlapping area ratio is set to 0.5 here.

Model training. According to the actual application scene, the present invention produces an image training set of markers in the scene. The image training data set mainly includes 9 types of targets, among which: 8 types of markers are regional samples, including the circular markers in the scene and the corner areas of the auxiliary lines; 1 type is a negative sample, and the area is randomly divided according to the collected images. , and perform manual screening to ensure that no marker samples are segmented. At the same time, in order to increase the diversity of samples, processing steps such as mirroring, distortion, and angle adjustment are added to the images in the dataset. In the process of training PCANet, the markers with similar structural features are combined for training and identification. The image training set consists of 400 images, and each category consists of 100 marker images, which can be divided into four categories: circular markers, “+” markers, “∟” markers, and negative samples .

The application effect of the present invention will be described in detail below in conjunction with experiments.

In order to verify the validity of the present invention, the validity, accuracy and real-time performance of the algorithm are analyzed and verified respectively. The operating environment of the program in the experiment is VS2013 configured with Opencv2.4.10 image processing library, and the image size used is 320×240.

(1) Algorithm validity verification

The algorithm of the present invention is used to detect and identify markers on the test data set of images, and the experimental results are shown in FIG. 4 . Image 4(a) is the image sequence of different markers collected in the scene; Fig. 4(b) is the edge detection result based on structural forest, it can be seen that the algorithm can effectively remove non-structural features such as spots in the background Figure 4(c) shows the extracted candidate regions for corner points of auxiliary lines and typical markers, including candidate regions generated by interference factors such as wheels and ground cracks in the background, where the auxiliary lines are represented by blue straight lines, The corners of the auxiliary line are represented by green dots, and the generated marker candidate area is represented by a green rectangle. From the results of the candidate area generation, most of the corners and marker areas can be effectively extracted, but in some scenes the edges and The corner area could not be effectively extracted, which is related to the response threshold of the auxiliary line in the dynamic clustering process; Figure 4(d) is the result of marker recognition. The valued hash coding and principal component analysis can effectively identify the interference area and the marker area, and the area identified as the marker is represented by a yellow rectangle.

(2) Performance comparison verification

Four comparison algorithms were constructed, as shown in Table 1. Compared with Algorithm 1, the classifier based on the structure forest and PCANet structure is not used. Since the HougP line detection cannot effectively extract the background line, the LSD detection line clustering and MSER feature extraction are used, and the HOG feature and the classification constructed by SVM are used to identify the markers Contrast Algorithm 2, using image enhancement processing based on structural forest, marker recognition using HOG features and a classifier constructed by SVM; Contrasting Algorithm 3, instead of using structural forest-based image enhancement, the traditional LSD algorithm is used to detect linear clusters. For dynamic clustering, the marker identification adopts the classifier based on the PCANet structure; the algorithm of the present invention adopts the structure forest and the PCANet structure to detect and identify the markers. The experimental comparison results are shown in Figure 5.

In Tables 2 to 5, the experimental statistical results of the accuracy, recall, comprehensive evaluation index and average time-consuming of the algorithm of the present invention and the comparison algorithm are respectively given. From the experimental results, in the process of detecting and identifying markers, the algorithm of the present invention has an average accuracy rate of 91.63% and a comprehensive evaluation index of 93.39%. good robustness. Moreover, the average time-consuming of a single frame of the algorithm of the present invention is 51.41 ms, which is higher than that of the other three comparison algorithms, and meets the requirement of real-time performance.

Table 1 Constraints of candidate regions

Table 2 The structure of the comparison algorithm in the experiment

Statistical results of the accuracy rate P in the experimental results of Table 3

Statistical results of recall rate R in table 4 experimental results

Statistical results of the comprehensive evaluation index F in the experimental results of Table 5

Table 6 Statistical results of single-frame time-consuming of each algorithm

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (8)

1. A marker detection and identification method based on a structural forest and PCANet is characterized by comprising the following steps:

firstly, preprocessing an image; acquiring a video image sequence, and carrying out distortion correction on the image by combining internal parameters of a camera; obtaining a mapping image of an image edge structure by adopting an edge detection algorithm based on a structural forest;

secondly, extracting a candidate region; for the extraction of auxiliary lines and angular point regions, on the basis of the obtained edge structure mapping image, extracting the skeletons of the auxiliary lines by adopting a K3M sequential iteration method through a dynamic clustering algorithm based on skeleton extraction, performing clustering analysis on the lines in Hough space, and updating in the category of the existing line cluster if the lines are judged as inner points of a certain category of line clusters; if the straight line is judged to be an external point of the straight line cluster, updating the category and the number of the straight line cluster at the same time, fitting an auxiliary line by using a least square algorithm, and solving an intersection point between the straight lines as an angular point area; for typical marker region extraction, in order to enhance the edge structure difference between a marker region and a background, the edge structures of the background and the marker in an edge structure mapping image are enhanced according to a formula; extracting the maximum stable extremum region in the image by adopting an MSER characteristic detector, and taking the maximum stable extremum region as a candidate region of the marker if the maximum stable extremum region meets the set condition; otherwise, as the interference area, deleting the area; the formula for enhancing the edge structures of the background and the marker in the edge structure mapping image is as follows:

I _bd ＝I _gray -I _edge ；

I _db ＝(1-I _edge )-I _gray ；

in the formula I _gray As a grey-scale map of the input image, I _bd As a result of enhancing edges in the image that are brighter than other image areas, I _db For brightness in image lower than other imagesThe result of enhancing the edge of the image area;

thirdly, identifying the marker; respectively calculating the binaryzation hash codes of the candidate regions according to the generated corner points and the candidate regions of the markers to obtain the features of the extended histogram; and (4) carrying out classification and identification by adopting a classifier of a pre-trained PCANet structure.

2. The structural forest and PCANet based marker detection and identification method as claimed in claim 1, wherein the first step specifically comprises:

the random forest is composed of N independent decision trees T _i (x) Each decision tree consists of layered nodes; for each decision tree T _i (x) Given its corresponding training set

Gain according to node separation information I _j The maximum principle, the separation function can be determined as:

in the formula (I), the compound is shown in the specification,

wherein theta is _j Parameters to maximize information gain; theta _j K is a certain quantized feature of x, and γ is a threshold corresponding to the feature; separating functions by recursively training nodes

And

until a predetermined decision tree depth or information gain threshold is reached; the information gain is defined as:

in the formula, H (S) _j ) Is Shannon entropy, p _y For training data set S _j Probability of corresponding output label y;

calculating I _j For each node j, mapping all labels y in the node to discretization labels c, and adopting c to replace y to calculate I _j ：

π:y∈Ya c∈C{1,2,L,k}

Outputting the structured random forest by mapping a high-dimensional output label y into a binary vector; judging whether the nodes with similar output labels y belong to the same marker by adopting a K-means clustering or principal component analysis dimension reduction quantization method, and giving specific label numbers C (1,2, L, K) of the nodes; in the training process, BSDS500 is used as a training set of the structured random forest to obtain an edge detection model based on the structured forest for edge detection.

3. The structural forest and PCANet based marker detection and identification method as claimed in claim 1, wherein the second step specifically comprises: a K3M sequential iteration algorithm is adopted, the algorithm is divided into two steps, firstly, a pseudo skeleton is extracted, connectivity analysis is carried out on the neighborhood of pixel points according to a fixed rotation direction, the outer contour of a marker is removed through continuous iterative corrosion, and the pseudo skeleton with the pixel width of 2 is extracted; then, carrying out weight coding on 8 neighborhoods of all pixel points on the pseudo skeleton, and extracting a real skeleton of the image;

extracting the skeleton to obtain a linear cluster l _j The coordinate converted into Hough space is (rho) _j ,θ _j ) Performing linear clustering on the data, wherein theta belongs to [0,180 DEG ]; dividing theta into 180 small intervals at equal intervals, voting all angles, and then calculating the sum of the lengths of straight lines of each class:

in the formula, n is the number of linear clusters, m is the number of each linear cluster, length _j Indicates the length of the detected straight line when length _j When the line length is larger than lambda, all the straight line clusters in the interval are used as new straight lines for clustering, and lambda is the minimum threshold value of the auxiliary line length;

after the image is subjected to Gaussian filtering, a structural forest training model is adopted, and a mapping chart of the structural edge can be obtained; the MSER characteristics have better extraction effect on areas higher than ambient brightness or lower than the ambient brightness; after the edge structure in the image is enhanced, the MSER characteristics are adopted to extract the maximum stable extremum region, and the image with the enhanced edge is represented as follows:

I _bd ＝I _gray -I _edge ；

I _db ＝(1-I _edge )-I _gray ；

in the formula I _gray As a grey-scale map of the input image, I _bd As a result of enhancing edges in the image that are brighter than other image areas, I _db As a result of enhancing the edges of the image that are less bright than the other image areas.

4. The structural forest and PCANet based marker detection and identification method as claimed in claim 3, wherein after the image enhancement processing, there is a significant difference between the regional edges of the markers and the background of the image; the MSER characteristic detector can be used for extracting pixel points with similar color information in the region, and an extreme value stable region in the image is extracted; meanwhile, the number of candidate regions can be reduced by setting relevant limiting conditions of the markers; the constraint conditions are as follows:

(a) area ratio constraint: in the MSER region, the ratio of the filling area of all the pixel points to the minimum circumscribed rectangle of the candidate region is used for removing curve lane markers in the image;

(b) aspect ratio constraint: the width and height ratio of the minimum circumscribed rectangle of the candidate area is used for removing the fine crack area in the image;

(c) and (3) width constraint: a width-limited range of the marker candidate set as a percentage of the image width;

(d) height constraint: the height limit range of the marker candidate is set as a percentage of the image height.

5. The structural forest and PCANet-based marker detection and identification method as claimed in claim 2, wherein the third step is implemented by the following steps: for an input image, the PCANet structure includes operations of zero averaging and a PCA filter, a mapping matrix of image features is formed by solving the first k eigenvectors, and the PCA filter of the first layer is expressed as:

in the formula, k ₁ ×k ₂ Is the size of the sliding window, q _l (XX ^T ) To find the first n feature vectors, W, of the image features _l ¹ Is the front L of the covariance matrix ₁ A feature mapping matrix corresponding to the maximum feature value; after passing through two layers of PCA filters, performing binary Hash coding on the output value, and outputting the number of coded bits and the number L of filters on the second layer ₂ The same, expressed as:

dividing the matrix output by the first layer into B blocks, counting the Hash code values of the B blocks, and cascading histogram features of the B blocks to form extended histogram features which represent the features extracted from the image:

in the formula (I), the compound is shown in the specification,

features representing sub-histograms outputting dimensions of

The vector of (2).

6. An automatic driving control system applying the structural forest and PCANet based marker detection and identification method as claimed in any one of claims 1-5.

7. A driver assistance system applying the structural forest and PCANet based marker detection and identification method as claimed in any one of claims 1-5.

8. A visual navigation control system applying the structural forest and PCANet based marker detection and identification method as claimed in any one of claims 1-5.

Images