CN116738211A - Road condition identification method based on multi-source heterogeneous data fusion - Google Patents

Road condition identification method based on multi-source heterogeneous data fusion Download PDF

Info

Publication number
CN116738211A
CN116738211A CN202310754496.0A CN202310754496A CN116738211A CN 116738211 A CN116738211 A CN 116738211A CN 202310754496 A CN202310754496 A CN 202310754496A CN 116738211 A CN116738211 A CN 116738211A
Authority
CN
China
Prior art keywords
data
road
fusion
vehicle
road condition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310754496.0A
Other languages
Chinese (zh)
Inventor
黄姣茹
慕嘉
高嵩
陈超波
钱富才
张晓艳
李继超
宋浩辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Technological University
Original Assignee
Xian Technological University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Technological University filed Critical Xian Technological University
Priority to CN202310754496.0A priority Critical patent/CN116738211A/en
Publication of CN116738211A publication Critical patent/CN116738211A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a road condition identification method based on multi-source heterogeneous data fusion. The method comprises the steps of firstly collecting road image data and vehicle speed, acceleration and wheel speed data by adopting a plurality of sensors, preprocessing all partial data, extracting features by adopting different algorithms, carrying out feature level fusion and dimension reduction on extracted feature vectors by adopting a multi-source heterogeneous data space-time fusion strategy, and finally training and testing by adopting a least square support vector machine based on particle swarm optimization, so as to establish a road condition identification model. The invention realizes accurate and efficient identification of road conditions caused by different bad weather by preprocessing, feature extraction and identification model establishment on the data acquired by the multiple sensors. The method can fully discover a plurality of sensor resources, and further comprehensively analyze the acquired data, so that the road condition classification recognition accuracy and the environmental adaptability are improved.

Description

Road condition identification method based on multi-source heterogeneous data fusion
Technical Field
The invention relates to the technical field of road condition identification, in particular to a road condition identification method based on multi-source heterogeneous data fusion.
Background
With the development of automatic driving technology, the intellectualization and unmanned vehicle have become the trend of modern automobiles, but the safety problem in the driving process is always very critical. Studies have shown that severe weather and its resulting road conditions such as water accumulation, snow accumulation and ice formation are major factors affecting the driving safety of vehicles. Especially under the ice and snow road surface condition, the traffic accident rate is obviously increased.
Modern smart vehicles themselves contain a large number of sensors, such as wheel speed sensors, vehicle radar, cameras, and the like. The abundant sensors provide possibility for the intelligent automobile to automatically learn the road surface, learn the road surface and automatically adapt to the road surface like a person. In the field of road condition identification, according to the principle on which the identification is based, two major categories are an indirect identification method based on vehicle dynamics and a direct identification method based on sensors:
1. the indirect recognition method based on the vehicle dynamics is mainly used for recognizing different road surface parameters, a model reflecting the correlation of various factors and road surface parameters is required to be established, the road surface parameters are calculated by measuring the correlation factors and utilizing the model, the indirect recognition method is widely applied to recognition of road surface adhesion coefficients and road surface roughness, the indirect recognition method does not have certain predictability, and the recognition accuracy is completely dependent on the accuracy of the establishment of the model.
2. The direct recognition method based on the vehicle-mounted sensor can recognize roads under different conditions before the tire is contacted with the road surface, and has certain predictability, but if single sensor data are used, the road surface characteristics are not sufficiently reflected, and the problem of low recognition precision is easily generated.
With the development of automatic driving technology, the current method for improving the driving safety of vehicles by identifying different road condition information is endless.
The Chinese patent CN111695418A is a method and a system for carrying out safe driving based on road condition detection, wherein road side equipment is utilized to identify and classify collected road images, output road condition information is generated into message information, and vehicle broadcasting is utilized to broadcast so as to enable a driver to make driving decisions. Although the vehicle running safety can be improved to a certain extent, the road side equipment is high in installation cost and small in application range, the broadcasting range of the message information is limited to a certain extent, and the vehicle cannot actively recognize the road condition information.
The Chinese patent CN111860322A is an unstructured road surface type recognition method based on multi-source sensor information fusion, which collects road image information, vehicle state and GPS information, and realizes recognition and classification of unstructured road surfaces by training different classifiers. The method for identifying the road surfaces in different conditions by adopting the multi-source information fusion method is characterized in that the training time of different classifiers is longer, and the initial decision operation cost is higher; and the accuracy of the output results depends on the setting of the decision between the two classifiers.
The existing fusion recognition method is concentrated on recognizing road surface unevenness or concentrated on researching decision-level fusion of data, vehicle dynamics feature data and road image feature data in different conditions are not fully utilized, and recognition accuracy is generally low.
Disclosure of Invention
In view of the above, the invention provides a road condition identification method based on multi-source heterogeneous data fusion, which is a method for carrying out feature fusion on vehicle dynamics feature data and image feature data, which can directly reflect road adhesion coefficients, so as to more fully describe road features caused by different weather and improve accuracy of road identification and environmental adaptability.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a road condition identification method based on multi-source heterogeneous data fusion comprises the following steps:
step 1), road surface perception information acquisition:
acquiring pavement image data and vehicle dynamics related data under different road conditions by using an unmanned intelligent vehicle;
step 2), data preprocessing and feature extraction:
preprocessing the pavement image data and the vehicle dynamics related data acquired in the step 1); extracting corresponding characteristic values by utilizing a plurality of characteristic extraction algorithms, and finally forming two characteristic matrixes for describing different road conditions;
step 3), multi-source heterogeneous data fusion and dimension reduction:
performing feature level fusion on the two feature matrixes describing different road conditions in the step 2) by utilizing a multi-source heterogeneous data space-time fusion strategy, and then performing dimension reduction on the fused feature matrixes by utilizing a principal component analysis method;
step 4), building a road condition identification model:
forming a data set by the feature matrix after the dimension reduction in the step 3) and the corresponding road type label; the data set is randomly divided into a training set and a testing set according to the ratio of 4:1; and finally, inputting the training set and the testing set into a least square support vector machine model based on particle swarm optimization for training and testing to obtain a road condition identification model.
Further, the vehicle dynamics related data in step 1) includes vehicle longitudinal speed, acceleration and wheel speed information.
Further, the method for preprocessing the pavement image data and extracting the characteristics in the step 2) comprises the following steps:
preprocessing road image data by using a USM and histogram equalization method; respectively extracting color features and texture features of the road image by using a color histogram method, ULBP and GLCM, and finally forming an image feature matrix;
the method for preprocessing the dynamics and extracting the characteristics of the vehicle comprises the following steps: preprocessing vehicle dynamics data by using a wavelet threshold denoising method; and extracting frequency domain features of the vehicle dynamics signals by using the power spectrum density, and finally forming a vehicle dynamics feature matrix.
Further, the specific method of the step 3) is as follows:
step 3.1) the multi-source heterogeneous data space-time fusion strategy comprises time data fusion and space data fusion:
time data fusion: taking a vehicle starting point as a coordinate origin, taking the output time of the integrated navigation system as a standard, keeping the sampling frequencies of different sensors the same and requiring to start sampling at the same time, and recording data in an offline mode to enable the acquired data to have a consistent system time stamp;
spatial data fusion: firstly, acquiring driving distance information of a vehicle and establishing a vehicle position axis x, taking the center of a front wheel of the vehicle as an origin of coordinates, secondly, dividing a vehicle dynamics feature vector into segments with equal length as image data, and then fusing each image feature vector with the segmented vehicle dynamics feature vector, wherein the feature vector formed by each segment of signals can be expressed as
Wherein the method comprises the steps ofRepresents a first type of road surface, f LA 、f WS F LS The longitudinal acceleration, the wheel speed and the longitudinal speed characteristic value are represented, and the vehicle dynamics data in 0.25 second after the final vehicle acquires a road picture are fused;
step 3.2) feature matrix dimension reduction: and (3) carrying out standardization and normalization processing on the fusion feature matrix, then reducing the dimension of the fusion feature matrix to 88 dimensions by using a PCA algorithm, reserving 75% of effective information, and providing the effective information for a least square support vector machine based on particle swarm optimization for classification recognition training.
Further, the specific method of the step 4 is as follows:
step 4.1, respectively forming a dataset by the road image features, the vehicle dynamics features, the fusion feature matrix subjected to dimension reduction and the corresponding road type labels, and then randomly dividing the dataset into a training set and a testing set according to the ratio of 4:1;
and 4.2, improving the problem of the original support vector machine, introducing a particle swarm optimization algorithm to optimize the parameter set of the least square support vector machine, and establishing a road condition identification model.
Further, the specific method of step 4.2 is as follows:
step 4.2.1, when the data scale reaches thousands or even tens of thousands, selecting an error square term as an optimization target by adopting a least square support vector machine model, taking equality constraint as a constraint condition, and realizing a final decision function by solving a linear equation set;
the Lagrangian function is introduced, and the least square support vector machine optimal classification function can be obtained through arrangement as follows:
wherein: k (x) k X) is a kernel function, a Gaussian radial basis kernel function is adopted, and the mathematical expression is as follows:
k(x,x k )＝exp{-|x-x k | 2 /σ 2 }
wherein sigma is the radial basis function width;
step 4.2.2, selecting a one-to-one coding classification method, and realizing multi-classification by constructing a plurality of classification LSSVMs and combining the classification LSSVMs with a combined coding method;
step 4.2.3 selecting particle swarm optimization algorithm to support the parameter set (γ, σ) in the vector machine for least squares 2 ) Optimizing; the road condition recognition model modeling flow is as follows:
(1) Reading in a road condition data sample set fused with the dimension reduction;
(2) Initializing PSO population, setting particle population number as 50, iteration number as 200, setting maximum value of inertial weight of speed update as 0.95 and minimum value as 0.6, and randomly generating a group (gamma, sigma) 2 ) As an initial position of the particles;
(3) According to the current (gamma, sigma 2 ) Performing LSSVM training, calculating model fitting error under given parameters, and taking the model fitting error as a fitness function, wherein the fitness function is defined as:
E＝1-P i
wherein E is an error, P i The training accuracy is given parameters;
(4) Using the model fitting error as an adaptive value, updating the optimal adaptive value sum corresponding to the particle and the population according to the adaptive value of the particle, and updating the speed and the position of each particle to obtain a new LSSVM parameter value (gamma, sigma) 2 )；
(5) If the maximum iteration times are not met or the ending condition is met, returning to the step (3), otherwise, outputting the optimal parameter value;
(6) And re-training the LSSVM by using the finally obtained optimal parameters, and establishing a road condition identification model.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention can fuse the vehicle dynamics characteristic data directly reflecting the road adhesion coefficient with the image characteristic data to more fully describe the road characteristics caused by different weather; specifically, as for the image features, the vehicle dynamics features can provide features which directly reflect friction coefficients of different roads, so that the defect that the image features are easily influenced by the environment is overcome to a certain extent; for vehicle dynamics features, the image features can provide visually distinguishable features for a road surface with similar vehicle dynamics features. Therefore, the road condition classification recognition accuracy and the environment adaptability can be improved by using the multi-source heterogeneous fusion characteristics.
2. The invention provides a method for optimizing a parameter set by using a Least square support vector machine based on particle swarm optimization as a road condition recognition model and combining a Least Square (LS) method with a particle swarm optimization algorithm (Particle Swarm Optimization, PSO). The invention adopts Gaussian radial basis function, so that the parameter groups needing to be optimized are (gamma, sigma) 2 ) And setting a model fitting error under a given parameter as an fitness function, and iteratively selecting a proper parameter set to improve the model classification recognition accuracy. The highest recognition rate reaches 89.06%, the recognition accuracy is improved by 3.38% compared with the conventional support vector machine, the recognition accuracy is improved by 0.78% compared with the least square support vector machine, and the applicability, accuracy and robustness of the support vector machine in classification of different road conditions are enhanced.
3. By training and testing the model, the invention fully verifies that the model can be applied to actual road condition identification.
4. The method adopts the least square support vector machine model to select the error square term as an optimization target, adopts the equality constraint as a constraint condition, and realizes a final decision function by solving the linear equation set, thereby reducing the solving difficulty to a certain extent, improving the solving speed and being more suitable for application in road condition identification.
Drawings
FIG. 1 is a technical roadmap of a road condition recognition method based on multi-source heterogeneous data fusion according to the present invention;
FIG. 2 is a diagram of eight road condition data to be identified in accordance with the present invention;
FIG. 3 is a schematic diagram of the classification recognition accuracy of the fusion feature matrix of the present invention;
FIG. 4 is a graph showing average recognition accuracy of road condition recognition models of different optimization algorithms according to the present invention.
FIG. 5 is a diagram of a confusion matrix of actual classification and predictive classification of the road condition recognition model of the present invention; wherein a is the actual classification and predictive classification map of the test set; b is a recognition accuracy confusion matrix diagram.
Detailed Description
The present invention will be described in detail below with reference to the drawings and the detailed description, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the detailed description is presented by way of example only and is not intended to limit the invention.
As shown in fig. 1, the invention provides a road surface type identification method based on multi-texture feature fusion of a road surface image, which specifically comprises the following steps:
and step 1, road surface perception information acquisition.
And acquiring pavement image data and vehicle dynamics related data under different pavement conditions by adopting an actual measurement experiment. Image data acquisition is carried out on roads (dry road surfaces, wet road surfaces, ponding road surfaces, rain and snow road surfaces, muddy snow road surfaces, compacted snow road surfaces, ice and snow road surfaces) with different conditions in different seasons by using the unmanned intelligent vehicle, wherein the road surfaces are all based on asphalt road surfaces. The unmanned intelligent vehicle is arranged to run along a straight line all the time and do reciprocating motion at a constant speed of 0-20 Km/h. The road image data is collected by using a vehicle-mounted camera, and the image frame rate of the collected image of the camera is set to be 100fps;
the vehicle dynamics related data comprise vehicle speed, acceleration and wheel speed (based on the left front wheel) in the running process of the vehicle, wherein the vehicle speed and the vehicle acceleration are collected by utilizing the integrated navigation system and the attitude angle sensor, and the wheel speed is collected by utilizing the Hall sensor. The acquisition frequency of each sensor was 100HZ. All data were collected using the ROS system and labeled using the same time stamp. Wherein the actual acquired data set is shown in fig. 2.
Step 2, data preprocessing and feature extraction: preprocessing the road surface image data and the vehicle dynamics related data acquired in the step 1, and extracting corresponding characteristic values by utilizing a plurality of characteristic extraction algorithms to form two characteristic matrixes for describing different road conditions. The method comprises the following specific steps:
and 2.1, preprocessing road image data and extracting features. Mainly comprises the following steps:
step 2.1.1, unifying the sizes of all the road image data to 256×256, and preprocessing the road image data by using USM and histogram equalization. To improve the contrast, brightness and texture feature definition of the picture.
And 2.1.2, respectively extracting color features and texture features of the road image by using a color histogram method, ULBP and GLCM. Wherein the extraction of the color features converts the picture into HSV color space, the hue H, the saturation S and the brightness V are utilized to represent the color, and finally 256-dimensional color feature vectors are extracted. The window size of each picture is set to be 32 multiplied by 32 in the texture features, 3776-dimensional ULBP texture feature vectors are finally extracted, the angular second-order distance of the gray level co-occurrence matrix in the four directions of deflection angles of 0 degree, 45 degree, 90 degree and 135 degree is calculated, five statistics of contrast, correlation, entropy and inverse distance are obtained, corresponding mean value, variance and standard deviation are obtained, and 15-dimensional GLCM texture feature vectors are finally extracted.
And 2.2, preprocessing vehicle dynamics data and extracting features. Mainly comprises the following steps:
and 2.2.1, preprocessing the acquired original signals of the vehicle speed, the acceleration and the wheel speed by utilizing a wavelet threshold method. The db6 wavelet is selected as a base wavelet, 4-layer wavelet decomposition is performed, and heuristic threshold and hard threshold functions are selected for denoising.
Step 2.2.2, converting the signal to a frequency domain and extracting the characteristics of the vehicle dynamics signal by using the power spectrum density. Wherein units of power spectral density are logarithmically converted such that units are converted to dB. The lower amplitude component is amplified and the periodic signal hidden in the low amplitude noise is more easily observed. And finally extracting 75-dimensional vehicle dynamics characteristic vectors.
Step 3, multi-source heterogeneous data fusion and dimension reduction: and (3) carrying out feature level fusion on the two feature matrixes describing different road conditions in the step (2) by utilizing a space-time fusion strategy, and carrying out dimension reduction treatment on the fused matrixes, wherein the method comprises the following specific steps of:
and 3.1, carrying out multi-source heterogeneous data feature fusion. Mainly comprises the following steps:
and 3.1.1, time data fusion. In the information acquisition process of the vehicle-mounted sensor, the camera shoots road image information in a certain range in front of the automobile, and the automobile dynamics data acquired by the vehicle-mounted sensor are response information of the automobile to contact marks on the road surface during running. Because the different sensors used in the invention have the same working frequency, the time data fusion is realized by only ensuring that the different sensors start to work at the same moment and have the same system time stamp, taking the starting point of the vehicle as the origin of coordinates, taking the output time of the combined navigation system as a standard, keeping the sampling frequency of the different sensors the same and requiring to start sampling at the same time, and recording the data in an offline mode, so that the acquired data has the same system time stamp;
and 3.1.2, spatial data fusion. First, driving mileage information of a vehicle is acquired, a vehicle position axis x is established, and the center of a front wheel of the vehicle is taken as an origin of coordinates. Next, the vehicle dynamics signal features are extracted and then formed into 6000-dimensional feature vectors and divided into 240 segments with equal length, and each image can be combined with the 25-dimensional vehicle dynamics feature vectors. Each segment of signal forms a characteristic vector of
Wherein the method comprises the steps ofRepresents a first type of road surface, f LA 、f WS F LS Indicating longitudinal acceleration, wheel speed and longitudinal direction
To a velocity characteristic value. And the final vehicle can be fused with vehicle dynamics data in the next 0.25 seconds after each road picture is acquired.
And 3.2, feature matrix dimension reduction. Mainly comprises the following steps:
and 3.2.1, normalizing and normalizing the feature matrix. And (3) carrying out standardization and normalization treatment on the fusion feature matrix by using a z-score method and a maximum normalization method so as to eliminate dimension and magnitude differences among different feature values.
And 3.2.2, performing feature matrix dimension reduction treatment. And (3) carrying out standardization and normalization processing on the fusion feature matrix, then reducing the dimension of the fusion feature matrix to 88 dimensions by using a PCA algorithm, reserving 75% of effective information, and providing the effective information for a least square support vector machine based on particle swarm optimization for classification recognition training.
Step 4, building a road condition identification model: and (3) inputting the feature matrix subjected to the dimension reduction in the step (3) into a least square support vector machine model based on particle swarm optimization for training and testing to obtain recognition results of different road conditions.
The method comprises the following specific steps:
and 4.1, respectively forming a dataset by the road image features, the vehicle dynamics features, the fusion feature matrix subjected to dimension reduction and the corresponding road type labels, and then randomly dividing the dataset into a training set and a testing set according to the ratio of 4:1.
And 4.2, improving the problem of the original support vector machine, introducing a particle swarm optimization algorithm to optimize the parameter set of the least square support vector machine, and establishing a road condition identification model. Mainly comprises the following steps:
step 4.2.1, in the road condition recognition model, due to the problem of a support vector machine, the constraint condition is inequality constraint:
s.t.y k [w T Φ(x k +b)]≥1-e k
wherein: w is a weight vector; c is a punishment coefficient which represents the influence degree of training errors on the objective function; e is a relaxation variable in the sense that outliers are introduced in the support vector; e, e k Not less than 0 and k=1, …, N, Φ (x) is a nonlinear mapping function; x is the input vector; b is the offset.
When the data size reaches a certain degree, the solving scale of the SVM algorithm can make some traditional methods difficult to adapt. And the least square support vector machine model selects an error square term as an optimization target, and adopts an equation constraint as a constraint condition to realize a final decision function by solving a linear equation set. The method reduces the solving difficulty to a certain extent, improves the solving speed, and is more suitable for application in road condition identification.
The original problem for the least squares support vector machine becomes an equality constraint:
s.t.y k [w T Φ(x k +b)]＝1-ξ k
wherein gamma is penalty factor, which has the same meaning as C in the previous formula; zeta type toy k Is an error vector. Solving the above by introducing a Lagrangian function, and bringing the calculation result to f (x) =w T In Φ (x) +b, the optimal classification function obtained by the arrangement is:
wherein: k (x) k X) is a kernel function, the invention adopts a Gaussian radial basis kernel function, and the mathematical expression is as follows:
k(x,x k )＝exp{-|x-x k | 2 /σ 2 }
where σ is the radial basis function width, which can affect the distribution of feature vectors in the new feature space after the feature vectors are mapped by the function.
Step 4.2.2, standard LSSVM solves the two classification problem. For the case of multiple classifications, the method of multi-objective optimization and combined coding is mainly adopted. The invention selects one-to-one coding classification method, constructs a plurality of classified LSSVMs and combines the classified LSSVMs with a combined coding method to realize multi-classification.
Step 4.2.3, parameter set (γ, σ) 2 ) The present invention selects particle swarm optimization algorithm to support the parameter set (gamma, sigma) in the vector machine for least squares 2 ) And (5) optimizing. The road condition recognition model modeling flow is as follows:
(1) Reading in a road condition data sample set fused with the dimension reduction;
(2) The PSO population is initialized, the number of particle populations is set to be 50, the iteration number is set to be 200, the maximum value of the inertia weight of the speed update is set to be 0.95, and the minimum value is set to be 0.6. And randomly generating a set (gamma, sigma) 2 ) As an initial position of the particles;
(3) According to the current (gamma, sigma 2 ) And performing LSSVM training, calculating a model fitting error under a given parameter, and taking the model fitting error as a fitness function. Wherein the fitness function is defined as:
E＝1-P i
wherein E is an error, P i Is the training accuracy under given parameters.
(4) Taking the model fitting error as an adaptive value, and updating the optimal adaptive value P corresponding to the particle and the population according to the adaptive value of the particle i k Andand according to
Updating the velocity V of each particle i k And positionThereby obtaining new LSSVM parameter values (gamma, sigma) 2 )；
(5) If the maximum iteration times are not met or the ending condition is met, returning to the step (3), otherwise, outputting the optimal parameter value;
(6) And re-training the LSSVM by using the finally obtained optimal parameters, and establishing a road condition identification model.
The invention uses the test set to carry out accuracy, tests the experimental result and analysis of the road condition recognition model, and mainly comprises the following steps:
and step 1, comparing the accuracy of classification recognition of the road type recognition method using the single feature and the fusion feature, wherein each feature uses an improved support vector machine as a machine learning classification model. The accuracy of the road type recognition model using different features on the test set is shown in fig. 3. The vehicle dynamic feature matrix and the image feature matrix have certain distinguishing property for classifying and identifying different road conditions, but the identification accuracy of single features is mostly lower than that of the feature matrix after fusion. The characteristics after fusion are used, so that the recognition accuracy of roads under different conditions is improved to a certain extent, and particularly, the recognition accuracy of the road is higher than 90% for the recognition of ponding road surfaces, asphalt dry road surfaces, muddy snow road surfaces, compacted snow road surfaces and rain and snow road surfaces. Therefore, the effectiveness of the multi-source heterogeneous data space-time fusion strategy in road condition classification is verified.
And 2, in order to select the machine learning classification model which is most suitable for road condition recognition, comparing the performances of five machine learning classification algorithms, namely a Support Vector Machine (SVM), a least squares-based support vector machine (LS-SVM), a particle swarm optimization-based support vector machine (PSO-SVM), a grid search optimization-based support vector machine (GS-SVM) and a particle swarm optimization-based least squares support vector machine (PSO-LSSVM), in terms of accuracy, and respectively performing training tests for 5 times or more on each model because the optimization algorithm has uncertainty. The average recognition accuracy of the road condition recognition models of different optimization algorithms is compared with that of a graph shown in fig. 4. The optimal recognition rate of the least square support vector machine based on particle swarm optimization reaches 89.06%, the recognition accuracy rate of the least square support vector machine is improved by 3.38% compared with that of the traditional support vector machine, and the recognition accuracy rate of the least square support vector machine is improved by 0.78% compared with that of the least square support vector machine, so that the model provided by the invention has excellent performance in road classification under different conditions.
And 3, an actual classification and prediction classification diagram and an identification accuracy confusion matrix diagram of the road condition identification model are shown in fig. 5. The image shows that the least square support vector machine based on particle swarm optimization has the least false detection quantity, and the identification accuracy rate of the least square support vector machine is higher than 90% for a ponding road surface, an asphalt dry road surface, a muddy snow road surface, a compacted snow road surface and a rain and snow road surface, so that the model is very suitable for road condition identification.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention. The components not explicitly described in this embodiment can be implemented by using the prior art.

Claims (6)

1. A road condition identification method based on multi-source heterogeneous data fusion is characterized by comprising the following steps of: the method comprises the following steps:
step 1), road surface perception information acquisition:
acquiring pavement image data and vehicle dynamics related data under different road conditions by using an unmanned intelligent vehicle;
step 2), data preprocessing and feature extraction:
preprocessing the pavement image data and the vehicle dynamics related data acquired in the step 1); extracting corresponding characteristic values by utilizing a plurality of characteristic extraction algorithms, and finally forming two characteristic matrixes for describing different road conditions;
step 3), multi-source heterogeneous data fusion and dimension reduction:
performing feature level fusion on the two feature matrixes describing different road conditions in the step 2) by utilizing a multi-source heterogeneous data space-time fusion strategy, and then performing dimension reduction on the fused feature matrixes by utilizing a principal component analysis method;
step 4), building a road condition identification model:
forming a data set by the feature matrix after the dimension reduction in the step 3) and the corresponding road type label; the data set is randomly divided into a training set and a testing set according to the ratio of 4:1; and finally, inputting the training set and the testing set into a least square support vector machine model based on particle swarm optimization for training and testing to obtain a road condition identification model.
2. The road condition identification method based on multi-source heterogeneous data fusion according to claim 1, wherein the method comprises the following steps: the vehicle dynamics related data in the step 1) comprise vehicle longitudinal speed, acceleration and wheel speed information.
3. The road condition identification method based on multi-source heterogeneous data fusion according to claim 1 or 2, wherein the method comprises the following steps: the method for preprocessing the pavement image data and extracting the characteristics in the step 2) comprises the following steps:
preprocessing road image data by using a USM and histogram equalization method; respectively extracting color features and texture features of the road image by using a color histogram method, ULBP and GLCM, and finally forming an image feature matrix;
the method for preprocessing the dynamics and extracting the characteristics of the vehicle comprises the following steps: preprocessing vehicle dynamics data by using a wavelet threshold denoising method; and extracting frequency domain features of the vehicle dynamics signals by using the power spectrum density, and finally forming a vehicle dynamics feature matrix.
4. The road condition identification method based on multi-source heterogeneous data fusion according to claim 3, wherein the specific method of the step 3) is as follows:
step 3.1) the multi-source heterogeneous data space-time fusion strategy comprises time data fusion and space data fusion:
time data fusion: taking a vehicle starting point as a coordinate origin, taking the output time of the integrated navigation system as a standard, keeping the sampling frequencies of different sensors the same and requiring to start sampling at the same time, and recording data in an offline mode to enable the acquired data to have a consistent system time stamp;
spatial data fusion: firstly, acquiring driving distance information of a vehicle and establishing a vehicle position axis x, taking the center of a front wheel of the vehicle as an origin of coordinates, secondly, dividing a vehicle dynamics feature vector into segments with equal length as image data, and then fusing each image feature vector with the segmented vehicle dynamics feature vector, wherein the feature vector formed by each segment of signals can be expressed as
Wherein the method comprises the steps ofRepresents a first type of road surface, f LA 、f WS F LS The longitudinal acceleration, the wheel speed and the longitudinal speed characteristic value are represented, and the vehicle dynamics data in 0.25 second after the final vehicle acquires a road picture are fused;
step 3.2) feature matrix dimension reduction: and (3) carrying out standardization and normalization processing on the fusion feature matrix, then reducing the dimension of the fusion feature matrix to 88 dimensions by using a PCA algorithm, reserving 75% of effective information, and providing the effective information for a least square support vector machine based on particle swarm optimization for classification recognition training.
5. The method for identifying road conditions based on multi-source heterogeneous data fusion according to claim 4, wherein the specific method in step 4 is as follows:
step 4.1, respectively forming a dataset by the road image features, the vehicle dynamics features, the fusion feature matrix subjected to dimension reduction and the corresponding road type labels, and then randomly dividing the dataset into a training set and a testing set according to the ratio of 4:1;
and 4.2, improving the problem of the original support vector machine, introducing a particle swarm optimization algorithm to optimize the parameter set of the least square support vector machine, and establishing a road condition identification model.
6. The road condition identification method based on multi-source heterogeneous data fusion according to claim 5, wherein the specific method of step 4.2 is as follows:
step 4.2.1, when the data scale reaches thousands or even tens of thousands, selecting an error square term as an optimization target by adopting a least square support vector machine model, taking equality constraint as a constraint condition, and realizing a final decision function by solving a linear equation set;
the Lagrangian function is introduced, and the least square support vector machine optimal classification function can be obtained through arrangement as follows:
wherein: k (x) k X) is a kernel function, a Gaussian radial basis kernel function is adopted, and the mathematical expression is as follows:
k(x,x k )＝exp{-|x-x k | 2 /σ 2 }
wherein sigma is the radial basis function width;
step 4.2.2, selecting a one-to-one coding classification method, and realizing multi-classification by constructing a plurality of classification LSSVMs and combining the classification LSSVMs with a combined coding method;
step 4.2.3 selecting particle swarm optimization algorithm to support the parameter set (γ, σ) in the vector machine for least squares 2 ) Optimizing; the road condition recognition model modeling flow is as follows:
(1) Reading in a road condition data sample set fused with the dimension reduction;
(2) Initializing PSO population, setting the number of particle population as 50, the iteration number as 200, setting the maximum value of inertial weight of speed update as 0.95 and the minimum value as 0.6, randomly generating a group (gamma,σ 2 ) As an initial position of the particles;
(3) According to the current (gamma, sigma 2 ) Performing LSSVM training, calculating model fitting error under given parameters, and taking the model fitting error as a fitness function, wherein the fitness function is defined as:
E＝1-P i
wherein E is an error, P i The training accuracy is given parameters;
(4) Using the model fitting error as an adaptive value, updating the optimal adaptive value sum corresponding to the particle and the population according to the adaptive value of the particle, and updating the speed and the position of each particle to obtain a new LSSVM parameter value (gamma, sigma) 2 )；
(5) If the maximum iteration times are not met or the ending condition is met, returning to the step (3), otherwise, outputting the optimal parameter value;
(6) And re-training the LSSVM by using the finally obtained optimal parameters, and establishing a road condition identification model.
CN202310754496.0A 2023-06-26 2023-06-26 Road condition identification method based on multi-source heterogeneous data fusion Pending CN116738211A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310754496.0A CN116738211A (en) 2023-06-26 2023-06-26 Road condition identification method based on multi-source heterogeneous data fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310754496.0A CN116738211A (en) 2023-06-26 2023-06-26 Road condition identification method based on multi-source heterogeneous data fusion

Publications (1)

Publication Number Publication Date
CN116738211A true CN116738211A (en) 2023-09-12

Family

ID=87913106

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310754496.0A Pending CN116738211A (en) 2023-06-26 2023-06-26 Road condition identification method based on multi-source heterogeneous data fusion

Country Status (1)

Country Link
CN (1) CN116738211A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117076872A (en) * 2023-10-17 2023-11-17 交通运输部公路科学研究所 Intelligent road equipment information acquisition testing method and system
CN117592004A (en) * 2024-01-19 2024-02-23 中国科学院空天信息创新研究院 PM2.5 concentration satellite monitoring method, device, equipment and medium
CN117852777A (en) * 2024-03-07 2024-04-09 北京华档致远科技有限公司 Linking method and system for multi-source heterogeneous data asset

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117076872A (en) * 2023-10-17 2023-11-17 交通运输部公路科学研究所 Intelligent road equipment information acquisition testing method and system
CN117076872B (en) * 2023-10-17 2023-12-26 交通运输部公路科学研究所 Intelligent road equipment information acquisition testing method and system
CN117592004A (en) * 2024-01-19 2024-02-23 中国科学院空天信息创新研究院 PM2.5 concentration satellite monitoring method, device, equipment and medium
CN117592004B (en) * 2024-01-19 2024-04-12 中国科学院空天信息创新研究院 PM2.5 concentration satellite monitoring method, device, equipment and medium
CN117852777A (en) * 2024-03-07 2024-04-09 北京华档致远科技有限公司 Linking method and system for multi-source heterogeneous data asset
CN117852777B (en) * 2024-03-07 2024-05-24 北京华档致远科技有限公司 Linking method and system for multi-source heterogeneous data asset

Similar Documents

Publication Publication Date Title
CN108983219B (en) Fusion method and system for image information and radar information of traffic scene
Roychowdhury et al. Machine learning models for road surface and friction estimation using front-camera images
CN107492251B (en) Driver identity recognition and driving state monitoring method based on machine learning and deep learning
CN116738211A (en) Road condition identification method based on multi-source heterogeneous data fusion
Lombacher et al. Semantic radar grids
Nguyen et al. Learning framework for robust obstacle detection, recognition, and tracking
CN109101924B (en) Machine learning-based road traffic sign identification method
Alhussan et al. Pothole and plain road classification using adaptive mutation dipper throated optimization and transfer learning for self driving cars
CN109050535B (en) Rapid terrain condition identification method based on vehicle attitude
CN111860274B (en) Traffic police command gesture recognition method based on head orientation and upper half skeleton characteristics
CN111723849A (en) Road adhesion coefficient online estimation method and system based on vehicle-mounted camera
CN114299002A (en) Intelligent detection system and method for abnormal road surface throwing behavior
CN106845458B (en) Rapid traffic sign detection method based on nuclear overrun learning machine
CN112115871B (en) High-low frequency interweaving edge characteristic enhancement method suitable for pedestrian target detection
Farag et al. Deep learning versus traditional methods for parking lots occupancy classification
Ye et al. Overhead ground wire detection by fusion global and local features and supervised learning method for a cable inspection robot
Yebes et al. Learning to automatically catch potholes in worldwide road scene images
Vaibhav et al. Real-time fog visibility range estimation for autonomous driving applications
CN114155720B (en) Vehicle detection and track prediction method for roadside laser radar
CN114235679B (en) Pavement adhesion coefficient estimation method and system based on laser radar
CN114332722A (en) Real-time estimation method for adhesion coefficient of mixed ice and snow road surface based on video data
CN114048536A (en) Road structure prediction and target detection method based on multitask neural network
CN114120150A (en) Road target detection method based on unmanned aerial vehicle imaging technology
xi Wang et al. Detection of MMW radar target based on Doppler characteristics and deep learning
CN115953660A (en) Point cloud 3D target detection method based on pseudo label and oriented to automatic driving

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination