CN110263844B - Method for online learning and real-time estimation of road surface state - Google Patents
Method for online learning and real-time estimation of road surface state Download PDFInfo
- Publication number
- CN110263844B CN110263844B CN201910526231.9A CN201910526231A CN110263844B CN 110263844 B CN110263844 B CN 110263844B CN 201910526231 A CN201910526231 A CN 201910526231A CN 110263844 B CN110263844 B CN 110263844B
- Authority
- CN
- China
- Prior art keywords
- road surface
- adhesion coefficient
- road
- estimation
- classifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/192—Recognition using electronic means using simultaneous comparisons or correlations of the image signals with a plurality of references
- G06V30/194—References adjustable by an adaptive method, e.g. learning
Abstract
The invention discloses a method for online learning and real-time estimation of a road surface state, which comprises three steps of classifier offline coarse training, online updating and real-time estimation, wherein the classifier offline coarse training comprises road surface image acquisition, image coarse labeling and classifier training, accurate road surface adhesion coefficients obtained according to dynamics are updated on line to correct a road surface adhesion coefficient estimation graph, an original image and the corrected road surface adhesion coefficient estimation graph are obtained for the online training of a classifier, and the classifier can estimate the accurate road surface adhesion coefficient graph in real time and is used for the active safety and intelligent function design of vehicles. The method can complete the estimation of the road surface adhesion coefficient by using a common vision sensor, has accurate estimation result, and provides a good foundation for the design of an active safety system and an intelligent driving system.
Description
Technical Field
The invention relates to the field of road surface state prediction and identification by machine vision, in particular to a method for online learning and real-time estimation of a road surface state.
Background
In the prior art, there are two main methods for estimating the road adhesion coefficient, one is a method for estimating the road adhesion coefficient based on the result, and the other is a method for estimating the road adhesion coefficient based on the cause. Although the result-based method is accurate and reliable in road adhesion coefficient estimation, the dynamic modeling is complex, and the real-time performance is difficult to guarantee; moreover, the dynamics-based method belongs to a result-based method, namely, the object is only contacted with the road surface to be estimated, and even if the estimation result is accurate, the timely intervention and control on some limit working conditions are difficult to generate. The method based on the reason has certain active predictability for road surface identification, can identify and estimate the road surface state before contact, but loses the commercial popularization value if the sensor is added too complicated or the cost is higher; secondly, the optical sensor has strict requirements on the working environment, ultrasonic waves, electromagnetic waves and the like are greatly influenced by the environment, the robustness is poor, and the problems that online learning cannot be realized exist, so that the accuracy is greatly influenced by a sample, and the performance of the method is obviously reduced on an untested road surface.
The above disadvantages need to be improved.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a method for online learning and real-time estimation of the road surface state.
The technical scheme of the invention is as follows:
a method for learning online and estimating road surface state in real time is characterized by comprising the following steps:
offline coarse training of a classifier: acquiring original images of different road surface working conditions through a vision sensor; marking the original image data to form a rough estimation data set of the road adhesion coefficient; end-to-end training is carried out on the rough classifier by utilizing the rough estimation data set of the road adhesion coefficient;
and (3) online updating: acquiring a first road surface image through the vision sensor, and inputting the first road surface image into the rough classifier to obtain a road surface adhesion coefficient estimation map; storing the original image and the road surface adhesion coefficient estimation map in a buffer area; positioning and tracking a vehicle through a vehicle-mounted sensor, and matching the wheel track of the vehicle with pixels in the road adhesion coefficient estimation map; obtaining an accurate first road adhesion coefficient according to vehicle dynamics, and correcting the adhesion coefficient of a pixel corresponding to the wheel track of the vehicle in the road adhesion coefficient estimation graph; and storing the corrected road adhesion coefficient estimation graph and the corresponding original image into a data pair, and performing online update training on the coarse classifier by using the data pair when the number of the data pair exceeds a preset value to obtain a fine classifier.
And (3) real-time estimation: acquiring a second road surface image through the vision sensor, and inputting the second road surface image to the fine classifier to obtain a road surface adhesion coefficient map in real time; positioning and tracking a vehicle through the vehicle-mounted sensor, and matching a wheel track of the vehicle with pixels in the road surface adhesion coefficient map; and obtaining an accurate second road adhesion coefficient in real time according to the vehicle dynamics.
The invention according to the above scheme is characterized in that the vision sensor is a monocular camera, a binocular camera or a stereoscopic 3D camera.
The invention according to the above aspect is characterized in that the coarse classifier is a decision tree, a random forest, a deep convolutional neural network, a combination of a deep convolutional neural network and a recurrent neural network and variants thereof, or a combination of a deep convolutional neural network and a conditional random field.
The invention according to the above scheme is characterized in that each pixel point of the road adhesion coefficient estimation graph is assigned with a road adhesion coefficient rough estimation result, and the result corresponds to a pixel at a position corresponding to the original image, so as to reflect the road surface state of the original image.
The invention according to the above scheme is characterized in that each pixel point of the road adhesion coefficient estimation map represents a roughly estimated road adhesion coefficient of a pixel at a corresponding position of the original image.
The invention according to the above aspect is characterized in that the road surface attachment coefficient estimation map may be acquired while the vehicle is stationary or moving.
The present invention according to the above aspect is characterized in that, when it is calculated that the wheel of the vehicle enters the range of the first road adhesion coefficient estimation map, the pixels of the road adhesion coefficient estimation map are associated and matched with the wheel track of the vehicle.
The present invention according to the above aspect is characterized in that the step S2 further includes: after the rough classifier is subjected to online updating training, emptying the original image stored in the buffer area and the road adhesion coefficient estimation graph corresponding to the original image, and acquiring data again for storage, wherein the updated rough classifier continues to generate the road adhesion coefficient estimation graph.
The invention according to the above aspect is characterized in that when the difference between the result of the road adhesion coefficient estimation graph and the result of the corrected road adhesion coefficient estimation graph is smaller than a preset value, the online update training is stopped, and the fine classifier is obtained.
The present invention according to the above aspect is characterized in that when the difference between the result of the road surface adhesion coefficient estimation map and the result of the vehicle dynamics estimation exceeds a preset value, the online update training is restarted.
The method can finish the estimation of the road adhesion coefficient by using a common vision sensor, and has simple installation and convenient realization; the method is an active prediction type, can accurately estimate the road surface adhesion coefficient in real time before the tire is in contact with the road surface, and provides a good basis for the design of an active safety system and an intelligent driving system.
Drawings
FIG. 1 is a flow chart of a method for learning online and estimating road surface conditions in real time according to an embodiment of the present invention;
FIG. 2 is a first vision sensor arrangement in accordance with an embodiment of the present invention;
FIG. 3 is a second vision sensor arrangement in accordance with an embodiment of the present invention;
FIG. 4 is a third vision sensor arrangement in accordance with an embodiment of the present invention;
FIG. 5 is a block diagram of a coarse training classifier according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating the structure of a classifier in the online update real-time estimation stage according to an embodiment of the present invention;
FIG. 7 illustrates the process of the coarse classifier and the modified fine classifier providing training data during the online update phase according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and embodiments:
as shown in fig. 1 to 7, a method of learning online and estimating a road surface state in real time includes the steps of:
s1: offline coarse training of a classifier: acquiring original images of different road surface working conditions through a vision sensor; marking original image data to form a rough estimation data set of a road adhesion coefficient; end-to-end training is carried out on the coarse classifier by utilizing the rough estimation data set of the road adhesion coefficient;
s2: and (3) online updating: acquiring a first road surface image through a vision sensor, and inputting the first road surface image into a rough classifier to obtain a road surface adhesion coefficient estimation graph; storing the original image and the road surface adhesion coefficient estimation map in a buffer area; positioning and tracking the vehicle through a vehicle-mounted sensor, and matching the wheel track of the vehicle with pixels in a road adhesion coefficient estimation map; obtaining an accurate first road adhesion coefficient according to vehicle dynamics, and correcting the adhesion coefficient of a pixel corresponding to a wheel track of the vehicle in a road adhesion coefficient estimation graph; and storing the corrected road adhesion coefficient estimation graph and the corresponding original image into a data pair, and performing online updating training on the coarse classifier by using the data pair when the number of the data pair exceeds a preset value to obtain a fine classifier.
S3: and (3) real-time estimation: acquiring a second road surface image through a vision sensor, and inputting the second road surface image into a fine classifier to obtain a road surface adhesion coefficient map in real time; positioning and tracking the vehicle through the vehicle-mounted sensor, and matching the wheel track of the vehicle with pixels in the road surface adhesion coefficient map; and obtaining an accurate second road surface adhesion coefficient in real time according to the vehicle dynamics.
Preferably, step S2 further includes: after the rough classifier is updated and trained on line, the original image stored in the buffer area and the corresponding road adhesion coefficient estimation image are emptied, data are collected again and stored, and the updated rough classifier continues to generate the road adhesion coefficient estimation image. And when the difference between the result of the road adhesion coefficient estimation graph and the result of the corrected road adhesion coefficient estimation graph is smaller than a preset value, stopping on-line updating training and obtaining the fine classifier. And when the difference between the result of the road adhesion coefficient estimation graph and the result of the vehicle dynamics estimation exceeds a preset value, the online updating training is restarted.
Preferably, the vision sensor is a monocular camera, a binocular camera or a stereoscopic 3D camera, the invention adopts the binocular camera as the vision sensing equipment, and the binocular camera is flexible in installation and processing, low in price and capable of being used for estimating depth information.
Preferably, the coarse classifier is a decision tree, a random forest, a deep convolutional neural network, a combination of a deep convolutional neural network and a recurrent neural network and variants thereof, or a combination of a deep convolutional neural network and a conditional random field.
Preferably, each pixel point of the road adhesion coefficient estimation map is assigned with a road adhesion coefficient rough estimation result, and the result corresponds to a pixel at a position corresponding to the original image, so that the road surface state of the original image is reflected.
Preferably, each pixel point of the road adhesion coefficient estimation map represents a roughly estimated road adhesion coefficient of a pixel at a corresponding position of the original image.
Preferably, the road surface adhesion coefficient estimation map may be acquired while the vehicle is stationary or moving.
Preferably, when it is calculated that the wheel of the vehicle enters the range of the first road adhesion coefficient estimation map, the pixels of the road adhesion coefficient estimation map are associated and matched with the wheel track of the vehicle.
Fig. 2-4 show three arrangements of vision sensors disposed on a vehicle. Fig. 2 is a scheme that a first vision sensor is arranged on a vehicle, and the first vision sensor is arranged behind a front windshield, and the position has the advantages of convenience in installation, wide estimation range of sight distance, difficulty in being influenced by weather and vibration and the like. Fig. 3 shows a second embodiment of the vision sensor arranged on the vehicle, which is installed at the air intake grid, and has the advantage of providing sufficient road adhesion preview range for subsequent active control and intelligent functions, with a medium line-of-sight in the installation position. Fig. 4 is a scheme of arranging a third vision sensor on a vehicle, and a scheme of installing the vision sensor near a tire, and the scheme has the main advantages that the acquired image is an image of the tire immediately contacting the road surface, the image classification and labeling are more accurate, the influence of the prediction error of the vehicle traveling direction is small, the vision sensor can be used immediately once the coarse classifier training is completed, and the complex matching work in the road surface adhesion coefficient map through the predicted wheel track is not needed. Preferably, the invention adopts a first scheme to collect pictures under various road conditions, including road scenes of structured and unstructured roads under various weather and lighting conditions. In the stage of acquiring the road working condition pictures, the road types, illumination and weather conditions and road structures contained in the data are mainly ensured to be as diverse as possible, wherein the road types include but are not limited to dry asphalt, wet concrete pavement, cobblestone pavement, muddy and slippery pavement, ice pavement, snow pavement and the like; weather conditions include, but are not limited to, sunny weather, rainy weather, heavy rainy weather, snowy weather, and the like; the lighting conditions include morning, noon, dusk, night, etc.
According to the collected image, different types of rough labeling are carried out on all pixels in the image data, and the type true value and the road surface adhesion coefficient corresponding to each road condition during labeling are shown in table 1. Wherein, 1-7 are effective pavement marks, and 0 is marks of other non-pavement areas.
TABLE 1
| Class truth value | Type of road | Road surface adhesion coefficient mu |
| 0 | Non-road surface area | μ＝0 |
| 1 | Dry asphalt | μ＝0.9 |
| 2 | Wet asphalt | μ＝0.7 |
| 3 | Wet concrete pavement | μ＝0.6 |
| 4 | Cobblestone pavement | μ＝0.5 |
| 5 | Muddy and slippery road surface | μ＝0.3 |
| 6 | Snow road surface | μ＝0.2 |
| 7 | Ice road surface | μ＝0.1 |
As can be seen from table 1, for each road type, a class label and a rough road adhesion coefficient are given. The main influencing factors of the tire-road adhesion coefficient include a road type, a road state, tire parameters and vehicle parameters, wherein the vehicle parameters can be directly obtained, but the road type and the road state are obtained by certain sensing equipment measurement, so that the road adhesion coefficient directly identified by vision is a rough estimation value based on prior knowledge. In order to directly correspond to the rough road surface assistance coefficient, the road surface adhesion coefficient of each road surface is not represented by a range, but is directly assigned a numerical value, and therefore, the error thereof is approximately within ± 0.1.
Inputting the marked image and a truth label (as shown in the road surface images in fig. 2 to fig. 4) into a coarse classifier together for end-to-end training, wherein the type of the coarse classifier can be selected from but not limited to a decision tree, a random forest, a deep convolutional neural network, a combination of the deep convolutional neural network and a recurrent neural network and a variant thereof, and a combination of the deep convolutional neural network and a conditional random field; the training can be selected from, but not limited to, machine learning frameworks such as Tensorflow, Pytrch, Caffe and the like, and the training method adopts a back-propagation random gradient descent method.
Preferably, the present invention employs a deep convolutional segmentation network of the encoder-decoder type as a coarse classifier to illustrate the classifier training portion. As shown in fig. 5, a coarse training network structure according to an embodiment of the present invention is configured in such a way that a first half portion extracts feature information from detail to the global from an image in a layer-by-layer convolution superposition manner, an encoder is a network with 15 layers, the number of network feature maps is [3,16,16,16,32,32,32,64,64,64,128, 256], respectively, in the layer-by-layer feature extraction process, each three layers are down-sampled, and feature maps are continuously reduced. In the convolutional neural network, a batch normalization layer, a pooling layer, an activation layer, and the like are also included. And in the latter decoder part, performing feature recovery on the reduced feature map, namely recovering the reduced feature map to the original image size and classifying the road type category of each pixel. In the process of recovering the characteristic diagram, a mode of directly copying the shallow network characteristic diagram and combining the upper sampling characteristic diagram is adopted to recover the information, and the convergence is accelerated.
After the offline training of the coarse classifier is completed, an online updating stage can be started; the online update phase process is shown in fig. 5. To better describe the online update process, the objects represented by all reference numerals in FIG. 7 are illustrated. 1 is a vehicle position at time T, 2 is an in-vehicle camera which is a vision sensor, 3 is a road surface image acquired by the camera at time T, 4 is a road surface adhesion coefficient estimation map roughly classified according to the road surface image at time T, 5 is a vehicle position at time T + Δ T, 6 is positions of four tires of the vehicle at time T + Δ T, 7 is a road surface image acquired by the camera at time T + Δ T, and 8 is a road surface adhesion coefficient estimation map roughly classified according to the road surface image at time T + Δ T; 9 is a correction map of a roughly classified road surface adhesion coefficient estimation map obtained from T + Δ T, in which four rectangles of different colors represent the accurate road surface adhesion coefficients at each position estimated from the four tire tracks, respectively; it should be noted that the present embodiment assumes that the vehicle travels along a straight line for easy understanding, and therefore a rectangular area is obtained, but in practice the vehicle tire trajectory cannot be a perfect rectangle, but the principle is consistent; the rectangle can be obtained through an odometer, a map and GPS positioning and other forms; the mapping relationship between the tire track and the picture pixel can be obtained by mapping the pixel from the image coordinate system to the vehicle coordinate system through Inverse perspective projection (IPM) of the image, and then mapping the pixel back to the image coordinate system after the corresponding relationship is completed. And 10 is a time boundary line of T ═ T and T ═ T + Δ T. Thus, a pair of data pairs of the original image 3 acquired with T ═ T and the road surface adhesion coefficient estimation map 9 after correction is formed and stored in the buffer.
The trained coarse classifier receives the image of the camera in the driving process as input and outputs an eight-dimensional vector which is equivalent to the length and width of the image. Assuming that the length and width of the input image are [ l, w ], the output of the trained network prediction is [ l, w,8], wherein for a pixel [ i, j ] at any position in the input image, the output [ i, j, k ] is obtained, and k belongs to [1,8], namely the probability that the pixel [ i, j ] belongs to each class of 8 classes of road surfaces. The maximum value of the eight-dimensional vector is taken in the height direction, and the index value of the class corresponding to the maximum value is taken as the road type to which the pixel belongs. After the classification of each pixel in the input image is determined, the road surface adhesion coefficient corresponding to each pixel point can be determined according to table 1.
And according to the content, continuously iterating to finish the acquisition and matching of the online correction marking data. And (3) on the assumption that 5000 data pairs are stored, updating the training coarse classifier on line to obtain a fine classifier with higher precision. As shown in fig. 6, comparing fig. 5 and fig. 6, it can be found that the two are only different in the number of characteristic patterns of the output layer. In the rough classifier, only rough division of the road surface into 8 classes is set, so that the labeling load can be reduced, and the convergence rate can be increased. In the fine classifier, the output is set to 1000 classes. Since the road surface adhesion coefficient is a number between 0 and 1, 0-999 classes represent adhesion coefficients of 0.000-0.999, respectively, and the correspondence between classes and road surface adhesion is shown in table 2:
watch two
Because the corresponding relation between the classification and the road surface adhesion coefficient value is different when the rough classifier and the fine classifier are trained, if 1 corresponds to a dry asphalt road surface during rough segmentation, the adhesion coefficient is set to be 0.9, and the rough classifier and the fine classifier need to be subjected to re-correspondence; the pixel with the coarse classifier category of 1 and the attachment coefficient of 0.9 is reassigned to have the category true value of 900, and other pixel points follow the same operation.
The road surface adhesion coefficient map (corresponding to reference numeral 9 in fig. 7) and the original image thereof (corresponding to reference numeral 3 in fig. 7) after the adjustment of the true value of the class is completed are input to the fine classifier shown in fig. 6 for training. Because the coarse classifier and the fine classifier have the same structure in the encoder part and the encoder part is mainly used for extracting image features, the encoder weight trained by the coarse segmentation network can be directly copied to the fine segmentation network for transfer learning, and thus the training can be accelerated to a great extent.
After the fine classifier is trained, precision verification is required. The checking step is as follows: continuously predicting the road adhesion coefficient map of 200 road surface images by using a fine classifier, and correcting the road adhesion coefficient estimation map according to vehicle dynamics, namely the step of on-line updating training of FIG. 5 to obtain a corrected map; comparing the difference between the road adhesion coefficient estimation graph and the correction graph, taking the average error or mean square deviation as an evaluation index, and stopping the on-line updating training process when the index is smaller than a set threshold; if the error is large, the classifier is considered not to obtain the accuracy of the dynamic estimation, the buffer area used for storing the original image and the corresponding attachment map is emptied, and the online learning process of image acquisition, rough segmentation result correction, model training and verification is continuously carried out.
After the fine classifier passes the verification, the road surface state can be estimated, the accuracy of the estimation result is consistent with the vehicle dynamics estimation, real-time estimation can be realized, the estimation effect has certain pre-aiming performance, and accurate road surface adhesion coefficient information can be obtained before the wheels are in contact with the road surface. After the fine estimation classifier is obtained, regular verification can still be carried out on the fine estimation classifier, and the verification period can be 1 day or 1 week; if the road condition changes greatly, the online training process can be started artificially.
It will be understood that modifications and variations can be made by persons skilled in the art in light of the above teachings and all such modifications and variations are intended to be included within the scope of the invention as defined in the appended claims.
The invention is described above with reference to the accompanying drawings, which are illustrative, and it is obvious that the implementation of the invention is not limited in the above manner, and it is within the scope of the invention to adopt various modifications of the inventive method concept and technical solution, or to apply the inventive concept and technical solution to other fields without modification.
Claims (10)
1. A method for learning online and estimating road surface state in real time is characterized by comprising the following steps:
offline coarse training of a classifier: acquiring original images of different road surface working conditions through a vision sensor; marking the original image data to form a rough estimation data set of the road adhesion coefficient; end-to-end training is carried out on the rough classifier by utilizing the rough estimation data set of the road adhesion coefficient;
and (3) online updating: acquiring a first road surface image through the vision sensor, and inputting the first road surface image into the rough classifier to obtain a road surface adhesion coefficient estimation map; storing the original image and the road surface adhesion coefficient estimation map in a buffer area; positioning and tracking a vehicle through a vehicle-mounted sensor, and matching the wheel track of the vehicle with pixels in the road adhesion coefficient estimation map; obtaining an accurate first road adhesion coefficient according to vehicle dynamics, and correcting the adhesion coefficient of a pixel corresponding to the wheel track of the vehicle in the road adhesion coefficient estimation graph; storing the corrected road adhesion coefficient estimation graph and the corresponding original image into a data pair, and performing online update training on the coarse classifier by using the data pair when the number of the data pair exceeds a preset value to obtain a fine classifier;
estimating time: acquiring a second road surface image through the vision sensor, and inputting the second road surface image to the fine classifier to obtain a road surface adhesion coefficient map in real time; positioning and tracking a vehicle through the vehicle-mounted sensor, and matching a wheel track of the vehicle with pixels in the road surface adhesion coefficient map; and obtaining an accurate second road adhesion coefficient in real time according to the vehicle dynamics.
2. The method of learning and estimating road surface conditions in real time on-line as claimed in claim 1, wherein the vision sensor is a monocular camera, a binocular camera or a stereo 3D camera.
3. The method of online learning and real-time estimation of road surface conditions according to claim 1, characterized in that the coarse classifier is a decision tree, a random forest, a deep convolutional neural network, a combination of a deep convolutional neural network and a recurrent neural network and variants thereof, or a combination of a deep convolutional neural network and a conditional random field.
4. The method for learning on-line and estimating a road surface condition in real time according to claim 1, wherein a rough road surface adhesion coefficient estimation result is assigned to each pixel point of the road surface adhesion coefficient estimation map, and the rough road surface adhesion coefficient estimation result corresponds to a pixel at a position corresponding to the original image, thereby reflecting the road surface condition of the original image.
5. The method for on-line learning and real-time estimation of road surface condition according to claim 1, wherein each pixel point of the road surface adhesion coefficient estimation map represents a rough estimated road surface adhesion coefficient of a pixel at a corresponding position of the original image.
6. The method of learning online and estimating road surface conditions in real time according to claim 1, characterized in that the road surface adhesion coefficient estimation map is acquired while the vehicle is stationary or moving.
7. The method of learning online and estimating a road surface condition in real time according to claim 1, characterized in that, when it is calculated that the wheels of the vehicle come within the range of the first one of the road adhesion coefficient estimation maps, the pixels of the road adhesion coefficient estimation map are associated and matched with the wheel track of the vehicle.
8. The method for learning and estimating road surface conditions in real time according to claim 1, wherein the step S2 further comprises: after the rough classifier is subjected to online updating training, emptying the original image stored in the buffer area and the road adhesion coefficient estimation graph corresponding to the original image, and acquiring data again for storage, wherein the updated rough classifier continues to generate the road adhesion coefficient estimation graph.
9. The method of claim 8, wherein when the difference between the result of the road adhesion coefficient estimation map and the result of the corrected road adhesion coefficient estimation map is less than a preset value, online update training is stopped and the fine classifier is obtained.
10. The method of online learning and real-time estimation of a road surface condition according to claim 9, characterized in that when the result of the road surface adhesion coefficient estimation map and the result of the vehicle dynamics estimation differ beyond a preset value, online update training is resumed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910526231.9A CN110263844B (en) | 2019-06-18 | 2019-06-18 | Method for online learning and real-time estimation of road surface state |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910526231.9A CN110263844B (en) | 2019-06-18 | 2019-06-18 | Method for online learning and real-time estimation of road surface state |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110263844A CN110263844A (en) | 2019-09-20 |
| CN110263844B true CN110263844B (en) | 2021-04-06 |
Family
ID=67918904
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910526231.9A Active CN110263844B (en) | 2019-06-18 | 2019-06-18 | Method for online learning and real-time estimation of road surface state |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110263844B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539474B (en) * | 2020-04-23 | 2022-05-10 | 大连理工大学 | Classifier model transfer learning method |
| CN111845709B (en) * | 2020-07-17 | 2021-09-10 | 燕山大学 | Road adhesion coefficient estimation method and system based on multi-information fusion |
| CN111860322B (en) * | 2020-07-20 | 2022-10-11 | 吉林大学 | Unstructured pavement type identification method based on multi-source sensor information fusion |
| CN112329533A (en) * | 2020-10-07 | 2021-02-05 | 江苏大学 | Local pavement adhesion coefficient estimation method based on image segmentation |
| CN112863247A (en) * | 2020-12-30 | 2021-05-28 | 潍柴动力股份有限公司 | Road identification method, device, equipment and storage medium |
| CN113985883A (en) * | 2021-11-01 | 2022-01-28 | 吉林大学 | Energy-saving, safe and cargo-comfortable control system based on heterogeneous truck fleet |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1758043A (en) * | 2005-11-03 | 2006-04-12 | 重庆邮电学院 | Cargo vehicle ABS road identification method |
| CN104354697A (en) * | 2014-10-08 | 2015-02-18 | 南京航空航天大学 | Method for estimating road adhesion coefficient according to on-line modified automobile state parameter |
| CN105184262A (en) * | 2014-06-12 | 2015-12-23 | 通用汽车环球科技运作有限责任公司 | Road Surface Condition Detection With Recursive Adaptive Learning And Validation |
| CN107406079A (en) * | 2013-10-17 | 2017-11-28 | 费泽姆股份有限公司 | For the system and method for the weather performance for predicting vehicle |
| CN107491736A (en) * | 2017-07-20 | 2017-12-19 | 重庆邮电大学 | A kind of pavement adhesion factor identifying method based on convolutional neural networks |
| CN108664715A (en) * | 2018-04-26 | 2018-10-16 | 长安大学 | A kind of surface gathered water track triple assessment and traffic safety analysis method |
| CN108715166A (en) * | 2018-04-28 | 2018-10-30 | 南京航空航天大学 | Intact stability index method of estimation based on deep learning |
| CN108985158A (en) * | 2018-06-08 | 2018-12-11 | 汉腾汽车有限公司 | A kind of road identification system and method |
| CN109466558A (en) * | 2018-10-26 | 2019-03-15 | 重庆邮电大学 | A kind of coefficient of road adhesion estimation method based on EKF and BP neural network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103605960B (en) * | 2013-11-15 | 2016-09-28 | 长安大学 | A kind of method for identifying traffic status merged based on different focal video image |
| JP6408852B2 (en) * | 2014-10-06 | 2018-10-17 | 株式会社ブリヂストン | Road surface classification system |
-
2019
- 2019-06-18 CN CN201910526231.9A patent/CN110263844B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1758043A (en) * | 2005-11-03 | 2006-04-12 | 重庆邮电学院 | Cargo vehicle ABS road identification method |
| CN107406079A (en) * | 2013-10-17 | 2017-11-28 | 费泽姆股份有限公司 | For the system and method for the weather performance for predicting vehicle |
| CN105184262A (en) * | 2014-06-12 | 2015-12-23 | 通用汽车环球科技运作有限责任公司 | Road Surface Condition Detection With Recursive Adaptive Learning And Validation |
| CN104354697A (en) * | 2014-10-08 | 2015-02-18 | 南京航空航天大学 | Method for estimating road adhesion coefficient according to on-line modified automobile state parameter |
| CN107491736A (en) * | 2017-07-20 | 2017-12-19 | 重庆邮电大学 | A kind of pavement adhesion factor identifying method based on convolutional neural networks |
| CN108664715A (en) * | 2018-04-26 | 2018-10-16 | 长安大学 | A kind of surface gathered water track triple assessment and traffic safety analysis method |
| CN108715166A (en) * | 2018-04-28 | 2018-10-30 | 南京航空航天大学 | Intact stability index method of estimation based on deep learning |
| CN108985158A (en) * | 2018-06-08 | 2018-12-11 | 汉腾汽车有限公司 | A kind of road identification system and method |
| CN109466558A (en) * | 2018-10-26 | 2019-03-15 | 重庆邮电大学 | A kind of coefficient of road adhesion estimation method based on EKF and BP neural network |
Non-Patent Citations (7)
| Title |
|---|
| Deep Denoising Autoencoding Method for Feature Extraction and Recognition of Vehicle Adhesion Status;Jing He等;《Journal of Sensors》;20180916;1-8 * |
| Model-based road friction estimation;Shim Taehyun等;《Vehicle System Dynamics》;20040430;第41卷(第4期);249-276 * |
| Research of Electric Power Steering System Assistance Characteritic Based on The Identification of The Road;Qiang Li等;《ICCIA 2012》;20121209;0402-0405 * |
| Road surface recognition sensor using an optical spatial filter;Yoda, Shigeru等;《Proceedings of the 1995 Intelligent Vehicles Symposium》;19951231;1000-1004 * |
| 基于深度学习的极限工况下车辆的状态估计;张凤娇等;《重庆理工大学学报(自然科学)》;20181015;第32卷(第10期);64-70 * |
| 基于神经网络的附着系数测算模型研究;王艳芳等;《拖拉机与农用运输车》;20110415;第38卷(第2期);36-40 * |
| 路面附着系数识别方法发展现状综述及展望;袁朝春等;《机械制造与自动化》;20180420;1-4、7 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110263844A (en) | 2019-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110263844B (en) | Method for online learning and real-time estimation of road surface state | |
| CN111862672B (en) | Parking lot vehicle self-positioning and map construction method based on top view | |
| CN111448478B (en) | System and method for correcting high-definition maps based on obstacle detection | |
| CN111551958A (en) | Mining area unmanned high-precision map manufacturing method | |
| US10552982B2 (en) | Method for automatically establishing extrinsic parameters of a camera of a vehicle | |
| CN102208035B (en) | Image processing system and position measuring system | |
| JP6301828B2 (en) | Apparatus for measuring the speed and position of a vehicle moving along a guiding track, and corresponding method and computer program product | |
| CN102398598B (en) | Use the Lane Fusion system of forward sight and rear view camera | |
| CN113361121B (en) | Road adhesion coefficient estimation method based on time-space synchronization and information fusion | |
| US20140348390A1 (en) | Method and apparatus for detecting traffic monitoring video | |
| CN111006655A (en) | Multi-scene autonomous navigation positioning method for airport inspection robot | |
| CN111860322A (en) | Unstructured pavement type identification method based on multi-source sensor information fusion | |
| CN112674646B (en) | Self-adaptive welting operation method based on multi-algorithm fusion and robot | |
| CN114475573B (en) | Fluctuating road condition identification and vehicle control method based on V2X and vision fusion | |
| CN110550041B (en) | Road adhesion coefficient estimation method based on cloud data sharing | |
| CN109753841A (en) | Lane detection method and apparatus | |
| CN114399748A (en) | Agricultural machinery real-time path correction method based on visual lane detection | |
| CN115923839A (en) | Vehicle path planning method | |
| CN115564865A (en) | Construction method and system of crowdsourcing high-precision map, electronic equipment and vehicle | |
| Hara et al. | Vehicle localization based on the detection of line segments from multi-camera images | |
| CN209214563U (en) | Multi-information fusion spectrum of road surface roughness real-time testing system | |
| Jeong et al. | Real-time lane detection for autonomous vehicle | |
| CN113029185B (en) | Road marking change detection method and system in crowdsourcing type high-precision map updating | |
| WO2021063756A1 (en) | Improved trajectory estimation based on ground truth | |
| KR102368262B1 (en) | Method for estimating traffic light arrangement information using multiple observation information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |