CN112329533A - Local pavement adhesion coefficient estimation method based on image segmentation - Google Patents
Local pavement adhesion coefficient estimation method based on image segmentation Download PDFInfo
- Publication number
- CN112329533A CN112329533A CN202011067813.4A CN202011067813A CN112329533A CN 112329533 A CN112329533 A CN 112329533A CN 202011067813 A CN202011067813 A CN 202011067813A CN 112329533 A CN112329533 A CN 112329533A
- Authority
- CN
- China
- Prior art keywords
- road surface
- image
- local
- road
- adhesion coefficient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000003709 image segmentation Methods 0.000 title claims abstract description 31
- 238000013135 deep learning Methods 0.000 claims abstract description 20
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 17
- 238000009825 accumulation Methods 0.000 claims description 14
- 239000010426 asphalt Substances 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 2
- 239000013598 vector Substances 0.000 claims description 2
- 238000003780 insertion Methods 0.000 claims 1
- 230000037431 insertion Effects 0.000 claims 1
- 238000002372 labelling Methods 0.000 abstract description 5
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a local pavement adhesion coefficient estimation method based on image segmentation, which comprises the following steps of 1: the offline pre-training image segmentation network specifically comprises: a. the method comprises the steps of collecting road surface images under different weather conditions by using CARLA software, b, carrying out local labeling on the collected road surface images under different weather conditions to form a data set of local road surface adhesion coefficient estimation, c, building a deep learning algorithm network model of image segmentation by using a deep learning means, and d, carrying out end-to-end training on a deep learning algorithm network framework of image segmentation by using the data set of local road surface adhesion coefficient estimation. Step 2: the method comprises the following steps of acquiring a real-time road surface image, and estimating a local adhesion coefficient of the road surface in real time, wherein the method specifically comprises the following steps: a. acquiring real-time road surface images by using a vehicle-mounted camera, classifying the real-time acquired images by using a pre-trained image segmentation network and positioning different types to form a real-time road condition map, and c, estimating local road surface adhesion coefficients of the real-time road condition map according to the road surface type.
Description
Technical Field
The invention belongs to the field of image segmentation, and particularly relates to a local pavement adhesion coefficient estimation method based on image segmentation.
Background
Better estimation of road adhesion coefficient has always been a very challenging problem. The road adhesion coefficient not only influences the dynamic property and the braking performance of the vehicle, but also influences the operation stability of the vehicle during running, accurately distinguishes the road adhesion coefficient in real time, and can greatly improve the safety and the comfort of the vehicle during running. With the continuous advancement of the industry towards intellectualization, the accurate estimation of the road adhesion coefficient can also greatly influence the path planning and decision of systems such as intelligent vehicles and robots. Therefore, the road surface adhesion coefficient is accurately estimated in high real-time, the driving safety of the vehicle can be greatly improved, and meanwhile, the planning and decision accuracy of the intelligent system can be improved.
At present, the estimation methods of the road adhesion coefficient mainly include three types, one type is traditional indirect estimation based on vehicle dynamic parameter identification, the second type is to acquire road surface data through sensors (sound, light, temperature sensors and the like) and estimate according to the relationship between the sensor data and the road adhesion coefficient, and the third type is to directly estimate the road adhesion coefficient based on the road surface of a vision sensor by means of deep learning.
Although the road adhesion coefficient estimation by adopting the dynamic modeling method is accurate and reliable, the model is complex, so that a plurality of vehicle dynamic parameters need to be acquired, and the instantaneity cannot be ensured; in addition, the method based on dynamic estimation requires the contact between the vehicle tire and the road surface for estimation, so that only the adhesion coefficient of the road surface in contact with the tire at present can be estimated, the road surface adhesion coefficient prediction cannot be carried out on the road surface which is about to pass through in the future, and the vehicle cannot be intervened and controlled in time. Although the vision-based method based on the deep learning means can make up for the defects of the dynamic method and has a certain advance predictability, the method only estimates the adhesion coefficient of the whole road surface and does not estimate the adhesion coefficient of the local road surface, such as local water accumulation, local snow accumulation, local icing and the like.
Disclosure of Invention
In order to overcome the defects of the existing road adhesion coefficient estimation method, the invention provides a local road adhesion coefficient estimation method based on image segmentation.
The technical steps of the invention are as follows:
step 1: the offline pre-training image segmentation network specifically comprises: a. the method comprises the steps of collecting road surface images under different weather conditions by using CARLA software, b, carrying out local labeling on the collected road surface images under different weather conditions to form a data set of local road surface adhesion coefficient estimation, c, building a deep learning algorithm network model of image segmentation by using a deep learning means, and d, carrying out end-to-end training on a deep learning algorithm network framework of image segmentation by using the data set of local road surface adhesion coefficient estimation.
Step 2: the method comprises the following steps of acquiring a real-time road surface image, and estimating a local adhesion coefficient of the road surface in real time, wherein the method specifically comprises the following steps: a. acquiring real-time road surface images by using a vehicle-mounted camera, classifying the real-time acquired images by using a pre-trained image segmentation network and positioning different types to form a real-time road condition map, and c, estimating local road surface adhesion coefficients of the real-time road condition map according to the road surface type.
Further, in the step 1a, road surface images of different weather conditions are collected, wherein the collection tool is CARLA simulation software, and the collected images include a local water accumulation image of the road surface, a local snow accumulation image of the road surface, a local icing image of the road surface, a normal asphalt road surface and the like.
Further, the deep learning algorithm network model for image segmentation comprises a basic network structure built by utilizing a residual error structure, a highly-driven attention module is added on the basic network, and the semantic segmentation neural network model is named as H-ResNet, wherein H represents the highly-driven attention module, and ResNet represents the basic network structure built by the residual error structure.
Further, the deep learning algorithm network model is a semantic segmentation algorithm network built by using a TensorFlow or Keras or Caffe2 or PyTorch or MXNet deep learning framework.
Further, the training method is a back propagation method of Batch Gradient Descent (Batch Gradient Descent) or random Gradient Descent (Stochastic Gradient Descent) or small-Batch Gradient Descent (Mini-Batch Gradient Descent) by using a single GPU or multiple GPUs.
Further, the vehicle-mounted camera in step 2a is a web camera or a USB camera.
Further, in the step 2b, the obtained images are classified and positioned into different categories, wherein the classification and the positioning of the different categories refer to that the dry asphalt is used as a background road surface, and water accumulation areas, snow accumulation areas, icing areas and the like or random combinations of any of the water accumulation areas, the snow accumulation areas, the icing areas and the like are distinguished and the distribution of the water accumulation areas, the snow accumulation areas, the icing areas and the like on the asphalt road surface is obtained.
The invention has the beneficial effects that:
1. the method can predict the pavement type of the pavement to be contacted, can obtain the distribution conditions of different local pavements on the same asphalt pavement, makes up the defects of a dynamic method and a visual method by means of deep learning, and provides a prerequisite for path planning and decision of intelligent vehicles and intelligent machines.
2. And an offline pre-training image segmentation network is adopted, so that the real-time performance is good, and the safety of the system is improved to a great extent.
3. The CARLA software is used for acquiring a data set required by pre-training, so that the time cost and the economic cost for acquiring image data are greatly reduced.
Drawings
FIG. 1 is a schematic overall flow diagram of the process of the present invention;
FIG. 2 is a schematic view of a high attention module;
FIG. 3 is a schematic diagram of a residual structure;
fig. 4 is a schematic diagram of the overall structure of H-ResNet.
Detailed Description
The invention will be further explained with reference to the drawings. It should be understood that the specific examples described herein are intended to be illustrative only and are not intended to be limiting.
The general technical process of the local road surface adhesion coefficient estimation method based on image segmentation is shown in the attached figure 1, and comprises the following steps:
step 1: the offline pre-training image segmentation network specifically comprises: a. the method comprises the steps of collecting road surface images under different weather conditions by using CARLA software, b, carrying out local labeling on the collected road surface images under different weather conditions to form a data set of local road surface adhesion coefficient estimation, c, building a deep learning algorithm network framework of image segmentation by using a deep learning means, d, carrying out end-to-end training on the deep learning algorithm network framework of image segmentation by using the data set of local road surface adhesion coefficient estimation, and e, transplanting a trained network model into a vehicle machine of an intelligent vehicle or an intelligent machine through ROS software.
Step 2: the method comprises the following steps of acquiring a real-time road surface image, and estimating a local adhesion coefficient of the road surface in real time, wherein the method specifically comprises the following steps: a. acquiring real-time road surface images by using a vehicle-mounted camera, classifying the real-time acquired images by using a pre-trained image segmentation network and positioning different types to form a real-time road condition map, and c, estimating local road surface adhesion coefficients of the real-time road condition map according to the road surface type.
Further, in the step 1a, road surface images of different weather conditions are collected, wherein the collection tool is CARLA simulation software, and the collected images include a local water accumulation image of the road surface, a local snow accumulation image of the road surface, a local icing image of the road surface, a normal asphalt road surface and the like.
The specific implementation procedure for steps 1 and 2 above is as follows:
the weather conditions of the scene are set in CARLA software, the collected image vehicles are driven on asphalt pavements with different weather conditions by a first person, and the pavement images are collected, so that the pavement types of the single-frame images are guaranteed to contain two or more than two of the pavement types in the attached list. At this stage, the data including the scene of the road, the illumination, the weather condition and the road structure are ensured to be as diverse as possible. The scene of the road includes but is not limited to a high-speed asphalt pavement, an urban downtown asphalt pavement and an urban suburban asphalt pavement; weather conditions include, but are not limited to, sunny weather, rainy weather, snowy weather, etc., and lighting conditions include, but are not limited to, morning, noon, dusk, night, etc.
And (3) local labeling of the image, namely performing local labeling of the road surface type and the corresponding road surface adhesion coefficient on the road surface image according to the road surface type and the corresponding road surface adhesion coefficient in the attached table 1 to form a data set required by pre-training.
TABLE 1
The semantic segmentation neural network algorithm model comprises a basic network structure constructed by using a residual error structure, a highly-driven attention module is added on the basic network, the semantic segmentation neural network is named as H-ResNet, wherein H represents the highly-driven attention module, and ResNet represents the basic network structure constructed by the residual error structure. The algorithm flow comprises a data import module, a data preprocessing module, a neural network forward propagation module, an activation function, a loss function, a backward propagation module and an optimization module.
As shown in fig. 3, the residual structure unit mainly includes: quick connection and identity mapping. As shown in fig. 3, X represents a characteristic input, mapped to an output by the shortcut connection identity on the right. weight layer represents the convolution weight layer, relu represents the activation function, and F (X) represents the residual error of the feature representation learned by X passing through the convolution weight layer. Let h (X) denote the feature that input X finally learns, it is now desirable that it can learn the residual, so that the feature that actually needs to be learned becomes f (X) ═ h (X) -X. This is so because residual learning is easier than direct learning of the original features. When the residual f (x) is 0, the convolution weight layer is only mapped identically, so that at least the performance of the network is not degraded, and actually the residual f (x) is not 0, which enables the convolution weight layer to learn new features based on the input features.
In this example, the structure of the residual error is adopted to prevent the problem of gradient disappearance in pre-training, so as to train the network better, the convolution operation with different convolution kernel sizes is carried out for a plurality of times in the structure of the residual error to better extract image features, and the convergence rate of neural network training can be increased by adopting the ReLU nonlinear activation function.
In this example, the convolution operation is formulated as follows, where w (x, y) represents a convolution kernel of size m × n, f (x, y) is an image, and · represents the convolution operation.
Wherein a represents the sum of pixels in the image width direction
b represents the sum of pixels in the image length direction
s represents the pixel position in the image width direction
t represents the pixel position in the image width direction
In this example, the ReLU activation function formula is as follows, and is a non-linear function, where the function is 0 when x, the abscissa, is equal to or less than 0, and the function value is equal to x when x is greater than 0.
In this embodiment, a highly-driven attention module is added to the basic network to improve the recognition accuracy and ensure the real-time performance of recognition.
The high-driven attention module is provided by analyzing the difference of pixels occupied by different scene types in different height levels of a single-frame image in an actual urban scene image. As shown in fig. 2, a single frame image is divided into three levels of height, i.e., upper, middle and lower levels, where the image-dominant pixel in the upper level is sky, the image-dominant pixel in the middle level is a vehicle, a pedestrian, a building, etc., and the image-dominant pixel in the lower level is a road surface, i.e., an area of interest. The attention module driven by the height can directly acquire the interested region in the image, so that the recognition accuracy is improved, and the recognition speed is accelerated.
The height-driven attention module includes operations of width pooling, down-sampling, calculating a height-driven attention feature map, inserting feature position coding, and the like. Wherein, the width pooling operation is to obtain a characteristic diagram in the image width direction, and the pooling mode is average pooling; the feature maps obtained through the width pooling operation are not all necessary, the down-sampling aims to remove unnecessary feature maps, the calculation of the down-sampled feature maps utilizes convolution operation, such as formula 1, to obtain the adjacent position relation of the feature maps, the final inserted feature position coding operation is to obtain the prior information of the vertical position of a specific object, and the position coding is generated through sine and cosine functions of different frequencies and then is added with the feature vectors of corresponding positions element by element.
In this embodiment, the overall structure of H-ResNet is shown in FIG. 4, where each ResNet stage represents a residual structure and H represents a highly driven attention module. The identification result is more accurate as the number of ResNet stages is larger, however, as the number of ResNet stages is increased, the network calculation amount is increased, and the real-time performance is reduced. The number of high-drive attention modules increases with the number of ResNet stages, in this example 4 ResNet stages, 3 high-drive attention modules are used. Each height driving attention module is inserted between the two residual error modules, the prior information of the vertical position is obtained once after the residual error modules pass each time, and the state information of the road surface can be more accurately obtained after the prior information of the vertical position is obtained for multiple times.
In this example, the deep learning framework is TensorFlow or Keras or Caffe2 or PyTorch or MXNet.
In this example, the hardware configuration of the experimental platform used for training the semantic segmentation model is a GPU of GeForce GTX 1080Ti and a CPU of i7-9700k with 64GB memory. In terms of software configuration, the experimental platform is based on Ubuntu18.04 of a 64-bit operating system. A network model is constructed by adopting a current mainstream deep learning frame pytorech and python languages, and high-performance parallel computing is carried out by using a parallel computing architecture CUDA and a GPU acceleration library CUDNN.
In this example, the training uses Focal local, also called focus Loss, as shown in equation 7, as a function of the training Loss. Focal local reduces the weight of samples which can be well classified by modifying a standard cross entropy Loss function, increases the weight of samples which are difficult to classify, and enables a model to quickly pay attention to difficult samples, namely relatively few samples, in the training process so as to solve the problem of class sample imbalance.
Where N represents the number of samples used for network training, i represents the sample index, yiRepresenting the label sample corresponding to each training sample, and alpha represents the weight parameter at [0,1]The middle value, gamma, is also a hyperparameter, piIn [0,1 ]]A median value, corresponding to yiAs a general theory of prediction of +1,
in this example, the low-level and high-level semantic information of the image is learned most quickly in order to ensure that the model is stable. The learning rate in model training is selected as the exponentially decaying learning rate, and the formula is shown in fig. 8.
decayed_lr＝init_lr×decay_rate(global_step/decay_steps) (8)
In the formula: init _ lr learning rate of initial setting
decay _ rate-attenuation coefficient
global _ step-iteration round number
decay step decay Rate
In this example, the number of images input during each iterative training is set to 8, and data enhancement is performed on the input images by means of random scale transformation, random angle rotation, image inversion and the like. The initial learning rate init _ lr is set to 0.001, the attenuation coefficient decay _ rete is set to 0.95, the iteration round number global _ step is 5400, and the attenuation speed decay _ steps is set to 50.
And inputting the collected data set into the constructed semantic segmentation neural network for end-to-end training, and integrating the trained model into a vehicle machine of an intelligent vehicle or other intelligent machines through ROS software.
And acquiring a real-time image by using a vehicle-mounted network camera or a USB camera, inputting the real-time image into a semantic segmentation model integrated in a vehicle machine, and distinguishing the categories of different areas on the road surface in real time and positioning the distribution of the categories so as to obtain a road surface condition distribution map.
In this example, the camera is installed in car windshield department, avoids receiving environmental disturbance, influences the quality of gathering the image.
The above-listed series of detailed descriptions are merely specific illustrations of possible embodiments of the present invention, and they are not intended to limit the scope of the present invention, and all equivalent means or modifications that do not depart from the technical spirit of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A local road surface adhesion coefficient estimation method based on image segmentation is characterized by comprising the following steps:
step 1: the offline pre-training image segmentation network model specifically comprises the following steps:
1.1. the road surface images of different weather conditions are collected,
1.2. the collected road surface images with different weather conditions are locally marked to form a data set of local road surface adhesion coefficient estimation,
1.3. constructing a deep learning network model of image segmentation,
1.4. end-to-end training is carried out on the deep learning algorithm network framework of image segmentation by utilizing a data set estimated by a local pavement adhesion coefficient;
step 2: the method comprises the following steps of acquiring a real-time road surface image, and estimating a local adhesion coefficient of the road surface in real time, wherein the method specifically comprises the following steps:
2.1. the road surface image is collected, and the road surface image,
2.2. classifying the images acquired in real time by using a pre-trained image segmentation network model and positioning different classes to form a real-time road condition map,
2.3. and estimating the local road adhesion coefficient of the real-time road condition map according to the road surface type.
2. The method for estimating the local road adhesion coefficient based on image segmentation as claimed in claim 1.1, wherein in step 1.1, road images of different weather conditions are collected, the collection tool used is CARLA simulation software, and the collected images include a local water accumulation image of the road, a local snow accumulation image of the road, a local ice image of the road and a normal asphalt road.
3. The method according to claim 1, wherein in 1.3, the deep learning network model for image segmentation comprises a base network structure constructed by using a residual structure, a highly-driven attention module is added to the base network, and the semantic segmentation neural network model is named as an H-ResNet model, wherein H represents the highly-driven attention module, and ResNet represents the base network structure constructed by the residual structure.
4. The method for estimating the local road adhesion coefficient based on the image segmentation as claimed in claim 3, wherein each ResNet stage in the H-ResNet model represents a residual structure, and H represents a highly-driven attention module; 4 ResNet stages and 3 height-driven attention modules are adopted, each height-driven attention module is inserted between two residual modules, the prior information of the vertical position is obtained once after the residual modules are passed each time, and the state information of the road surface can be more accurately obtained after the prior information of the vertical position is obtained for multiple times.
5. The method according to claim 4, wherein the residual error unit comprises: quick connection and identity mapping; setting X to represent characteristic input, and mapping to output through quick connection and identity; weight layer represents a convolution weight layer, relu represents an activation function, f (X) represents a residual error represented by a feature learned by X through the convolution weight layer, h (X) represents a feature finally learned by input X, and the feature to be learned is equivalent to f (X) ═ h (X) -X;
the activation function adopts a ReLU nonlinear activation function: when x is less than or equal to 0, the function is 0, and when x is greater than 0, the function is equal to x:
the operation formula of the convolutional layer is as follows:
where w (x, y) represents a convolution kernel of size m × n, f (x, y) represents an image, and represents a convolution operation.
6. The method of claim 4, wherein the elevation-driven attention module is designed for different pixels occupied by different scene types in different elevation levels of a single frame image in an actual urban scene image, specifically, the single frame image is divided into three elevation levels, namely, an upper elevation level, a middle elevation level, a lower elevation level, an image dominant pixel in the upper elevation level is sky, an image dominant pixel in the middle elevation level is a vehicle, a pedestrian, a building and the like, and an image dominant pixel in the lower elevation level is a road surface, namely a region of interest.
The height-driven attention module comprises width pooling, down-sampling, calculation of a height-driven attention feature map, and insertion feature position encoding; wherein, the width pooling operation is to obtain a characteristic map in the image width direction, and the pooling mode is average pooling; the feature maps obtained through the width pooling operation are not all necessary, unnecessary feature maps are removed through downsampling, the calculation of the downsampled feature maps utilizes convolution operation, such as formula 1, to obtain the adjacent position relation of the feature maps, the last operation of inserting the feature position codes can obtain the prior information of the vertical position of a specific object, the position codes are generated through sine and cosine functions with different frequencies, and then the prior information and the feature vectors of the corresponding positions are added element by element.
7. The method for estimating the local road adhesion coefficient based on image segmentation according to claim 1, wherein the training method in 1.4 is a back propagation method of Batch Gradient Descent (Batch Gradient Descent) or random Gradient Descent (Stochastic Gradient Descent) or small Batch Gradient Descent (Mini-Batch Gradient Descent) using a single GPU or multiple GPUs;
the Focal local used for training is obtained by modifying a cross entropy Loss function of a standard:
the exponentially decaying learning rate is selected as the learning rate during model training, as shown in equation 8:
decayed_lr＝init_lr×decay_rate(global_step/decay_steps) (8)
in the formula: init _ lr learning rate of initial setting
decay _ rate-attenuation coefficient
global _ step-iteration round number
decay steps decay rate.
8. The method for estimating the local road adhesion coefficient based on image segmentation as claimed in claim 1, wherein the means for acquiring the road image in 2.1 is a vehicle-mounted camera, which can be a network camera or a USB camera.
9. The method of claim 1, wherein the classifying and locating the different classes in 2.2 is to use dry asphalt as a background road surface, and to distinguish a water-accumulated area, a snow-accumulated area, an ice-formed area, or any combination thereof and obtain the distribution of the water-accumulated area, the snow-accumulated area, the ice-formed area, or any combination thereof on the asphalt road surface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011067813.4A CN112329533B (en) | 2020-10-07 | 2020-10-07 | Local road surface adhesion coefficient estimation method based on image segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011067813.4A CN112329533B (en) | 2020-10-07 | 2020-10-07 | Local road surface adhesion coefficient estimation method based on image segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112329533A true CN112329533A (en) | 2021-02-05 |
| CN112329533B CN112329533B (en) | 2024-05-14 |
Family
ID=74314547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011067813.4A Active CN112329533B (en) | 2020-10-07 | 2020-10-07 | Local road surface adhesion coefficient estimation method based on image segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112329533B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113312976A (en) * | 2021-04-30 | 2021-08-27 | 淮阴工学院 | Braking distance calculation method based on combination of image processing and road adhesion coefficient |
| CN113435409A (en) * | 2021-07-23 | 2021-09-24 | 北京地平线信息技术有限公司 | Training method and device of image recognition model, storage medium and electronic equipment |
| CN113887532A (en) * | 2021-11-17 | 2022-01-04 | 安徽省公共气象服务中心 | Method for identifying and correcting accumulated snow images of expressway based on scene classification |
| CN114332715A (en) * | 2021-12-30 | 2022-04-12 | 武汉华信联创技术工程有限公司 | Method, device and equipment for identifying snow through automatic meteorological observation and storage medium |
| CN114332722A (en) * | 2021-12-31 | 2022-04-12 | 吉林大学 | Real-time estimation method for adhesion coefficient of mixed ice and snow road surface based on video data |
| CN114820819A (en) * | 2022-05-26 | 2022-07-29 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN118691838A (en) * | 2024-08-22 | 2024-09-24 | 云途信息科技(杭州)有限公司 | Urban ponding depth estimation method and system based on multi-reference object system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263844A (en) * | 2019-06-18 | 2019-09-20 | 北京中科原动力科技有限公司 | A kind of method of on-line study and real-time estimation pavement state |
| CN110378416A (en) * | 2019-07-19 | 2019-10-25 | 北京中科原动力科技有限公司 | A kind of coefficient of road adhesion estimation method of view-based access control model |
| CN111723849A (en) * | 2020-05-26 | 2020-09-29 | 同济大学 | Road adhesion coefficient online estimation method and system based on vehicle-mounted camera |
-
2020
- 2020-10-07 CN CN202011067813.4A patent/CN112329533B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263844A (en) * | 2019-06-18 | 2019-09-20 | 北京中科原动力科技有限公司 | A kind of method of on-line study and real-time estimation pavement state |
| CN110378416A (en) * | 2019-07-19 | 2019-10-25 | 北京中科原动力科技有限公司 | A kind of coefficient of road adhesion estimation method of view-based access control model |
| CN111723849A (en) * | 2020-05-26 | 2020-09-29 | 同济大学 | Road adhesion coefficient online estimation method and system based on vehicle-mounted camera |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113312976A (en) * | 2021-04-30 | 2021-08-27 | 淮阴工学院 | Braking distance calculation method based on combination of image processing and road adhesion coefficient |
| CN113435409A (en) * | 2021-07-23 | 2021-09-24 | 北京地平线信息技术有限公司 | Training method and device of image recognition model, storage medium and electronic equipment |
| CN113887532A (en) * | 2021-11-17 | 2022-01-04 | 安徽省公共气象服务中心 | Method for identifying and correcting accumulated snow images of expressway based on scene classification |
| CN114332715A (en) * | 2021-12-30 | 2022-04-12 | 武汉华信联创技术工程有限公司 | Method, device and equipment for identifying snow through automatic meteorological observation and storage medium |
| CN114332722A (en) * | 2021-12-31 | 2022-04-12 | 吉林大学 | Real-time estimation method for adhesion coefficient of mixed ice and snow road surface based on video data |
| CN114820819A (en) * | 2022-05-26 | 2022-07-29 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN114820819B (en) * | 2022-05-26 | 2023-03-31 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN118691838A (en) * | 2024-08-22 | 2024-09-24 | 云途信息科技(杭州)有限公司 | Urban ponding depth estimation method and system based on multi-reference object system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112329533B (en) | 2024-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112329533B (en) | Local road surface adhesion coefficient estimation method based on image segmentation | |
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN111598030B (en) | Method and system for detecting and segmenting vehicle in aerial image | |
| CN110147763B (en) | Video semantic segmentation method based on convolutional neural network | |
| CN108830188B (en) | Vehicle detection method based on deep learning | |
| CN109858372B (en) | Lane-level precision automatic driving structured data analysis method | |
| CN106228125B (en) | Method for detecting lane lines based on integrated study cascade classifier | |
| CN113506300B (en) | Picture semantic segmentation method and system based on rainy day complex road scene | |
| US20230358533A1 (en) | Instance segmentation imaging system | |
| CN105513354A (en) | Video-based urban road traffic jam detecting system | |
| CN110849807B (en) | Monitoring method and system suitable for road visibility based on deep learning | |
| CN110717886A (en) | Pavement pool detection method based on machine vision in complex environment | |
| CN113065578A (en) | Image visual semantic segmentation method based on double-path region attention coding and decoding | |
| CN109034024B (en) | Logistics vehicle type classification and identification method based on image target detection | |
| Xing et al. | Traffic sign recognition using guided image filtering | |
| CN113052159A (en) | Image identification method, device, equipment and computer storage medium | |
| CN117351702A (en) | Intelligent traffic management method based on adjustment of traffic flow | |
| CN212009589U (en) | Video identification driving vehicle track acquisition device based on deep learning | |
| CN116630702A (en) | Pavement adhesion coefficient prediction method based on semantic segmentation network | |
| CN114973199A (en) | Rail transit train obstacle detection method based on convolutional neural network | |
| CN117763423A (en) | Intelligent automobile laser radar point cloud anomaly detection method based on deep learning | |
| CN116665153A (en) | Road scene segmentation method based on improved deep bv3+ network model | |
| CN112785610B (en) | Lane line semantic segmentation method integrating low-level features | |
| CN111046723A (en) | Deep learning-based lane line detection method | |
| CN117636268A (en) | Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |