CN116311103A - Incremental learning-based pavement ponding detection method, device, medium and equipment - Google Patents
Incremental learning-based pavement ponding detection method, device, medium and equipment Download PDFInfo
- Publication number
- CN116311103A CN116311103A CN202310518422.7A CN202310518422A CN116311103A CN 116311103 A CN116311103 A CN 116311103A CN 202310518422 A CN202310518422 A CN 202310518422A CN 116311103 A CN116311103 A CN 116311103A
- Authority
- CN
- China
- Prior art keywords
- loss function
- ponding
- model
- incremental training
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 117
- 238000012549 training Methods 0.000 claims abstract description 109
- 230000006870 function Effects 0.000 claims abstract description 90
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000009825 accumulation Methods 0.000 claims abstract description 40
- 230000008569 process Effects 0.000 claims abstract description 27
- 238000013140 knowledge distillation Methods 0.000 claims abstract description 23
- 238000004364 calculation method Methods 0.000 claims description 25
- 239000002352 surface water Substances 0.000 claims description 18
- 238000009826 distribution Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000004913 activation Effects 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000002679 ablation Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 206010027175 memory impairment Diseases 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009469 supplementation Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
A pavement ponding detection method, device, medium and equipment based on incremental learning, the method comprises: in the process of carrying out water accumulation detection by the water accumulation detection model, periodically establishing an incremental training task aiming at the water accumulation detection model, and periodically acquiring a pavement water accumulation image in the water accumulation detection process to generate a corresponding image sample data set; constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function; starting a current incremental training task, and performing incremental training on the ponding detection model by using the target loss function and an image sample data set corresponding to each training task until the model converges or the expected precision is reached; and carrying out pavement ponding detection by using the ponding detection model after regular training. The pavement water accumulation detection method can continuously update the model and improve the detection accuracy and stability.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a pavement ponding detection method, device, medium and equipment based on incremental learning.
Background
The accumulated water on the road surface is a puddle formed by accumulation of rainwater on the road surface, and is a potential safety hazard for pedestrians and drivers. For example, in rainy days, the driver needs to pay attention to the situation of road surface water all the time, and adjusts the vehicle speed and the driving route according to the situation so as to ensure driving safety. The traditional pavement ponding detection method mainly depends on manual inspection and field equipment monitoring, and is extremely low in efficiency and very wasteful of human resources. In recent years, with the continuous development of computer vision and machine learning technologies, a road surface water accumulation detection method based on image processing and machine learning is gradually and widely used.
The existing pavement water accumulation detection method based on image processing and machine learning mainly has the following problems. Firstly, aiming at ponding conditions of different pavements and different weather conditions, the robustness of a detection algorithm is insufficient and the detection accuracy is not high. Secondly, most algorithms have poor real-time performance, and cannot meet the requirement of real-time monitoring. Finally, it is also the most important point that most algorithms have weak learning ability and adaptability at present, and cannot quickly learn and adapt to new scenes.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, apparatus, medium and device for detecting road surface water based on incremental learning, which address the problems of the prior art.
The invention discloses a pavement ponding detection method based on incremental learning, which comprises the following steps:
in the process of carrying out water accumulation detection by the water accumulation detection model, periodically establishing an incremental training task aiming at the water accumulation detection model, and periodically acquiring a pavement water accumulation image in the water accumulation detection process to generate a corresponding image sample data set;
constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function;
starting a current incremental training task, and performing incremental training on the ponding detection model by using the target loss function and an image sample data set corresponding to each training task until the model converges or the expected precision is reached;
in the incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task;
and carrying out pavement ponding detection by using the ponding detection model after regular training.
Further, in the above road surface water accumulation detection method, the objective loss function is:
wherein ,
；
Further, in the road surface ponding detection method, the ponding detection model comprises an encoder and a decoder, a feature extraction module in the encoder adopts a Ghost-block module, and the Ghost-block module is used for carrying out linear operation after carrying out convolution calculation on an input image so as to obtain a plurality of feature images.
Further, in the above road surface water accumulation detection method, the step of performing a linear operation after performing a convolution calculation on the input image by using the Ghost-block module includes:
by means of
The convolution kernel size is +.>
Is>
Performing convolution operation with an input image to obtain m output feature images;
and carrying out linear operation on the output characteristic diagram, wherein the operation formula is as follows:
wherein ,
?>
Output feature map, < >>
Indicate->
The output feature map is +.>
A linear operation, s is an empirical value, +.>
Is an output characteristic diagram after linear operation.
Further, in the above road surface water detection method, the decoder is configured to fuse the shallow feature map and the deep feature map by using a long jump connection and a gated attention mechanism, so that for each pixel point of the feature map output by each level of the coding layer, the current attention coefficient tends to generate a response value higher than that of the background region in the road surface water region.
The invention also discloses a pavement ponding detection device based on incremental learning, which comprises:
the task building module is used for regularly building an incremental training task aiming at the ponding detection model in the ponding detection process of the ponding detection model, and regularly acquiring a pavement ponding image in the ponding detection process so as to generate a corresponding image sample data set;
the loss function construction module is used for constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function;
the incremental training module is used for starting a current incremental training task, and performing incremental training on the ponding detection model by utilizing the target loss function and the image sample data set corresponding to each training task until the model converges or the expected precision is reached;
in the incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task;
and the detection module is used for detecting the accumulated water on the road surface by using the accumulated water detection model after regular training.
Further, in the road surface water accumulation detection device, the target loss function is:
wherein ,
；
Further, in the road surface ponding detection device, the ponding detection model comprises an encoder and a decoder, a feature extraction module in the encoder adopts a Ghost-block module, and the Ghost-block module is used for carrying out linear operation after carrying out convolution calculation on an input image so as to obtain a plurality of feature images.
The invention also discloses a computer device comprising a memory and a processor, wherein the memory stores a program which when executed by the processor implements any of the methods described above.
The invention also discloses a computer readable storage medium having stored thereon a program which when executed by a processor implements any of the methods described above.
The pavement ponding detection method can continuously update the model, improve the detection accuracy and stability, save the calculation resources and have wide application prospect and economic value. The method can be applied to various traffic safety fields, and has important significance for preventing traffic accidents and improving traffic safety level.
Drawings
FIG. 1 is a diagram of an overall structure of an EU-Net model in an embodiment of the present invention;
FIG. 2 is a flow chart of a method for detecting water in a road area according to an embodiment of the invention;
FIG. 3 is a block diagram of a road area water detection device in an embodiment of the invention;
fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
Embodiments of the present invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular implementations of embodiments of the invention are disclosed in detail as being indicative of some of the ways in which the principles of embodiments of the invention may be employed, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all alternatives, modifications and equivalents as may be included within the spirit and scope of the appended claims.
The pavement ponding detection method is applied to a ponding detection model, and the pavement ponding is detected in real time through the ponding detection model.
At present, three modes of road surface water accumulation detection based on deep learning are respectively image classification, target detection and semantic segmentation. The accumulated water detection model in the embodiment adopts a semantic segmentation model, and adopts a pavement accumulated water detection method based on semantic segmentation, so that not only can better precision be obtained, but also a specific accumulated water area can be estimated well.
A common convolutional neural network-based image segmentation model U-Net aims at segmenting an input image into a plurality of semantic regions. It consists of an encoder for compressing the input image into a feature representation and a decoder for restoring the feature map to the resolution of the input image by upsampling while generating a corresponding segmentation mask.
U-Net performs excellently in many image segmentation tasks, especially for small sample data sets and unbalanced categories. Based on the advantages of U-Net, the embodiment takes the U-Net as a model base and improves the model, and the improved model is called an EU-Net model, namely a ponding detection model in the embodiment of the invention.
As shown in FIG. 1, the EU-Net model in the embodiment of the invention comprises an input module, a lightweight encoder, a decoder and an output module. The encoder is composed of a series of operations such as convolution and pooling, and is used for continuously abstracting the input image characteristics and extracting high-level characteristic representations. The decoder gradually restores the output characteristic diagram of the encoder to the size of the original input image through operations such as up-sampling, deconvolution and the like, and performs characteristic fusion and detail supplementation by using a long jump connection and a gating attention mechanism in the process, so that an output result of road surface water detection is finally obtained.
Considering that the original U-Net model contains many standard convolution modules, the overall parameters are very large and redundant, thus affecting the final reasoning efficiency. The ponding detection model in this embodiment employs a custom lightweight encoder-decoder architecture. The convolution module in the lightweight encoder adopts a Ghost-block module, and the Ghost-block module is a module formed by tensor grouping, cross channel information transmission and other technologies, so that model parameters and calculated amount can be effectively reduced, and meanwhile, high precision and strong expression capability of a model can be maintained.
In this embodiment, the Ghost-block module is configured to perform a linear operation after performing a convolution calculation on an input image, so as to obtain a plurality of feature maps. Specifically, the original standard convolution module is formed by a convolution module with
The convolution layer of the filter and a nonlinear activation function can be expressed as:
wherein ,
is a convolution kernel +.>
Is a bias item->
Is an input image, < >>
For the output characteristic diagram of the corresponding original convolution module, < >>
Is an activation function.
As a modification, the present embodiment adopts only the belt
The convolution kernel size is +.>
Is>
Is at the same time->
Performing convolution operation to obtain m output feature graphs +.>
. Here->
. Subsequently, this is->
The characteristic diagrams are respectively processed by a series of linear operations to finally obtain the equivalent +.>
The formula of the feature map is as follows:
here, the
Representation->
The%>
Personal profile->
Indicate->
The feature map is subject to->
And (3) linear operation for generating a corresponding output characteristic diagram. The original output is therefore re-expressed as:
in this way, the matrix operation with longer time consumption in the original convolution module can be replaced by the linear operation with higher efficiency.
Furthermore, in the EU-Net model, the decoder utilizes a long jump connection and a gating attention mechanism to fuse the shallow space information and the deep semantic information of the feature map output by the encoder, namely, the high-level feature information from the encoder can be reinjected into the decoder, so that the accuracy of a segmentation result and the detail retaining capability are improved. Considering that some noise information may exist in the image, the original jump connection mode may cause that the model cannot effectively extract the features with more discriminant by directly fusing the shallow space and the deep semantic information. For this reason, the present embodiment introduces a gating Attention mechanism (Attention-oriented) on the basis of the long-hop connection to improve this situation.
Specifically, each pixel point of the feature map is output for each level of the coding layer
Current attention coefficient->
Higher response values tend to be generated in the area of the road surface water accumulation, and lower response values are generated in the background area, so that the detection accuracy and the robustness of the model can be further improved>
。
The further step may be obtained by element-wise multiplying the input feature map and the attention coefficient
The output of attention is layer-gated. In addition, the mechanism can use gating vector +.>
To determine the focal region of each pixel. The following is an attention expression in the present embodiment:
here the number of the elements is the number,
and />
Input feature maps from deep and shallow layers respectively; element->
And
respectively representing different linear transformation operations, for simplicity the input tensor is here calculated directly using a channel-by-channel 1 x 1 convolution, while omitting the corresponding offset value calculation. />
and />
The ReLU and Sigmoid activation functions are represented, respectively. Finally, by adding features->
And attention coefficient->
The +.sup.th can be obtained further by element multiplication>
The output of attention is layer-gated.
Notably, unlike normalizing the attention coefficients using a softmax activation function in the image classification task, the embodiment of the invention selects a sigmoid activation function to avoid a sparser activation of softmax at the output, thereby facilitating easier convergence of the training process.
Referring to fig. 2, the method for detecting the accumulated water on the road surface in the embodiment of the invention includes steps S11 to S14.
Step S11, in the process of carrying out water accumulation detection by the water accumulation detection model, an incremental training task aiming at the water accumulation detection model is established regularly, and a pavement water accumulation image in the water accumulation detection process is acquired regularly so as to generate a corresponding image sample data set.
In the embodiment, the EU-Net model is utilized to detect the accumulated water on the road surface, and the EU-Net model acquires the accumulated water image on the road surface in real time after being on line and detects the accumulated water. And in the process of carrying out ponding detection in real time, an incremental training task is established regularly, a plurality of pavement ponding images are acquired regularly, and a corresponding image sample data set is generated according to the pavement ponding images acquired in each period. Each incremental training task will correspond to one incremental data, i.e., one image sample dataset. And performing incremental training on the EU-Net model regularly according to the incremental training task established regularly.
Step S12, constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function.
In order to retain the previous knowledge and alleviate catastrophic forgetfulness, the objective loss function constructed in this embodiment is a composite loss function, and different penalty terms are adopted for the incremental training samples (the image sample data set corresponding to the current incremental training task) and the old training samples (the image sample data set corresponding to each task before the current task). For old training samples, using knowledge distillation (knowledge distillation), models are trained to output similar features using predictions of the model over the old task as pseudo-tags or soft-tags. The knowledge distillation loss calculation is introduced, and the purpose is to keep the knowledge of the old task as much as possible while training on the new task, so that the soft label output by the model is smoother, and the overfitting is avoided in the training process. For incremental training samples, the cross-entropy (cross-entropy) loss function is directly used to perform conventional optimization work, with the objective of training the model on new tasks to obtain better classification performance. Specifically, the objective loss function is:
wherein ,
for cross entropy loss function, +.>
Is indicated at +.>
Knowledge on incremental training tasksIdentifying distillation loss function, ++>
And t is the total number of incremental training tasks. I.e., the image sample data of the t-th incremental task employs a cross entropy loss function, and the image sample data of the 1~t-1 task employs a knowledge distillation loss function.
In the present invention, the temperature parameter of the softmax function is used
To control the distribution of the soft labels and set them to a smaller value. Specifically, for the->
Incremental training tasks, corresponding image sample data set E i N pavement ponding image samples are contained, which are->
The knowledge distillation loss function for the individual incremental training tasks can be expressed as:
c is the total number of categories and,
for ponding detection model at->
Sample->
Predicted as category->
Probability distribution of->
For ponding detection model at->
Sample->
For category->
Is a true probability distribution of (c).
And S13, starting a current incremental training task, and performing incremental training on the ponding detection model by using the target loss function and the image sample data set corresponding to each training task until the model converges or the expected precision is reached.
In the current incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task.
Furthermore, in the execution process of the current incremental training task, the random initialization weight is not adopted, and the model weight trained in the previous task is adopted as the initial weight.
And S14, detecting the accumulated water of the road surface by using the accumulated water detection model after regular training.
And detecting the road area water in real time on the basis of a periodically trained ponding model. When the water accumulation area in the specific area is detected to be larger than a given threshold value, current data are fed back to the control console in real time so as to trigger an alarm to generate linkage.
It can be appreciated that before the water image of the road area is detected by using the water accumulation detection model, the region of interest, such as the region easy to accumulate water, such as an intersection, a bridge, a tunnel and the like, can be manually or automatically marked in the collected water accumulation image of the road surface. By limiting the detection area, the calculation amount of the algorithm can be reduced, the detection efficiency can be improved, and the possibility of false detection can be reduced.
Further, when the area of the accumulated water in the designated area is larger than a given threshold in the prediction result output by the accumulated water detection model, the system automatically sends alarm information to related personnel and performs other predefined linkage operations. These operations may include, but are not limited to: the real-time inspection is performed through video monitoring, and maintenance personnel are notified to go to the site for maintenance, road traffic control and the like. Through the step, the road surface ponding detection system can timely and accurately predict and respond to the occurrence of road surface ponding, and the safety and smoothness of road surface traffic are ensured.
Generally, before the water accumulation detection model is put on line, a pre-model training is required to be performed on the water accumulation detection model, and the water accumulation detection model is put on line after the training is completed. The early model training can be used as an independent implementation task, and the part of image data does not participate in training calculation in the subsequent incremental training; alternatively, the early model training may be the most initial incremental training task.
Specifically, the early model training includes steps S01-S05.
S01: and (5) data acquisition. And acquiring relevant pavement ponding image data sets of different heights, visual angles, distances, environments, weather, illumination and the like by using cameras arranged on the road. In the data acquisition process, the acquired data set is subjected to related data cleaning work, and unqualified images such as overexposure, distortion, blurring and the like are filtered. In addition, the cleaned data set is divided into two types, one type is background data of the area without water accumulation, and the data is regarded as a negative sample and is used for suppressing false detection by later model training. Another type of data is a foreground dataset, wherein each picture contains ponding targets with different sizes and shapes so as to facilitate subsequent marking. Through the steps, a high-quality representative data set can be obtained, and a basis is provided for subsequent model training and optimization.
S02: and (5) marking data. After the data acquisition and cleaning work is completed, the data labeling is needed next to facilitate the subsequent semantic segmentation model training. In addition, the embodiment can use the CVAT based on the online labeling tool for marking correspondingly. Specifically, each pixel in each foreground data set needs to be manually marked as belonging to a 'ponding area' or a 'non-ponding area' by a marking tool respectively. After the labeling is completed, each picture is converted into a labeling file, wherein each pixel is labeled as '1' (belonging to the ponding area) or '0' (belonging to the non-ponding area), and finally, labeling information is stored in json format. In addition, during the labeling process, care should be taken to ensure the accuracy and consistency of the labeling. Meanwhile, the balance of the data set, namely the quantity of the ponding area and the quantity of the non-ponding area need to be balanced relatively, so that unbalance of model training caused by inclination of the data set is avoided.
S03: and (5) data division. Finally, the entire image dataset is divided into a training set, a validation set and a test set. Wherein 70% of the data is used as a training set of the model, and the remaining 30% of the data is used as a verification and test set for training, tuning and evaluation of the model.
S04: and (5) preprocessing data. Here, a batch of data is randomly extracted from the training set according to rules, and corresponding data pipeline processing is performed on the data, including operations such as image size scaling, data enhancement and normalization. To increase the diversity and generalization capability of the data set, the data enhancement portion may employ rotation, flipping, scaling, cropping, color channel conversion, color space adjustment, and the like. These operations may improve the robustness and generalization ability of the model to some extent.
S05: and (5) model training. Aiming at the EU-Net model constructed before online, training is carried out by utilizing image data in a training set, and testing is carried out based on a testing set. The EU-Net model which is qualified in test can be used for detecting the road surface ponding.
Referring to fig. 3, in an embodiment of the present invention, a pavement water detection device based on incremental learning includes:
the task building module 31 is configured to periodically build an incremental training task for the ponding detection model during the ponding detection process of the ponding detection model, and periodically acquire a pavement ponding image during the ponding detection process to generate a corresponding image sample data set;
a loss function construction module 32 for constructing a target loss function including a cross entropy loss function and a knowledge distillation loss function;
the incremental training module 33 is configured to start a current incremental training task, and perform incremental training on the ponding detection model by using the objective loss function and an image sample data set corresponding to each training task until the model converges or reaches an expected accuracy;
in the incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task;
the detection module 34 is configured to perform pavement water accumulation detection by using the water accumulation detection model after regular training.
Further, in the road surface water accumulation detection device, the target loss function is:
wherein ,
；
Further, in the road surface ponding detection device, the ponding detection model comprises an encoder and a decoder, a feature extraction module in the encoder adopts a Ghost-block module, and the Ghost-block module is used for carrying out linear operation after carrying out convolution calculation on an input image so as to obtain a plurality of feature images.
The pavement water detection device provided by the embodiment of the invention has the same implementation principle and technical effects as those of the embodiment of the method, and for the sake of brief description, the corresponding contents in the embodiment of the method can be referred to for the parts of the embodiment of the device which are not mentioned.
In another aspect, referring to fig. 4, a computer device according to an embodiment of the present invention includes a processor 10, a memory 20, and a computer program 30 stored in the memory and capable of running on the processor, where the processor 10 implements the method for detecting the surface water according to the above description when executing the computer program 30.
The computer device may be, but is not limited to, a personal computer, a server, etc. The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, etc.
The memory 20 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 20 may in some embodiments be an internal storage unit of a computer device, such as a hard disk of the computer device. The memory 20 may also be an external storage device of the computer device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like. Further, the memory 20 may also include both internal storage units and external storage devices of the computer apparatus. The memory 20 may be used not only for storing application software installed in a computer device, various types of data, and the like, but also for temporarily storing data that has been output or is to be output.
Optionally, the computer device may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), a network interface, a communication bus, etc., and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the computer device and for displaying a visual user interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), and is typically used to establish a communication connection between the device and other electronic devices. The communication bus is used to enable connected communication between these components.
It should be noted that the structure shown in fig. 4 does not constitute a limitation of the computer device, and in other embodiments, the computer device may include fewer or more components than shown, or may combine certain components, or may have a different arrangement of components.
The invention also proposes a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method for detecting road surface water as described above.
Those of skill in the art will appreciate that the logic and/or steps represented in the flow diagrams or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus (e.g., a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus). For the purposes of this description, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (10)
1. The pavement ponding detection method based on incremental learning is characterized by comprising the following steps of:
in the process of carrying out water accumulation detection by the water accumulation detection model, periodically establishing an incremental training task aiming at the water accumulation detection model, and periodically acquiring a pavement water accumulation image in the water accumulation detection process to generate a corresponding image sample data set;
constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function;
starting a current incremental training task, and performing incremental training on the ponding detection model by using the target loss function and an image sample data set corresponding to each training task until the model converges or the expected precision is reached;
in the incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task;
and carrying out pavement ponding detection by using the ponding detection model after regular training.
2. The method of claim 1, wherein the target loss function is:
wherein ,
；
3. The method for detecting the accumulated water on the road surface according to claim 1, wherein the accumulated water detection model comprises an encoder and a decoder, a feature extraction module in the encoder adopts a Ghost-block module, and the Ghost-block module is used for carrying out linear operation after carrying out convolution calculation on an input image so as to obtain a plurality of feature images.
4. The method for detecting surface water according to claim 3, wherein the step of performing a linear operation after performing a convolution calculation on the input image by the Ghost-block module comprises:
by means of
The convolution kernel size is +.>
Is>
Performing convolution operation with an input image to obtain m output feature images;
and carrying out linear operation on the output characteristic diagram, wherein the operation formula is as follows:
wherein ,
?>
Output feature map, < >>
Indicate->
The output feature map is +.>
A linear operation, s is an empirical value, +.>
Is an output characteristic diagram after linear operation.
5. A method of detecting surface water as claimed in claim 3 wherein the decoder is arranged to fuse the shallow and deep profiles using long jump connections and a gated attention mechanism to produce a response in the surface water region that is higher than the background region for each pixel of the profile output by each level of the coding layer.
6. The utility model provides a road surface ponding detection device based on increment study which characterized in that includes:
the task building module is used for regularly building an incremental training task aiming at the ponding detection model in the ponding detection process of the ponding detection model, and regularly acquiring a pavement ponding image in the ponding detection process so as to generate a corresponding image sample data set;
the loss function construction module is used for constructing a target loss function, wherein the target loss function comprises a cross entropy loss function and a knowledge distillation loss function;
the incremental training module is used for starting a current incremental training task, and performing incremental training on the ponding detection model by utilizing the target loss function and the image sample data set corresponding to each training task until the model converges or the expected precision is reached;
in the incremental training process, the cross entropy loss function is used for carrying out loss calculation according to the image sample data set corresponding to the current incremental training task, and the knowledge distillation loss function is used for carrying out loss calculation according to the image sample data set corresponding to all the incremental training tasks before the current incremental training task;
and the detection module is used for detecting the accumulated water on the road surface by using the accumulated water detection model after regular training.
7. The surface water detection apparatus as claimed in claim 6, wherein the target loss function is:
wherein ,
；
8. The apparatus of claim 6, wherein the water accumulation detection model includes an encoder and a decoder, and the feature extraction module in the encoder uses a Ghost-block module, and the Ghost-block module is configured to perform a linear operation after performing a convolution calculation on the input image, so as to obtain a plurality of feature maps.
9. A computer device comprising a memory and a processor, the memory storing a program that when executed by the processor implements the method of any of claims 1-5.
10. A computer readable storage medium, on which a program is stored, which program, when being executed by a processor, implements the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310518422.7A CN116311103A (en) | 2023-05-10 | 2023-05-10 | Incremental learning-based pavement ponding detection method, device, medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310518422.7A CN116311103A (en) | 2023-05-10 | 2023-05-10 | Incremental learning-based pavement ponding detection method, device, medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116311103A true CN116311103A (en) | 2023-06-23 |
Family
ID=86799835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310518422.7A Pending CN116311103A (en) | 2023-05-10 | 2023-05-10 | Incremental learning-based pavement ponding detection method, device, medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116311103A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110675415A (en) * | 2019-12-05 | 2020-01-10 | 北京同方软件有限公司 | Road ponding area detection method based on deep learning enhanced example segmentation |
| CN111160155A (en) * | 2019-12-17 | 2020-05-15 | 北京文安智能技术股份有限公司 | Accumulated water detection method and device |
| CN113205142A (en) * | 2021-05-08 | 2021-08-03 | 浙江大学 | Target detection method and device based on incremental learning |
| CN114419381A (en) * | 2022-04-01 | 2022-04-29 | 城云科技（中国）有限公司 | Semantic segmentation method and road ponding detection method and device applying same |
| CN114925828A (en) * | 2022-05-30 | 2022-08-19 | 平安科技（深圳）有限公司 | Increment training method, device, equipment and medium based on multiple classifiers |
| CN115471700A (en) * | 2022-09-16 | 2022-12-13 | 中国科学院计算技术研究所 | Knowledge transmission-based image classification model training method and classification method |
| CN115620000A (en) * | 2022-09-30 | 2023-01-17 | 成都国铁电气设备有限公司 | Intelligent identification method for accumulated water of slag-free ballast bed |
-
2023
- 2023-05-10 CN CN202310518422.7A patent/CN116311103A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110675415A (en) * | 2019-12-05 | 2020-01-10 | 北京同方软件有限公司 | Road ponding area detection method based on deep learning enhanced example segmentation |
| CN111160155A (en) * | 2019-12-17 | 2020-05-15 | 北京文安智能技术股份有限公司 | Accumulated water detection method and device |
| CN113205142A (en) * | 2021-05-08 | 2021-08-03 | 浙江大学 | Target detection method and device based on incremental learning |
| CN114419381A (en) * | 2022-04-01 | 2022-04-29 | 城云科技（中国）有限公司 | Semantic segmentation method and road ponding detection method and device applying same |
| CN114925828A (en) * | 2022-05-30 | 2022-08-19 | 平安科技（深圳）有限公司 | Increment training method, device, equipment and medium based on multiple classifiers |
| CN115471700A (en) * | 2022-09-16 | 2022-12-13 | 中国科学院计算技术研究所 | Knowledge transmission-based image classification model training method and classification method |
| CN115620000A (en) * | 2022-09-30 | 2023-01-17 | 成都国铁电气设备有限公司 | Intelligent identification method for accumulated water of slag-free ballast bed |
Non-Patent Citations (2)
| Title |
|---|
| KAI HAN等: "GhostNet: More Features from Cheap Operations", 《PROCEEDINGS OF THE IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION(CVPR)》, pages 1580 - 1589 * |
| OZAN OKTAY等: "Attention U-Net: Learning Where to Look for the Pancreas", 《ARXIV》, pages 1 - 10 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038409B (en) | Pedestrian detection method | |
| CN110188807B (en) | Tunnel pedestrian target detection method based on cascading super-resolution network and improved Faster R-CNN | |
| CN114120102A (en) | Boundary-optimized remote sensing image semantic segmentation method, device, equipment and medium | |
| CN110060508B (en) | Automatic ship detection method for inland river bridge area | |
| CN102682301B (en) | Adaptation for clear path detection with additional classifiers | |
| Megalingam et al. | Indian traffic sign detection and recognition using deep learning | |
| CN111860439A (en) | Unmanned aerial vehicle inspection image defect detection method, system and equipment | |
| CN111259827B (en) | Automatic detection method and device for water surface floating objects for urban river supervision | |
| Ma et al. | Automatic defogging, deblurring, and real-time segmentation system for sewer pipeline defects | |
| He et al. | A feature fusion method to improve the driving obstacle detection under foggy weather | |
| Vaiyapuri et al. | Automatic Vehicle License Plate Recognition Using Optimal Deep Learning Model. | |
| CN110717886A (en) | Pavement pool detection method based on machine vision in complex environment | |
| Ichim et al. | Road detection and segmentation from aerial images using a CNN based system | |
| Zhang et al. | Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: A new solution to small training datasets | |
| WO2022219402A1 (en) | Semantically accurate super-resolution generative adversarial networks | |
| Saeed et al. | Gravel road classification based on loose gravel using transfer learning | |
| CN114782919A (en) | Road grid map construction method and system with real and simulation data enhanced | |
| Xing et al. | Traffic sign recognition from digital images by using deep learning | |
| Boppana et al. | Comparative analysis of single-stage yolo algorithms for vehicle detection under extreme weather conditions | |
| Gu et al. | Thermal image colorization using Markov decision processes | |
| Munawar et al. | Modern crack detection for bridge infrastructure maintenance using machine learning | |
| Saravanarajan et al. | Improving semantic segmentation under hazy weather for autonomous vehicles using explainable artificial intelligence and adaptive dehazing approach | |
| Baduge et al. | Assessment of crack severity of asphalt pavements using deep learning algorithms and geospatial system | |
| CN116630268A (en) | Road disease detection method, system, equipment and medium | |
| CN116311103A (en) | Incremental learning-based pavement ponding detection method, device, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230623 |