CN117237639A - Pavement crack detection system based on YOLOv8 semantic segmentation - Google Patents
Pavement crack detection system based on YOLOv8 semantic segmentation Download PDFInfo
- Publication number
- CN117237639A CN117237639A CN202311289399.5A CN202311289399A CN117237639A CN 117237639 A CN117237639 A CN 117237639A CN 202311289399 A CN202311289399 A CN 202311289399A CN 117237639 A CN117237639 A CN 117237639A
- Authority
- CN
- China
- Prior art keywords
- data
- crack
- module
- group
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 110
- 230000011218 segmentation Effects 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 40
- 238000012544 monitoring process Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 238000012423 maintenance Methods 0.000 claims abstract description 20
- 238000012360 testing method Methods 0.000 claims description 25
- 238000000034 method Methods 0.000 claims description 16
- 238000012795 verification Methods 0.000 claims description 15
- 239000000284 extract Substances 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000007726 management method Methods 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000013508 migration Methods 0.000 claims description 4
- 230000005012 migration Effects 0.000 claims description 4
- 230000008439 repair process Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 230000001629 suppression Effects 0.000 claims description 2
- 238000007405 data analysis Methods 0.000 abstract 1
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a pavement crack detection system based on YOLOv8 semantic segmentation, which belongs to the technical field of crack detection and comprises an acquisition processing module, a target detection module, a semantic segmentation module, a calibration positioning module, a monitoring alarm module, a storage management module, a training update module, a monitoring platform, a report analysis module, a maintenance scheduling module and an integrated deployment module; the invention can more accurately classify and identify cracks with different types and severity, reduce false alarm rate, enable the system to perform more intelligent data analysis, help maintainers to better know the conditions of the cracks, and thus maintain the cracks more pertinently, improve the usability and fault tolerance of the system, ensure the safety and durability of the data, avoid the risk of data loss, improve the performance of the system, and ensure that the system can still normally operate under high load.
Description
Technical Field
The invention relates to the technical field of crack detection, in particular to a pavement crack detection system based on YOLOv8 semantic segmentation.
Background
Road networks, which are an important component of urban infrastructure, carry traffic for a large number of vehicles and pedestrians. Over time, however, the road surface inevitably becomes cracked, pitted, and damaged, which threatens traffic safety and also leads to an increase in maintenance costs. Therefore, it is important to detect and maintain road cracks in time. Conventional road inspection typically relies on manual inspection, which is time consuming, labor intensive and error prone. In recent years, the development of computer vision and deep learning techniques has provided new solutions for automated crack detection. An automatic crack detection system becomes an important tool for improving road maintenance efficiency and traffic safety.
The existing pavement crack detection system has high false alarm rate, and is not beneficial to help maintenance personnel to better know the state of cracks; in addition, the availability and fault tolerance of the existing pavement crack detection system are low, the safety and durability of data cannot be guaranteed, and the system cannot normally operate under the condition of high load, so that the pavement crack detection system based on the YOLOv8 semantic segmentation is provided.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides a pavement crack detection system based on YOLOv8 semantic segmentation.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a pavement crack detection system based on YOLOv8 semantic segmentation comprises an acquisition processing module, a target detection module, a semantic segmentation module, a calibration positioning module, a monitoring alarm module, a storage management module, a training update module, a monitoring platform, a report analysis module, a maintenance scheduling module and an integrated deployment module;
the acquisition processing module is used for acquiring pavement information from each group of data sources and processing and labeling the acquired data;
the target detection module is used for detecting the position of a crack in the image and generating a boundary box;
the semantic segmentation module is used for further identifying and classifying the cracks and determining the type and severity of the cracks;
the calibration positioning module is used for correcting geographical coordinates of the crack by using a GPS positioning technology;
the monitoring alarm module is used for monitoring the road surface state in real time, and immediately sending an alarm notice if a crack is found;
the storage management module is used for storing and managing the collected data;
the training updating module is used for adjusting the optimization target detection model and the semantic segmentation model;
the monitoring platform is used for providing a visual user interface for maintenance personnel and administrators, and is used for monitoring the road surface state, checking detection results and alarming information;
the report analysis module is used for generating a report and analyzing data;
the maintenance scheduling module is used for making a maintenance plan and scheduling maintenance team modules;
the integrated deployment module is used for deploying the whole system to different hardware platforms and ensuring the normal operation of the system.
As a further scheme of the invention, the specific steps of the acquisition processing module for processing labels are as follows:
step one: after the acquisition processing module acquires original road surface images from each group of data sources, confirming the required image input width and input height according to preset data, and then adjusting each group of original road surface images by an equal proportion adjustment method;
step two: calculating scaling factors in the horizontal direction and the vertical direction, firstly creating a group of blank images, keeping the size of the blank images consistent with the preset size, then determining the position of each group of pixels in the adjusted pavement image in the original pavement image, calculating the gray value of each pixel through a bilinear interpolation method, filling the gray value into the corresponding position in the adjusted image to obtain a final pavement image, and storing the final pavement image on the blank image;
step three: and carrying out normalization processing on the adjusted pavement images, obtaining the position information of the corresponding cracks in each pavement image, distributing a group of unique integer labels for each group of crack class boundary boxes, calculating the information of the boundary boxes, and generating labels for each group of cracks, wherein the labels comprise class labels and boundary box information.
As a further scheme of the invention, the specific calculation formula of the equal proportion adjustment method in the step one is as follows:
W new type ＝H Order of (A) ×M Original source
(2)
H New type ＝W Order of (A) /M Original source
(3)
Wherein M is Original source Representing the original aspect ratio case; w (W) Original source Representing the original width; h Original source Representing the original height; w (W) New type Representing a new width; h New type Representing a new width;
the specific calculation formulas of the scaling factors in the horizontal direction and the vertical direction are as follows:
wherein α represents a horizontal scaling factor; beta represents a vertical scaling factor;
the specific calculation formula of the bilinear interpolation method is as follows:
wherein I is Order of (A) Representing a target pixel value; i Left upper part 、I Lower left 、I Upper right I Lower right Representing the most recent four raw pixel values; (delta) x ,δ y ) Representing the relative position of the target pixel in the nearest four pixel coordinates; c represents a normalization coefficient.
As a further scheme of the invention, the specific steps of crack detection of the target detection module are as follows:
step (1): the target detection module builds a group of YOLOv8 models, uses the weight trained by the COCO data set as the initial weight of the model, then collects the past road surface image data with labels, and then carries out data enhancement on each group of image data and integrates the image data into a road surface crack data set;
step (2): randomly dividing a crack data set into a training set, a verification set and a test set, then training a YOLO v8 model by using training data in the training set, and gradually adjusting weights of the model through a YOLO loss function in the training process to minimize a target detection error;
step (3): after each training is finished, evaluating the performance of the model through a verification set, checking the generalization capability of the model on unseen data, adjusting the fine tuning and super-parameters of the model parameters according to the verification result, evaluating the finally trained model through a test set after the training is finished, acquiring performance indexes, deploying the model into an API or an embedded system through an integrated deployment module, and continuously monitoring the performance of the model in practical application;
step (4): inputting the pavement image subjected to data preprocessing into a YOLOv8 model, wherein the YOLOv8 model extracts input image features through a plurality of convolution layers, pooling layers and full-connection layers and performs target detection to obtain a target detection result;
step (5): performing target detection on feature graphs with different scales, sending the detected features into a bidirectional feature pyramid, performing feature fusion, performing classification regression on fusion results to output a detection frame, and decoding the detection results in the feature graphs into coordinate and size information and category probability of a boundary frame by a model;
step (6): matching each group of detection frames with the detection result in the feature map to determine the best matched detection frame, predicting the object category probability contained in each group of detection frames by using a softmax activation function, and adjusting the position and the size of the boundary frame according to the dimension of the feature map and the dimension of the detection frame;
step (7): after the detection is completed, the detection results of the overlapping and low confidence are removed through non-maximum suppression so as to screen out the final crack detection results, and the YOLOv8 target detection module outputs the detection results containing the crack position and the category information in the form of a boundary frame.
As a further scheme of the invention, the semantic segmentation module identifies and classifies the specific steps as follows:
step I: the semantic segmentation module extracts crack areas from the detection result, extracts corresponding crack images from the original images for each group of crack areas, and then performs feature extraction on the crack images through the convolution and pooling layers by the semantic segmentation model so as to capture detail and texture information of cracks;
step II: decoding the acquired data through a decoder to acquire a pixel-level segmentation result, distributing each group of pixels in the crack image to a crack type or a background type, removing small segmentation areas, filling the segmentation areas to connect broken parts, distributing the crack type to each crack area through counting the number of the crack pixels or calculating the proportion of the crack pixels, and outputting a high-resolution crack segmentation mask;
step III: collecting various knowledge and information related to pavement cracks from the Internet and a crack database, classifying, de-duplicating and screening the collected pavement crack knowledge, identifying and extracting entities in the processed pavement crack knowledge through an NLP technology, extracting corresponding attributes of each entity from the related knowledge information, and establishing a relation among the entities to form connection of pavement crack knowledge maps;
step IV: processing the entity, attribute and relation into a corresponding graph structure in a triplet mode, selecting a proper graph database to store and manage the pavement crack knowledge graph, continuously updating and maintaining the pavement crack knowledge graph, matching the crack entity in the crack area with the corresponding entity in the knowledge graph to analyze and inquire, analyzing and inquiring the crack corresponding type information according to the analysis result, checking with a crack segmentation mask, and if deviation exists, feeding back to a worker for manual checking.
As a further scheme of the invention, the training update module adjusts and optimizes the concrete steps as follows:
step (1): the training updating module collects detection data output by a plurality of groups of YOLOv8 model models or semantic segmentation models, selects a group of observation data from the detection data as verification data, uses the rest of the observation data to fit a group of test models, uses the verification data to verify the accuracy of the test models, and repeatedly calculates the prediction capacity of the prediction models through root mean square errors;
step (2): initializing a parameter range, establishing a data sample, listing all possible data results, dividing the sample, selecting any subset of each group of data as a test set, selecting other subsets as training sets, predicting the test set after training a model, and counting the root mean square error of the test result;
step (3): and replacing the test set with another subset, taking the rest subset as a training set, counting root mean square errors again until all data are predicted once, selecting the corresponding combined parameter with the minimum root mean square error as the optimal parameter in the data interval, and replacing the original parameter of the corresponding model.
As a further scheme of the invention, the data storage of the storage management module comprises the following specific steps:
the first step: dividing each group of system data according to a preset time interval to obtain a plurality of groups of data blocks, generating the identification of each group of data blocks through a hash algorithm, and collecting each group of node information;
and a second step of: selecting proper nodes to store each group of data blocks according to the data block dividing rule and the node load condition through a load balancing algorithm, and after the data block storage is completed, configuring and copying a specified number of data blocks to a plurality of groups of nodes according to the requirements of a system and available resources;
and a third step of: when the data stored by the nodes changes, the data update is transmitted from one node to other nodes through a data synchronization algorithm, then the node operation condition is automatically detected, and the data migration or repair is carried out on the fault node.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, input image features are extracted through a trained YOLOv8 model and target detection is carried out to obtain target detection results, then the model decodes the detection results in the feature map into coordinates and size information of boundary boxes and class probability, each group of detection boxes is matched with the detection results in the feature map to determine the best matched detection boxes, then the final crack detection results are screened out, after a semantic segmentation module extracts crack regions from the detection results, a decoder is used for decoding acquired data to obtain pixel-level segmentation results, each group of pixels in the crack image is allocated to a crack class or background class, then small segmentation regions are removed, the segmentation regions are filled to be connected with broken parts, the crack class is allocated to each crack region through statistics of the number of crack pixels or calculation of the proportion of the crack pixels, meanwhile, a group of crack knowledge is constructed, a crack entity in the crack region is matched with a corresponding entity in the knowledge map to inquire, the crack corresponding type information is inquired according to the result mask, if deviation exists, the crack is fed back manually, the analysis is carried out to the analysis of the crack class information, the crack class is more accurately and the analysis is carried out, the analysis is better, the analysis is carried out, and the analysis is better is carried out on the analysis is carried out on the crack analysis state, and the analysis is better, and the system is better in terms of the analysis is better, and the analysis is better in terms of the conditions.
2. The invention divides each group of system data according to a preset time interval to obtain a plurality of groups of data blocks, then generates the identification of each group of data blocks through a hash algorithm, collects each group of node information, selects proper nodes to store each group of data blocks according to a data block dividing rule and a node load condition, configures and copies a specified number of data blocks to a plurality of groups of nodes according to the requirement and available resources of the system after the data blocks are stored, when the data stored by the nodes are changed, the data updating is transmitted from one node to other nodes through a data synchronization algorithm, then automatically detects the node operation condition, and performs data migration or repair on the fault node, thereby improving the availability and fault tolerance of the system, ensuring the safety and durability of the data, avoiding the risk of data loss, improving the system performance and ensuring that the system can still normally run under the high load condition.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
Fig. 1 is a system block diagram of a pavement crack detection system based on YOLOv8 semantic segmentation.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
Example 1
Referring to fig. 1, a pavement crack detection system based on YOLOv8 semantic segmentation includes an acquisition processing module, a target detection module, a semantic segmentation module, a calibration positioning module, a monitoring alarm module, a storage management module, a training update module, a monitoring platform, a report analysis module, a maintenance scheduling module and an integrated deployment module.
The acquisition processing module is used for acquiring pavement information from each group of data sources and processing and labeling the acquired data.
Specifically, after acquiring an original pavement image from each group of data sources, an acquisition processing module confirms the required image input width and input height according to preset data, adjusts each group of original pavement images through an equal proportion adjustment method, calculates scaling factors in the horizontal direction and the vertical direction, creates a group of blank images, keeps the blank image size consistent with the preset size, determines the position of each group of pixels in the adjusted pavement image in the original pavement image, calculates the gray value of each pixel through a bilinear interpolation method, fills the gray value of each pixel in the adjusted image to the corresponding position in the adjusted image to acquire a final pavement image, stores the final pavement image in the blank image, normalizes the adjusted pavement image, acquires corresponding crack position information in each pavement image, distributes a group of unique integer labels for each group of crack type boundary boxes, calculates the information of the boundary boxes, and generates labels for each group of cracks, including the type labels and the boundary box information.
In this embodiment, the specific calculation formula of the equal proportion adjustment method is as follows:
W new type ＝H Order of (A) ×M Original source
(2)
H New type ＝W Order of (A) /M Original source
(3)
Wherein M is Original source Representing the original aspect ratio case; w (W) Original source Representing the original width; h Original source Representing the original height; w (W) New type Representing a new width; h New type Representing a new width;
the scaling factors in the horizontal and vertical directions are specifically calculated as follows:
wherein α represents a horizontal scaling factor; beta represents a vertical scaling factor;
the specific calculation formula of the bilinear interpolation method is as follows:
wherein I is Order of (A) Representing a target pixel value; i Left upper part 、I Lower left 、I Upper right I Lower right Representing the most recent four raw pixel values; (delta) x ,δ y ) Representing the relative position of the target pixel in the nearest four pixel coordinates; c represents a normalization coefficient.
The object detection module is used for detecting the position of a crack in the image and generating a boundary box.
Specifically, the target detection module builds a group of YOLOv8 models, uses the weight trained by the COCO data set as the initial weight of the model, then collects the past pavement image data with labels, performs data enhancement on each group of image data and integrates the image data into pavement crack data sets, randomly divides the crack data sets into a training set, a verification set and a test set, then uses training data in the training set to train the YOLOv8 models, gradually adjusts the weight by a YOLO loss function to minimize target detection errors in the training process, evaluates the model performance by the verification set after each training, checks the generalization capability of the model on unseen data, performs fine adjustment of model parameters and super parameter adjustment according to the verification result, evaluates the final trained model by the test set after the training is finished to obtain performance indexes, the model is deployed as an API or an embedded system through an integrated deployment module, the performance of the model in practical application is continuously monitored, a pavement image subjected to data preprocessing is input into a YOLOv8 model, the YOLOv8 model extracts input image features through a plurality of convolution layers, pooling layers and full connection layers and performs target detection to obtain target detection results, target detection is performed on feature images with different scales, the detected features are sent into a bidirectional feature pyramid to perform feature fusion, the fusion results are subjected to classification regression to output detection frames, the model decodes the detection results in the feature images into coordinates and size information of boundary frames and class probabilities, each group of detection frames is matched with the detection results in the feature images to determine the best matched detection frames, the class probabilities of objects contained in each group of detection frames are predicted by using a softmax activation function, and then the position and the size of the boundary frame are adjusted according to the dimension of the feature map and the dimension of the detection frame, after detection is completed, the detection result of removing the overlapping and the detection result of low confidence coefficient are restrained by a non-maximum value so as to screen out a final crack detection result, and the YOLOv8 target detection module outputs the detection result containing the crack position and the category information in the form of the boundary frame.
The semantic segmentation module is used for further identifying and classifying the cracks and determining the type and severity of the cracks.
Specifically, the semantic segmentation module extracts crack areas from detection results, extracts corresponding crack images from original images for each group of crack areas, then the semantic segmentation model performs feature extraction on the crack images through a convolution layer and a pooling layer to capture details and texture information of cracks, decodes the obtained data through a decoder to obtain pixel-level segmentation results, distributes each group of pixels in the crack images to crack categories or background categories, removes small segmentation areas, fills the segmentation areas to connect broken parts, distributes the crack categories to each crack area through counting the number of the crack pixels or calculating the proportion of the crack pixels, simultaneously outputs high-resolution crack segmentation masks, collects various knowledge and information related to the pavement cracks from the Internet and a crack database, classifies, de-duplicates and screens the collected pavement crack knowledge, identifies and extracts entities in the pavement crack knowledge after processing through an NLP technology, extracts corresponding attributes of each entity from the related knowledge information, establishes a relation between the entities to form connection of the pavement crack knowledge, analyzes the entity, manages the corresponding relation between the entity and the map, analyzes the crack map by adopting a form of a triplet, analyzes the corresponding relation between the map and the crack map, and the map is properly analyzed and the map is analyzed by a map, and the map is analyzed and the map is then the map is analyzed and the map is not matched with the map and the map is analyzed if the map is matched with the map and the map is analyzed.
The calibration positioning module is used for correcting geographical coordinates of the crack by using a GPS positioning technology; the monitoring alarm module is used for monitoring the road surface state in real time, and if cracks are found, an alarm notification is immediately sent.
Example 2
Referring to fig. 1, a pavement crack detection system based on YOLOv8 semantic segmentation includes an acquisition processing module, a target detection module, a semantic segmentation module, a calibration positioning module, a monitoring alarm module, a storage management module, a training update module, a monitoring platform, a report analysis module, a maintenance scheduling module and an integrated deployment module.
The storage management module is used for storing and managing the collected data.
Specifically, the training update module collects detection data output by a plurality of groups of YOLOv8 model models or semantic segmentation models, selects one group of observation data from the detection data as verification data, uses the rest of observation data to simulate a group of test models, uses the verification data to verify the accuracy of the test models, repeatedly calculates the prediction capacity of the prediction models through root mean square errors for a plurality of times, initializes a parameter range, establishes data samples, lists all possible data results, divides the samples, selects any subset as a test set for each group of data, predicts the test set after the test model is trained, counts root mean square errors of the test results, replaces the test set with another subset, then takes the rest subset as a training set, counts root mean square errors again until all data are predicted once, and selects corresponding combination parameters, namely optimal parameters in a data interval, and replaces original parameters of the corresponding models.
The training updating module is used for adjusting the optimization target detection model and the semantic segmentation model.
Specifically, each group of system data is segmented according to a preset time interval to obtain a plurality of groups of data blocks, then the identifications of each group of data blocks are generated through a hash algorithm, each group of node information is collected, each group of data blocks are stored through selecting proper nodes according to a data block division rule and a node load condition and through a load balancing algorithm, after the data blocks are stored, a specified number of data blocks are configured and copied to a plurality of groups of nodes according to the requirements of the system and available resources, when the data stored by the nodes are changed, the data update is transmitted to other nodes from one node through a data synchronization algorithm, then the node operation condition is automatically detected, and the data migration or repair is carried out on the fault nodes.
The monitoring platform is used for providing a visual user interface for maintenance personnel and administrators, and is used for monitoring the road surface state, checking detection results and alarming information; the report analysis module is used for generating a report and analyzing data; the maintenance scheduling module is used for making a maintenance plan and scheduling maintenance team; the integrated deployment module is used for deploying the whole system to different hardware platforms and ensuring the normal operation of the system.
Claims (7)
1. The pavement crack detection system based on the YOLOv8 semantic segmentation is characterized by comprising an acquisition processing module, a target detection module, a semantic segmentation module, a calibration positioning module, a monitoring alarm module, a storage management module, a training update module, a monitoring platform, a report analysis module, a maintenance scheduling module and an integrated deployment module;
the acquisition processing module is used for acquiring pavement information from each group of data sources and processing and labeling the acquired data;
the target detection module is used for detecting the position of a crack in the image and generating a boundary box;
the semantic segmentation module is used for further identifying and classifying the cracks and determining the type and severity of the cracks;
the calibration positioning module is used for correcting geographical coordinates of the crack by using a GPS positioning technology;
the monitoring alarm module is used for monitoring the road surface state in real time, and immediately sending an alarm notice if a crack is found;
the storage management module is used for storing and managing the collected data;
the training updating module is used for adjusting the optimization target detection model and the semantic segmentation model;
the monitoring platform is used for providing a visual user interface for maintenance personnel and administrators, and is used for monitoring the road surface state, checking detection results and alarming information;
the report analysis module is used for generating a report and analyzing data;
the maintenance scheduling module is used for making a maintenance plan and scheduling maintenance team modules;
the integrated deployment module is used for deploying the whole system to different hardware platforms and ensuring the normal operation of the system.
2. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 1, wherein the specific steps of the acquisition processing module processing labels are as follows:
step one: after the acquisition processing module acquires original road surface images from each group of data sources, confirming the required image input width and input height according to preset data, and then adjusting each group of original road surface images by an equal proportion adjustment method;
step two: calculating scaling factors in the horizontal direction and the vertical direction, firstly creating a group of blank images, keeping the size of the blank images consistent with the preset size, then determining the position of each group of pixels in the adjusted pavement image in the original pavement image, calculating the gray value of each pixel through a bilinear interpolation method, filling the gray value into the corresponding position in the adjusted image to obtain a final pavement image, and storing the final pavement image on the blank image;
step three: and carrying out normalization processing on the adjusted pavement images, obtaining the position information of the corresponding cracks in each pavement image, distributing a group of unique integer labels for each group of crack class boundary boxes, calculating the information of the boundary boxes, and generating labels for each group of cracks, wherein the labels comprise class labels and boundary box information.
3. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 2, wherein the specific calculation formula of the equal proportion adjustment method in the step one is as follows:
W new type ＝H Order of (A) ×M Original source
(2)
H New type ＝W Order of (A) /M Original source
(3)
Wherein M is Original source Representing the original aspect ratio case; w (W) Original source Representing the original width; h Original source Representing the original height; w (W) New type Representing a new width; h New type Representing a new width;
the specific calculation formulas of the scaling factors in the horizontal direction and the vertical direction are as follows:
wherein α represents a horizontal scaling factor; beta represents a vertical scaling factor;
the specific calculation formula of the bilinear interpolation method is as follows:
wherein I is Order of (A) Representing a target pixel value; i Left upper part 、I Lower left 、I Upper right I Lower right Representing the most recent four raw pixel values; (delta) x ,δ y ) Representing the relative position of the target pixel in the nearest four pixel coordinates; c represents a normalization coefficient.
4. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 2, wherein the specific steps of the target detection module crack detection are as follows:
step (1): the target detection module builds a group of YOLOv8 models, uses the weight trained by the COCO data set as the initial weight of the model, then collects the past road surface image data with labels, and then carries out data enhancement on each group of image data and integrates the image data into a road surface crack data set;
step (2): randomly dividing a crack data set into a training set, a verification set and a test set, then training a YOLO v8 model by using training data in the training set, and gradually adjusting weights of the model through a YOLO loss function in the training process to minimize a target detection error;
step (3): after each training is finished, evaluating the performance of the model through a verification set, checking the generalization capability of the model on unseen data, adjusting the fine tuning and super-parameters of the model parameters according to the verification result, evaluating the finally trained model through a test set after the training is finished, acquiring performance indexes, deploying the model into an API or an embedded system through an integrated deployment module, and continuously monitoring the performance of the model in practical application;
step (4): inputting the pavement image subjected to data preprocessing into a YOLOv8 model, wherein the YOLOv8 model extracts input image features through a plurality of convolution layers, pooling layers and full-connection layers and performs target detection to obtain a target detection result;
step (5): performing target detection on feature graphs with different scales, sending the detected features into a bidirectional feature pyramid, performing feature fusion, performing classification regression on fusion results to output a detection frame, and decoding the detection results in the feature graphs into coordinate and size information and category probability of a boundary frame by a model;
step (6): matching each group of detection frames with the detection result in the feature map to determine the best matched detection frame, predicting the object category probability contained in each group of detection frames by using a softmax activation function, and adjusting the position and the size of the boundary frame according to the dimension of the feature map and the dimension of the detection frame;
step (7): after the detection is completed, the detection results of the overlapping and low confidence are removed through non-maximum suppression so as to screen out the final crack detection results, and the YOLOv8 target detection module outputs the detection results containing the crack position and the category information in the form of a boundary frame.
5. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 4, wherein the semantic segmentation module identifies and classifies the specific steps as follows:
step I: the semantic segmentation module extracts crack areas from the detection result, extracts corresponding crack images from the original images for each group of crack areas, and then performs feature extraction on the crack images through the convolution and pooling layers by the semantic segmentation model so as to capture detail and texture information of cracks;
step II: decoding the acquired data through a decoder to acquire a pixel-level segmentation result, distributing each group of pixels in the crack image to a crack type or a background type, removing small segmentation areas, filling the segmentation areas to connect broken parts, distributing the crack type to each crack area through counting the number of the crack pixels or calculating the proportion of the crack pixels, and outputting a high-resolution crack segmentation mask;
step III: collecting various knowledge and information related to pavement cracks from the Internet and a crack database, classifying, de-duplicating and screening the collected pavement crack knowledge, identifying and extracting entities in the processed pavement crack knowledge through an NLP technology, extracting corresponding attributes of each entity from the related knowledge information, and establishing a relation among the entities to form connection of pavement crack knowledge maps;
step IV: processing the entity, attribute and relation into a corresponding graph structure in a triplet mode, selecting a proper graph database to store and manage the pavement crack knowledge graph, continuously updating and maintaining the pavement crack knowledge graph, matching the crack entity in the crack area with the corresponding entity in the knowledge graph to analyze and inquire, analyzing and inquiring the crack corresponding type information according to the analysis result, checking with a crack segmentation mask, and if deviation exists, feeding back to a worker for manual checking.
6. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 5, wherein the training update module adjusts and optimizes the concrete steps as follows:
step (1): the training updating module collects detection data output by a plurality of groups of YOLOv8 model models or semantic segmentation models, selects a group of observation data from the detection data as verification data, uses the rest of the observation data to fit a group of test models, uses the verification data to verify the accuracy of the test models, and repeatedly calculates the prediction capacity of the prediction models through root mean square errors;
step (2): initializing a parameter range, establishing a data sample, listing all possible data results, dividing the sample, selecting any subset of each group of data as a test set, selecting other subsets as training sets, predicting the test set after training a model, and counting the root mean square error of the test result;
step (3): and replacing the test set with another subset, taking the rest subset as a training set, counting root mean square errors again until all data are predicted once, selecting the corresponding combined parameter with the minimum root mean square error as the optimal parameter in the data interval, and replacing the original parameter of the corresponding model.
7. The pavement crack detection system based on YOLOv8 semantic segmentation according to claim 1, wherein the storage management module data storage specifically comprises the following steps:
the first step: dividing each group of system data according to a preset time interval to obtain a plurality of groups of data blocks, generating the identification of each group of data blocks through a hash algorithm, and collecting each group of node information;
and a second step of: selecting proper nodes to store each group of data blocks according to the data block dividing rule and the node load condition through a load balancing algorithm, and after the data block storage is completed, configuring and copying a specified number of data blocks to a plurality of groups of nodes according to the requirements of a system and available resources;
and a third step of: when the data stored by the nodes changes, the data update is transmitted from one node to other nodes through a data synchronization algorithm, then the node operation condition is automatically detected, and the data migration or repair is carried out on the fault node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289399.5A CN117237639A (en) | 2023-10-07 | 2023-10-07 | Pavement crack detection system based on YOLOv8 semantic segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289399.5A CN117237639A (en) | 2023-10-07 | 2023-10-07 | Pavement crack detection system based on YOLOv8 semantic segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237639A true CN117237639A (en) | 2023-12-15 |
Family
ID=89087824
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311289399.5A Pending CN117237639A (en) | 2023-10-07 | 2023-10-07 | Pavement crack detection system based on YOLOv8 semantic segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237639A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952977A (en) * | 2024-03-27 | 2024-04-30 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
-
2023
- 2023-10-07 CN CN202311289399.5A patent/CN117237639A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952977A (en) * | 2024-03-27 | 2024-04-30 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
| CN117952977B (en) * | 2024-03-27 | 2024-06-04 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| US20210319561A1 (en) | Image segmentation method and system for pavement disease based on deep learning | |
| CN109165582B (en) | Urban street garbage detection and cleanliness assessment method | |
| CN112287807B (en) | Remote sensing image road extraction method based on multi-branch pyramid neural network | |
| CN109859163A (en) | A kind of LCD defect inspection method based on feature pyramid convolutional neural networks | |
| CN108961238A (en) | Display screen quality determining method, device, electronic equipment and storage medium | |
| CN108921839A (en) | Continuous casting billet quality detection method, device, electronic equipment and storage medium | |
| CN117237639A (en) | Pavement crack detection system based on YOLOv8 semantic segmentation | |
| CN111160432A (en) | Automatic classification method and system for panel production defects | |
| CN111539432A (en) | Method for extracting urban road by using multi-source data to assist remote sensing image | |
| CN117271683A (en) | Intelligent analysis and evaluation method for mapping data | |
| CN114862832A (en) | Method, device and equipment for optimizing defect detection model and storage medium | |
| CN115205255A (en) | Stone automatic grading method and system based on deep learning | |
| CN117409341B (en) | Unmanned aerial vehicle illumination-based image analysis method and system | |
| CN113971666A (en) | Power transmission line machine inspection image self-adaptive identification method based on depth target detection | |
| CN115830302B (en) | Multi-scale feature extraction fusion power distribution network equipment positioning identification method | |
| CN117197085A (en) | Road rapid-inspection image pavement disease detection method based on improved YOLOv8 network | |
| US20230314169A1 (en) | Method and apparatus for generating map data, and non-transitory computer-readable storage medium | |
| CN115170783A (en) | Expressway pavement crack detection method using street view image | |
| CN110163081A (en) | SSD-based real-time regional intrusion detection method, system and storage medium | |
| CN114140707A (en) | Power grid fault inspection method | |
| Yan et al. | An automatic pavement crack detection system with FocusCrack Dataset | |
| Ferrer-Espinoza et al. | Evaluation of the use of Cascade Detection algorithms based on Machine Learning for Crack Detection in asphalt Pavements | |
| CN113361968A (en) | Power grid infrastructure worker safety risk assessment method based on artificial intelligence and big data | |
| CN116188973B (en) | Crack detection method based on cognitive generation mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |