CN116052110B - Intelligent positioning method and system for pavement marking defects - Google Patents
Intelligent positioning method and system for pavement marking defects Download PDFInfo
- Publication number
- CN116052110B CN116052110B CN202310310272.0A CN202310310272A CN116052110B CN 116052110 B CN116052110 B CN 116052110B CN 202310310272 A CN202310310272 A CN 202310310272A CN 116052110 B CN116052110 B CN 116052110B
- Authority
- CN
- China
- Prior art keywords
- marking
- road
- feature
- pavement marking
- foreground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000001514 detection method Methods 0.000 claims abstract description 81
- 230000007246 mechanism Effects 0.000 claims abstract description 46
- 238000000605 extraction Methods 0.000 claims abstract description 16
- 238000013528 artificial neural network Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 25
- 238000005299 abrasion Methods 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 10
- 238000012216 screening Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 101100481876 Danio rerio pbk gene Proteins 0.000 claims description 3
- 101100481878 Mus musculus Pbk gene Proteins 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 abstract description 6
- 238000012423 maintenance Methods 0.000 description 8
- 230000006872 improvement Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000008439 repair process Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an intelligent positioning method and system for pavement marking defects, relates to the technical field of computer vision image target detection, and solves the problem that the existing target detection algorithm is inaccurate in pavement marking defect detection and positioning; when the convolutional neural network is used for constructing a model, the central Net of the target detection network is used as a base line, after the deep characteristic information is extracted by utilizing the residual neural network, a self-fusion coding mechanism is embedded in the key characteristic refinement module, the self-fusion coding mechanism takes account of long-distance relation modeling of image characteristic information, deep extraction of local characteristic details and spatial position information coding of the target, and simultaneously, the deep high semantic information and shallow characteristic detail information of the network model are fused better, so that the defect of the convolutional neural network on global modeling is overcome.
Description
Technical Field
The invention relates to the technical field of computer vision image target detection, in particular to an intelligent positioning method and system for pavement marking defects.
Background
With the continuous perfection of road infrastructure, the large-scale operation maintenance and management work of expressways, urban roads and rural roads is gradually scheduled. The road marking is taken as an important aspect of road traffic assets and also as an important marker for ensuring the safety of road traffic, and plays a role in the aspects of lane guidance, ordered traffic and the like. Periodic inspection and maintenance of the pavement markings and maintenance of the integrity of the pavement markings is an essential task.
The type of damage to a conventional pavement marking is mainly manifested by the loss of pavement marking (loss of original contour) and wear of pavement marking (retention of original contour, but dullness of color, and severe wear). Traditional manual detection methods are time-consuming, laborious, inefficient and of low accuracy. Therefore, many scholars engaged in road detection develop a series of road marking detection methods based on traditional detection algorithms, including image threshold segmentation, filters, road parametric modeling, and the like. However, these conventional detection models are limited to specific working environments, are greatly affected by the hyper-parameters of the models, have poor generalization ability, and have no capability of being three times against one another. Therefore, the traditional detection algorithm has great disadvantages in the positioning of pavement marking defects, and is difficult to be widely applied to practical engineering.
With the continuous development of artificial intelligence deep learning, various target detection algorithms based on convolutional neural networks have been successfully applied to face recognition, image similarity comparison, bridge crack detection, road disease detection, precise instrument damage detection and identification and the like. In the field of general image recognition, the conventional target detection algorithm has a better recognition effect and generalization capability than the conventional algorithm model, but still has a further research and development space in the accurate positioning of pavement marking defects. Because of the variety of weather, changes in illumination, and the effects of disturbances on various roadways involved in a real road detection scenario, this makes locating pavement marking defects a very challenging task. It is further noted that, in the case of a missing reticle, the original white feature of the reticle is destroyed, and global feature information must be considered to detect such missing reticle. However, through experimental verification, the existing target detection algorithm is very easy to generate a large number of misidentification and inaccurate target boundary positioning on pavement marking defect detection positioning. Even though the existing target detection algorithm mostly adopts a mechanism for increasing attention and a multi-scale modeling method to improve the detection accuracy, the long-distance overall modeling capability is still slightly lacking.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the invention provides an intelligent positioning method and system for pavement marking defects based on a vehicle-mounted intelligent mobile phone, which solve the technical problems that the existing target detection algorithm is applied to the situation that regression bounding boxes are positioned inaccurately in the pavement marking defect detection process and lacks the capability of overall construction feature information, long-distance modeling and spatial position information extraction.
The invention is realized by the following technical scheme:
the intelligent positioning method for the pavement marking defect is characterized by comprising the following steps of:
step 1: constructing a pavement marking damage target detection algorithm model;
step 2: acquiring road foreground 2D image data, and preprocessing the acquired foreground 2D image data;
step 3: inputting the preprocessed road foreground image data into a road marking damage target detection algorithm model to obtain the location of road marking loss and abrasion;
in the step 1, the pavement marking damage target detection algorithm model takes a convolutional target detection network CenterNet as a base line, and a self-fusion coding mechanism is embedded in a key feature refinement module after deep feature information is extracted by using a residual neural network.
Preferably, the pretreatment is specifically as follows:
normalization processing is performed on the foreground 2D image data according to the following steps:
wherein i, j represent the row number and column number of the picture respectively; channel represents the channel of the picture, and the value is 0,1 and 2; c (i, j, channel) represents the pixel value of the original image corresponding to a certain channel; p (i, j, channel) represents the pixel value of the processed picture on a certain channel, and P (i, j, channel) is between (-1, 1); the three values in mean and std correspond to the mean and variance of the three channels R, G, B, respectively.
Preferably, the step 1 specifically includes the following steps:
step 1.1: collecting road foreground 2D image data by using a vehicle-mounted smart phone, and establishing a road foreground 2D image database;
step 1.2: constructing coordinate position data of pavement marking missing and abrasion corresponding to the collected 2D image data of the road foreground, and forming pavement marking defect truth value data;
step 1.3: scaling all acquired road foreground 2D image data to a uniform size, and scaling true value data of pavement marking defects based on scaling factors of the road foreground 2D images;
step 1.4: and training a pavement marking breakage target detection algorithm model based on the road prospect 2D image data of the step 1.3 and the truth value data of the pavement marking breakage of the step 1.3.
Preferably, the step 1.1 specifically includes:
step 1.11: acquiring road foreground 2D image data of expressways, urban roads and rural roads, constructing a large data platform of the road foreground 2D image, selecting samples from the large data platform to form a training data sample base of a training algorithm model,
the training data sample library comprises: a highway marking missing sample, a highway marking wear sample, an urban road marking missing sample, an urban road marking wear sample, a rural road marking missing sample, and a rural road marking wear sample;
step 1.12: selecting part of various marking wear sample data from a constructed road foreground 2D image big data platform by adopting a pre-training model screening method to pre-train a road marking damage target detection algorithm model to obtain an algorithm model with classification recognition capability;
step 1.13: inputting the collected expressway marking missing sample, expressway marking abrasion sample, urban road marking missing sample, urban road marking abrasion sample, rural road marking missing sample and rural road marking abrasion sample into the algorithm model with classification recognition capability obtained in the step 1.12 for N times to obtain classification information of pavement marking defect in each piece of image data, wherein the probability of marking missing, marking abrasion and background of each piece of image is respectively recorded as P i1 、P i2 、P i3 I=0, 1,2,3, … …, X is the total number of various types of reticle samples collected;
step 1.14: taking various marked line samples max { P i1 ,P i2 ,P i3 The method comprises the steps of obtaining the number of various marking samples respectively, wherein the number is the sample type of a single image;
step 1.15: and randomly extracting the same amount of sample data from each marked line sample database by using a random sampling method so as to achieve the purpose of balancing positive and negative samples, thereby constructing a road prospect 2D image database.
Preferably, the step 1.2 specifically includes:
and constructing coordinate position data of pavement marking missing and abrasion corresponding to the road foreground 2D image data based on the road foreground 2D image database, and forming pavement marking defect truth value data.
Preferably, the step 1.3 specifically includes:
and scaling all acquired road foreground 2D image data into 640pixel multiplied by 3 by adopting a bilinear interpolation method, and scaling true value data of pavement marking defects by adopting a conventional scaling method based on the scaling factors of the road foreground 2D image.
Preferably, the step 1.4 specifically includes the following steps:
step 1.41: road foreground 2D image data scaled to 640 pixels by 3 is input to a feature extraction network
In the ResNet101, after feature extraction of the ResNet101, a preliminary feature layer with the size of 20 pixels multiplied by 2048 is obtained;
step 1.42: the 20pixel 2048 extracted for ResNet101 with step size 2 4pixel 256 Conv2DTranspose, 4pixel 128 Conv2DTranspose, 4pixel 64 Conv2DTranspose, respectively, was preliminary
The feature layer is subjected to up-sampling treatment to obtain a 160pixel×160pixel×64 key feature layer; wherein 4pixel×4pixel× 256 Conv2DTranspose represents a transposed convolutional layer having a convolutional kernel size of 4×4 and a convolutional kernel number of 256;
step 1.43: modeling the long-distance relation of image feature information, deep extraction of local feature details and spatial position information coding of a target are carried out on the key feature layer information based on a self-fusion coding mechanism, so that a refined graph is obtained;
step 1.44: and removing a noise frame based on a regression decoding mechanism, extracting an effective target frame from the depth of the thinned image, constructing optimal coordinate position information, and obtaining a pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information.
Preferably, the step 1.43 specifically includes the following steps:
step 1.431: the method comprises the steps of respectively using three full-connection layer Dense to obtain Query, key and Value matrix feature vector representations of the Key feature layer for the input Key feature layer;
step 1.432: inputting the input key feature layer into a spatial feature information coding and thinning branch module to obtain a thinned spatial feature map representation;
step 1.433: performing matrix multiplication operation on the Query and Key values obtained in the step 1.431, and multiplying the obtained result by a Scale factor Scale to obtain a feature map;
step 1.434: converting all elements in the matrix of the feature map obtained in the step 1.433 into relative probabilities among different elements by using a softMax activation function to obtain a matrix feature layer with probability distribution;
step 1.435: performing matrix multiplication on the Value obtained in the step 1.431 and the matrix feature layer with probability distribution obtained in the step 1.434 to obtain a further feature refinement result;
step 1.436: and (3) accessing the result obtained in the step (1.435) into a full connection layer, and performing splicing fusion with the 1.432 refined space feature map to obtain a final output refined map.
Preferably, the step 1.44 specifically includes the following steps:
step 1.441: extracting a thermodynamic diagram detection head, a regression frame width and height detection head and a regression frame center offset detection head of a regression decoding mechanism into feature layers of a refined diagram, respectively inputting the feature layers into a light-weight decoding module, wherein the light-weight decoding module consists of a 5pixel×5pixel×64 depth separable convolution layer, layer regularization and a 5pixel×5pixel×64 depth separable convolution layer so as to enhance the regression capability of a decoding stage on a target position;
step 1.442: performing a 3×3 max pooling operation on the feature layer refined in 1.441 to detect whether the value of the current hot spot is larger than all eight neighboring points;
step 1.443: screening K target frames meeting the requirements from the results in 1.442 by utilizing a TopK algorithm;
step 1.444: and further screening the K target frames meeting the requirements obtained in the step 1.443 by using a confidence threshold value to obtain optimal coordinate position information, and obtaining the pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information.
A pavement marking defect intelligent positioning system of a vehicle-mounted intelligent mobile phone comprises an acquisition module, a model construction module and an intelligent identification module;
the acquisition module is used for collecting road foreground 2D image data of a target road section in real time;
the model construction module is used for constructing an intelligent algorithm model;
the intelligent recognition module is used for carrying out unified operation of image size and normalization processing of images on the acquired road foreground 2D data, conveying the processed image data to a target detection algorithm, and carrying out regression to obtain actual position information of road marking damage so as to finish positioning of road marking loss and abrasion.
Compared with the prior art, the invention has the following advantages and beneficial effects:
according to the pavement marking defect intelligent positioning method based on the vehicle-mounted intelligent mobile phone, when a model is built through a convolutional neural network, a target detection network center Net is used as a base line, after deep characteristic information is extracted through a residual neural network, a self-fusion coding mechanism is embedded into a key characteristic refinement module, the self-fusion coding mechanism is an improvement of a self-attention mechanism in the transducer, and a spatial characteristic information coding and refinement branch module is added on the basis of the original self-attention mechanism, so that the self-fusion coding mechanism takes into consideration long-distance relation modeling of image characteristic information, deep extraction of local characteristic details and spatial position information coding of a target, and meanwhile, deep high semantic information and shallow characteristic detail information of the network model are better fused, and the defect of the convolutional neural network in global modeling is overcome. In addition, the self-fusion coding mechanism combines the advantages of the self-attention mechanism, is good at capturing long-distance dependence between image characteristic information, is more beneficial to searching the similarity relationship between sub-pixels in the image, and further improves the detection and positioning effects of the whole model. After the key feature detection head is added with a regression decoding mechanism, a lightweight decoding module is designed in the regression decoding mechanism, and the module has the characteristics of high reasoning speed, can effectively remove noise frames and extract an effective target frame in depth, so that the damaged coordinate position of the pavement marking can be positioned more accurately and effectively, and reliable and effective positioning information is provided for maintenance and repair of the later-stage marking.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the drawings that are needed in the examples will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and that other related drawings may be obtained from these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a flow chart of an intelligent pavement marking defect positioning method based on a vehicle-mounted smart phone;
FIG. 2 is a schematic diagram of an image object detection positioning model structure according to the present invention;
FIG. 3 is a schematic diagram of a structure of a regression decoding mechanism;
FIG. 4 is a schematic diagram of a feature refinement module self-fusion coding mechanism;
Detailed Description
The following further details the invention with reference to examples and drawings in order to make the objects, technical solutions and advantages of the invention more apparent, the illustrative embodiments of the invention and their description are only for explaining the invention and not limiting it.
Example 1: the embodiment provides an intelligent positioning method for pavement marking defects, which is shown in fig. 1 and 2 and comprises the following steps:
step 1: constructing a pavement marking damage target detection algorithm model;
in the step 1, the target detection intelligent algorithm model with damaged pavement marking takes a convolutional target detection network center Net as a base line, after deep characteristic information is extracted by utilizing a residual neural network, a self-fusion coding mechanism is embedded in a key characteristic refinement module, the self-fusion coding mechanism is an improvement of a self-attention mechanism in a transducer, and a spatial characteristic information coding and refinement branch module is added on the basis of the original self-attention mechanism, so that the self-fusion coding mechanism combines long-distance relation modeling of image characteristic information, deep extraction of local characteristic details and spatial position information coding of targets, and simultaneously better fuses deep high semantic information and shallow characteristic detail information of the network model. After the key feature detection head is added with a regression decoding mechanism, a lightweight decoding module is designed in the regression decoding mechanism, and the module has the characteristics of high reasoning speed, can effectively remove noise frames and extract an effective target frame in depth, so that the damaged coordinate position of the pavement marking can be positioned more accurately and effectively, and reliable and effective positioning information is provided for maintenance and repair of the later-stage marking.
The construction method of the pavement marking damage target detection algorithm model specifically comprises the following steps:
step 1.1: collecting road foreground 2D image data by using a vehicle-mounted smart phone, and establishing a road foreground 2D image database;
the step 1 specifically includes:
step 1.11: acquiring road foreground 2D image data of expressways, urban roads and rural roads, constructing a large data platform of the road foreground 2D image, selecting samples from the large data platform to form a training data sample base of a training algorithm model,
the training data sample library comprises: a highway marking missing sample, a highway marking wear sample, an urban road marking missing sample, an urban road marking wear sample, a rural road marking missing sample, and a rural road marking wear sample; wherein the marked lines comprise lane lines, straight marked lines, turning marked lines, speed limiting marked lines, warning marked lines and the like;
step 1.12: selecting part (at least 100) of various marking abrasion sample data from a constructed road foreground 2D image big data platform by adopting a pre-training model screening method, and pre-training a road marking damage target detection algorithm model to obtain an algorithm model with classification recognition capability;
the purpose of this step is not to accurately locate, but to let the algorithm model have a certain classification capability to efficiently construct balanced positive and negative sample data;
step 1.13: inputting the collected expressway marking missing sample, expressway marking abrasion sample, urban road marking missing sample, urban road marking abrasion sample, rural road marking missing sample and rural road marking abrasion sample into the algorithm model with classification recognition capability obtained in the step 1.12 for N times to obtain classification information of pavement marking defect in each piece of image data, wherein each piece of image is marking missing and marking abrasionThe probabilities of the background are respectively marked as P i1 、P i2 、P i3 I=0, 1,2,3, … …, X is the total number of various types of reticle samples collected;
step 1.14: taking various marked line samples max { P i1 ,P i2 ,P i3 The method comprises the steps of obtaining the number of various marking samples respectively, wherein the number is the sample type of a single image;
step 1.15: and randomly extracting the same amount of sample data from each marked line sample database by using a random sampling method so as to achieve the purpose of balancing positive and negative samples, thereby constructing a road prospect 2D image database.
For the target detection model, the data involved in training determines the performance of the model after training. In short, the size of the data volume and the distribution of positive and negative samples in the data set have a direct impact on the training performance of the model. When the data volume is insufficient, a problem of insufficient model training is liable to occur, which may lead to the occurrence of a problem of model under-fitting. When the data with similar data distribution is too much, the problem of model overfitting can be caused, and the generalization capability of the model is poor. For the training data sample library, it should contain multiple modes, multiple scenes, and various kinds of respective scene modes as much as possible, including but not limited to, a sunny scene, a rainy scene, a cloudy scene, a high-speed scene, an urban road scene, and a rural road scene. However, in a practical scenario, a scenario mode is very different in sample data, it is difficult to accurately classify the sample data by using a manual method, the proportion of the sample data amount of each type is difficult to determine, and it is difficult to achieve a sample data set covering all the scenario mode types, usually only a certain amount of sample data can be collected, and then training data is selected by using a manual selection method. The main disadvantage of this method is that it depends entirely on subjective performance of the person, the results chosen by different persons are different, there is no fixed measurement, and the manually chosen samples do not necessarily meet the requirement for positive and negative sample balance in model training. Furthermore, in the case of large sample sets, manual selection is very time-consuming and laborious. Therefore, the embodiment is based on a pre-training model screening method, only a small amount of manpower is needed in the early stage, and the model preliminary classification performance can be used in the later stage to quickly construct a training data sample library with balanced positive and negative samples;
step 1.2: and constructing coordinate position data of pavement marking missing and abrasion corresponding to the road foreground 2D image data based on the road foreground 2D image database, and forming pavement marking defect truth value data.
Step 1.3: scaling all acquired road foreground 2D image data to a uniform size, and scaling true value data of pavement marking defects based on scaling factors of the road foreground 2D images;
the step 1.3 specifically includes:
and scaling all acquired road foreground 2D image data into 640pixel multiplied by 3 by adopting a bilinear interpolation method, and scaling true value data of pavement marking defects by adopting a conventional scaling method based on the scaling factors of the road foreground 2D image.
Step 1.4: and training a pavement marking breakage target detection algorithm model based on the road prospect 2D image data of the step 1.3 and the truth value data of the pavement marking breakage of the step 1.3.
The step 1.4 specifically comprises the following steps:
step 1.41: road foreground 2D image data scaled to 640 pixels by 3 is input to a feature extraction network
In the ResNet101, after feature extraction of the ResNet101, a preliminary feature layer with the size of 20 pixels multiplied by 2048 is obtained;
step 1.42: 20X 2048 preliminary extraction of ResNet101 with step size 2 of 4pixel X256 Conv2DTranspose, 4pixel X128 Conv2DTranspose, 4pixel X64 Conv2DTranspose, respectively
The feature layer is subjected to up-sampling treatment to obtain a 160pixel×160pixel×64 key feature layer; wherein 4pixel×4pixel× 256 Conv2DTranspose represents a transposed convolutional layer having a convolutional kernel size of 4×4 and a convolutional kernel number of 256; the feature layer provides key high-semantic information for a subsequent feature refinement module and a feature detection head module, and is a key module for successfully regressing the coordinates of a target object;
step 1.43: modeling the long-distance relation of image feature information, deep extraction of local feature details and spatial position information coding of a target are carried out on the key feature layer information based on a self-fusion coding mechanism, so that a refined graph is obtained;
step 1.44: and removing a noise frame based on a regression decoding mechanism, extracting an effective target frame from the depth of the thinned image, constructing optimal coordinate position information, and obtaining a pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information.
The step 1.43 specifically comprises the following steps:
the self-fusion coding mechanism is an improvement on the self-attention mechanism in the transducer, and a spatial feature information coding and refining branch module is added on the basis of the original self-attention mechanism, as shown in fig. 4, so that the spatial features of the target can be deeply extracted and corresponding position coding information can be established, the understanding of the target detection model on the defect features of the road marking is enhanced, and the specific reasoning steps after improvement are as follows:
step 1.431: the method comprises the steps of respectively using three full-connection layer Dense to obtain Query, key and Value matrix feature vector representations of the Key feature layer for the input Key feature layer;
step 1.432: inputting the input key feature layer into a spatial feature information coding and thinning branch module to obtain a thinned spatial feature map representation; the module consists of a 3×3×128 spatially separable convolutional layer, a batch regularization layer, a ReLU nonlinear activation function layer, and a 3×3×128 spatially separable convolutional layer. The 3×3×128 spatially separable convolution layer represents a spatially separable convolution layer having a convolution kernel size of 3×3 and a convolution kernel number of 128;
step 1.433: performing matrix multiplication operation on the Query and Key values obtained in the step 1.431, and multiplying the obtained result by a Scale factor Scale;
step 1.434: converting all elements in the matrix of the feature map obtained in the step 1.433 into relative probabilities among different elements by using a softMax activation function to obtain a matrix feature layer with probability distribution;
step 1.435: performing matrix multiplication on the Value obtained in the step 1.431 and the matrix characteristic layer with probability distribution obtained in the step 1.434;
step 1.436: and (3) accessing the result obtained in the step (1.435) into a full connection layer, and performing splicing fusion with the 1.432 refined space feature map to obtain a final output refined map.
The step 1.44 specifically comprises the following steps:
the lightweight decoding module is designed in the regression decoding mechanism, as shown in fig. 3, and has the characteristics of high reasoning speed, and can effectively remove noise frames and extract an effective target frame in depth, so that the damaged coordinate position of the pavement marking can be positioned more accurately and effectively, and reliable and effective positioning information can be provided for maintenance and repair of the later-stage marking.
Step 1.441: extracting a thermodynamic diagram detection head, a regression frame width and height detection head and a regression frame center offset detection head of a regression decoding mechanism into feature layers of a refined diagram, respectively inputting the feature layers into a light-weight decoding module, wherein the light-weight decoding module consists of a 5pixel×5pixel×64 depth separable convolution layer, layer regularization and a 5pixel×5pixel×64 depth separable convolution layer so as to enhance the regression capability of a decoding stage on a target position; where a 5pixel by 64 depth separable convolution layer represents a depth separable convolution layer with a convolution kernel size of 5 x 5 with a convolution kernel number of 64. The advantage of using the depth separable convolution layer is that it has few parameters, high efficiency and fast reasoning speed, which is convenient for real-time detection;
the thermodynamic diagram detection head consists of a 3pixel×3pixel×64 convolution layer and a 1pixel×1 convolution layer, and the refinement feature layer processed by the feature refinement module is processed to obtain a feature layer with the size of 160pixel×160pixel×1. The feature detection Head (Heat Map Head) has a main function of acquiring feature information of a target center point. Wherein a 3pixel×3pixel×64 convolution layer represents a convolution layer having a convolution kernel size of 3×3 and a convolution kernel number of 64, and a 1pixel×1 convolution layer represents a convolution layer having a convolution kernel size of 1×1 and a convolution kernel number of 1.
Regression frame width and height detection Head is composed of a 3pixel×3pixel×64 convolution layer and a 1pixel×1pixel×2 convolution layer, and the feature detection Head (Anchor Size Head) is mainly used for correcting the Size of the width and height of the prediction target frame. The method also processes the refined feature layer processed by the feature refinement module, and finally obtains the feature layer with the size of 160 pixels multiplied by 2. Wherein a 3pixel×3pixel×64 convolution layer represents a convolution layer having a convolution kernel size of 3×3 and a convolution kernel number of 64, and a 1pixel×1pixel×2 convolution layer represents a convolution layer having a convolution kernel size of 1×1 and a convolution kernel number of 2.
The Regression frame center offset detection Head consists of a 3pixel×3pixel×64 convolution layer and a 1pixel×1pixel×2 convolution layer, and the feature detection Head (Regression Head) is used for Regression prediction of the offset value between the target frame and the center point of the real frame. The method is also used for processing the refined feature layer processed by the feature refinement module, and finally obtaining the feature layer with the size of 160 pixels multiplied by 2. Wherein a 3pixel×3pixel×64 convolution layer represents a convolution layer having a convolution kernel size of 3×3 and a convolution kernel number of 64, and a 1pixel×1pixel×2 convolution layer represents a convolution layer having a convolution kernel size of 1×1 and a convolution kernel number of 2.
Step 1.442: performing a 3×3 max pooling operation on the feature layer refined in 1.441 to detect whether the value of the current hot spot is larger than all surrounding eight neighboring points; this is similar to NMS operation in mainstream target detection post-processing, which can be used to screen out redundant boxes;
step 1.443: screening K target frames meeting the requirements from the results in 1.442 by utilizing a TopK algorithm;
the K value is 100, wherein each target frame meeting the requirements comprises six parameters, namely the upper left corner coordinates (Xmin, ymin) of the target, the lower right corner coordinates (Xmax, ymax) of the target, a target Score and a target Class;
step 1.444: and further screening the K target frames meeting the requirements obtained in the step 1.443 by using a confidence threshold value to obtain optimal coordinate position information, and obtaining the pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information.
Step 2: acquiring road foreground 2D image data, and preprocessing the acquired foreground 2D image data;
the pretreatment is specifically as follows:
normalization processing is performed on the foreground 2D image data according to the following steps:
wherein i, j represent the row number and column number of the picture respectively; channel represents the channel of the picture, and the value is 0,1,2 (three channels of R, G and B are shared); c (i, j, channel) represents the pixel value of the original image corresponding to a certain channel; p (i, j, channel) represents the pixel value of the processed picture on a certain channel, and P (i, j, channel) is between (-1, 1); the three values in mean and std correspond to the mean and variance of the R, G and B channels respectively, the mean and variance data are universal image data tested on a large data set ImageNet, and the method is suitable for preprocessing processes of most images.
Step 3: inputting the preprocessed road foreground image data into a road marking damage target detection algorithm model to obtain the location of road marking loss and abrasion;
the scheme of the invention is redesigned and added with a key feature refinement module with a self-fusion coding mechanism and a regression decoding mechanism, and is used for rapidly positioning and identifying pavement marking defects. Compared with the traditional convolutional neural network, the self-fusion coding mechanism takes account of long-distance relation modeling of image feature information, deep extraction of local feature details and spatial position information coding of targets, and simultaneously better fuses deep high-semantic information and shallow feature detail information of a network model, so that similar relations among sub-pixels in an image are better searched, and the detection effect of the whole model is further improved. The final identification positioning effect of the image target detection model obtained by the method has good robustness, and can well position the worn place of the pavement marking and the place where the pavement marking is missing, thereby providing reliable and effective positioning information for subsequent maintenance and repair of the pavement marking.
Example 2: the embodiment provides a vehicle-mounted intelligent mobile phone-based intelligent positioning method and system for pavement marking defects, which are used for realizing the intelligent positioning method for pavement marking defects in embodiment 1, and comprise an acquisition module, a model construction module and an intelligent identification module;
the acquisition module is used for collecting road foreground 2D image data of a target road section in real time;
the model construction module is used for constructing an intelligent algorithm model;
the intelligent recognition module is used for carrying out unified operation of image size and normalization processing of images on the acquired road foreground 2D data, conveying the processed image data to a target detection algorithm, and carrying out regression to obtain actual position information of pavement marking damage so as to finish positioning of pavement marking loss and abrasion;
the intelligent algorithm model for detecting the damaged target of the pavement marking takes a convolutional target detection network CenterNet as a baseline, a key feature refinement module is embedded with a self-fusion coding mechanism after deep feature information is extracted by utilizing a residual neural network, the self-fusion coding mechanism is an improvement of a self-attention mechanism in a transducer, and a space feature information coding and refinement branch module is added on the basis of the original self-attention mechanism, so that the self-fusion coding mechanism takes account of long-distance relation modeling of image feature information, deep extraction of local feature details and space position information coding of the target, and simultaneously better fuses deep high semantic information and shallow feature detail information of the network model. After the key feature detection head is added with a regression decoding mechanism, a lightweight decoding module is designed in the regression decoding mechanism, and the module has the characteristics of high reasoning speed, can effectively remove noise frames and extract an effective target frame in depth, so that the damaged coordinate position of the pavement marking can be positioned more accurately and effectively, and reliable and effective positioning information is provided for maintenance and repair of the later-stage marking.
Example 3: the present embodiment tests 1500 actually measured road foreground 2D image data based on a conventional SSD300 algorithm model, YOLOv3-s algorithm model, YOLOv3-m algorithm model, YOLOv4-s algorithm model, YOLOv4-m algorithm model, center net algorithm model, and the image object detection positioning model of the above embodiment, and the indexes of different algorithm networks are as follows in table 1:
table 1 shows the results of the index test of different target detection algorithm models
The index adopts three indexes which are the most representative in the field of the intelligent algorithm at present, namely Recall rate (Recall), precision and F1-measure score, and the larger the score value is, the better the generalization capability and performance of the algorithm model are indicated. It is worth mentioning that the F1-measure index can be more comprehensive to reflect the excellent performance of the algorithm network, which is the harmonic mean of Recall and Precision:
F1-measure =2*Recall*Precision/(Recall+Precision))；
wherein is multiplied by;
and currently mainstream network models: compared with SSD300, YOLOv3-s, YOLOv3-m, YOLOv4-s, YOLOv4-m and CenterNet, the algorithm provided by the invention has obvious advantages in positioning and identifying the marking defect.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention. The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (7)
1. The intelligent positioning method for the pavement marking defect is characterized by comprising the following steps of:
step 1: constructing a pavement marking damage target detection algorithm model;
step 2: acquiring road foreground 2D image data, and preprocessing the acquired foreground 2D image data;
step 3: inputting the preprocessed road foreground image data into a road marking damage target detection algorithm model to obtain the location of road marking loss and abrasion;
in the step 1, the pavement marking damage target detection algorithm model takes a convolutional target detection network CenterNet as a base line, and a self-fusion coding mechanism is embedded in a key feature refinement module after deep feature information is extracted by using a residual neural network;
the step 1 specifically comprises the following steps:
step 1.1: collecting road foreground 2D image data by using a vehicle-mounted smart phone, and establishing a road foreground 2D image database;
step 1.2: constructing coordinate position data of pavement marking missing and abrasion corresponding to the collected 2D image data of the road foreground, and forming pavement marking defect truth value data;
step 1.3: scaling all acquired road foreground 2D image data to a uniform size, and scaling true value data of pavement marking defects based on scaling factors of the road foreground 2D images;
step 1.4: training a pavement marking breakage target detection algorithm model based on the road foreground 2D image data of the step 1.3 and the truth value data of the pavement marking breakage of the step 1.3;
the step 1.4 specifically comprises the following steps:
step 1.41: the road foreground 2D image data scaled into 640 pixels multiplied by 3 is input into a feature extraction network ResNet101, and after feature extraction of the ResNet101, a preliminary feature layer with the size of 20 pixels multiplied by 2048 is obtained;
step 1.42: up-sampling the 20pixel×20pixel×2048 preliminary feature layers extracted by the res net101 by using 4pixel×4pixel× 256 Conv2DTranspose, 4pixel×4pixel× 128 Conv2DTranspose and 4×4× 64 Conv2DTranspose with a step length of 2, respectively, to obtain 160pixel×160pixel×64 key feature layers; wherein 4pixel×4pixel× 256 Conv2DTranspose represents a transposed convolutional layer having a convolutional kernel size of 4×4 and a convolutional kernel number of 256;
step 1.43: modeling the long-distance relation of image feature information, deep extraction of local feature details and spatial position information coding of a target are carried out on the key feature layer information based on a self-fusion coding mechanism, so that a refined graph is obtained;
step 1.44: removing a noise frame based on a regression decoding mechanism, extracting an effective target frame from the depth of the thinned image, constructing optimal coordinate position information, and obtaining a pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information;
the step 1.43 specifically comprises the following steps:
step 1.431: the method comprises the steps of respectively using three full-connection layer Dense to obtain Query, key and Value matrix feature vector representations of the Key feature layer for the input Key feature layer;
step 1.432: inputting the input key feature layer into a spatial feature information coding and thinning branch module to obtain a thinned spatial feature map representation;
step 1.433: performing matrix multiplication operation on the Query and Key values obtained in the step 1.431, and multiplying the obtained result by a Scale factor Scale to obtain a feature map;
step 1.434: converting all elements in the matrix of the feature map obtained in the step 1.433 into relative probabilities among different elements by using a softMax activation function to obtain a matrix feature layer with probability distribution;
step 1.435: performing matrix multiplication on the Value obtained in the step 1.431 and the matrix feature layer with probability distribution obtained in the step 1.434 to obtain a further feature refinement result;
step 1.436: and (3) accessing the result obtained in the step (1.435) into a full connection layer, and performing splicing fusion with the 1.432 refined space feature map to obtain a final output refined map.
2. The intelligent positioning method for pavement marking defects according to claim 1, wherein the preprocessing is specifically as follows:
normalization processing is performed on the foreground 2D image data according to the following steps:
wherein i, j represent the row number and column number of the picture respectively; channel represents the channel of the picture, and the value is 0,1 and 2; c (i, j, channel) represents the pixel value of the original image corresponding to a certain channel; p (i, j, channel) represents the pixel value of the processed picture on a certain channel, and P (i, j, channel) is between (-1, 1); the three values in mean and std correspond to the mean and variance of the three channels R, G, B, respectively.
3. The intelligent positioning method for pavement marking defects according to claim 1, wherein the step 1.1 specifically comprises:
step 1.11: acquiring road foreground 2D image data of expressways, urban roads and rural roads, constructing a large data platform of the road foreground 2D image, selecting samples from the large data platform, and forming a training data sample base of a training algorithm model, wherein the training data sample base comprises: a highway marking missing sample, a highway marking wear sample, an urban road marking missing sample, an urban road marking wear sample, a rural road marking missing sample, and a rural road marking wear sample;
step 1.12: selecting part of various marking wear sample data from a constructed road foreground 2D image big data platform by adopting a pre-training model screening method to pre-train a road marking damage target detection algorithm model to obtain an algorithm model with classification recognition capability;
step 1.13: inputting the collected expressway marking missing sample, expressway marking abrasion sample, urban road marking abrasion sample, rural road marking missing sample and rural road marking abrasion sample into the algorithm model with classification recognition capability obtained in the step 1.12 for N times to obtain classification information of pavement marking defect in each piece of image data, wherein the probability of marking missing, marking abrasion and background in each piece of image data is respectively recorded as Pi1, pi2, pi3, i=0, 1,2,3, … …, X and X is the total number of all types of collected marking samples;
step 1.14: taking various marking samples max { Pi1, pi2 and Pi3} as sample types of single images, and respectively obtaining the number of various marking samples;
step 1.15: and randomly extracting the same amount of sample data from each marked line sample database by using a random sampling method so as to achieve the purpose of balancing positive and negative samples, thereby constructing a road prospect 2D image database.
4. The intelligent pavement marking defect locating method according to claim 3, wherein the step 1.2 specifically comprises:
and constructing coordinate position data of pavement marking missing and abrasion corresponding to the road foreground 2D image data based on the road foreground 2D image database, and forming pavement marking defect truth value data.
5. The intelligent pavement marking defect locating method according to claim 4, wherein said step 1.3 specifically comprises: and scaling all acquired road foreground 2D image data into 640pixel multiplied by 3 by adopting a bilinear interpolation method, and scaling true value data of pavement marking defects by adopting a conventional scaling method based on scaling factors of the road foreground 2D image, wherein 640 multiplied by 640 is wide and high, and 3 is RGB three channels.
6. The intelligent positioning method for pavement marking defects according to claim 1, wherein the step 1.44 specifically comprises the following steps:
step 1.441: extracting a thermodynamic diagram detection head, a regression frame width and height detection head and a regression frame center offset detection head of a regression decoding mechanism into feature layers of a refined diagram, respectively inputting the feature layers into a light-weight decoding module, wherein the light-weight decoding module consists of a 5pixel×5pixel×64 depth separable convolution layer, layer regularization and a 5pixel×5pixel×64 depth separable convolution layer so as to enhance the regression capability of a decoding stage on a target position;
step 1.442: performing a 3×3 max pooling operation on the feature layer refined in 1.441 to detect whether the value of the current hot spot is larger than all eight neighboring points;
step 1.443: screening K target frames meeting the requirements from the results in 1.442 by utilizing a TopK algorithm;
step 1.444: and further screening the K target frames meeting the requirements obtained in the step 1.443 by using a confidence threshold value to obtain optimal coordinate position information, and obtaining the pavement marking defect target detection, identification and positioning model based on the optimal coordinate position information.
7. An intelligent positioning system for pavement marking defects of a vehicle-mounted intelligent mobile phone is used for realizing the intelligent positioning method for pavement marking defects according to any one of claims 1-6, and is characterized by comprising an acquisition module, a model construction module and an intelligent identification module;
the acquisition module is used for collecting road foreground 2D image data of a target road section in real time;
the model construction module is used for constructing an intelligent algorithm model;
the intelligent recognition module is used for carrying out unified operation of image size and normalization processing of images on the acquired road foreground 2D data, conveying the processed image data to a target detection algorithm, and carrying out regression to obtain actual position information of road marking damage so as to finish positioning of road marking loss and abrasion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310310272.0A CN116052110B (en) | 2023-03-28 | 2023-03-28 | Intelligent positioning method and system for pavement marking defects |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310310272.0A CN116052110B (en) | 2023-03-28 | 2023-03-28 | Intelligent positioning method and system for pavement marking defects |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116052110A CN116052110A (en) | 2023-05-02 |
| CN116052110B true CN116052110B (en) | 2023-06-13 |
Family
ID=86133510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310310272.0A Active CN116052110B (en) | 2023-03-28 | 2023-03-28 | Intelligent positioning method and system for pavement marking defects |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116052110B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117129527A (en) * | 2023-08-30 | 2023-11-28 | 江苏瑞意隆建设工程有限公司 | Urban road paving quality detection method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111832493B (en) * | 2020-07-17 | 2024-09-10 | 平安科技（深圳）有限公司 | Image traffic signal lamp detection method and device, electronic equipment and storage medium |
| CN111950467B (en) * | 2020-08-14 | 2021-06-25 | 清华大学 | Fusion network lane line detection method based on attention mechanism and terminal equipment |
| CN112200161B (en) * | 2020-12-03 | 2021-03-02 | 北京电信易通信息技术股份有限公司 | Face recognition detection method based on mixed attention mechanism |
| CN114913565B (en) * | 2021-01-28 | 2023-11-17 | 腾讯科技（深圳）有限公司 | Face image detection method, model training method, device and storage medium |
| CN113038001A (en) * | 2021-02-26 | 2021-06-25 | 维沃移动通信有限公司 | Display method and device and electronic equipment |
| CN113191296A (en) * | 2021-05-13 | 2021-07-30 | 中国人民解放军陆军炮兵防空兵学院 | Method for detecting five parameters of target in any orientation based on YOLOV5 |
| CN113421269B (en) * | 2021-06-09 | 2024-06-07 | 南京瑞易智能科技有限公司 | Real-time semantic segmentation method based on double-branch deep convolutional neural network |
| CN113888754B (en) * | 2021-08-20 | 2024-04-26 | 北京工业大学 | Vehicle multi-attribute identification method based on radar vision fusion |
| CN114359153B (en) * | 2021-12-07 | 2024-08-02 | 湖北工业大学 | Insulator defect detection method based on improvement CENTERNET |
| CN114299303B (en) * | 2021-12-07 | 2024-06-14 | 集美大学 | Ship target detection method, terminal equipment and storage medium |
| CN114612803B (en) * | 2022-01-25 | 2024-09-24 | 安徽理工大学 | Improved CENTERNET transmission line insulator defect detection method |
| CN114565655B (en) * | 2022-02-28 | 2024-02-02 | 上海应用技术大学 | Depth estimation method and device based on pyramid segmentation attention |
| CN115331146A (en) * | 2022-08-15 | 2022-11-11 | 广东技术师范大学 | Micro target self-adaptive detection method based on data enhancement and feature fusion |
| CN115409797A (en) * | 2022-08-29 | 2022-11-29 | 上海大学 | PCB defect image detection method based on improved deep learning algorithm |
| CN115546768B (en) * | 2022-12-01 | 2023-04-07 | 四川蜀道新能源科技发展有限公司 | Pavement marking identification method and system based on multi-scale mechanism and attention mechanism |
-
2023
- 2023-03-28 CN CN202310310272.0A patent/CN116052110B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116052110A (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | RENet: Rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks | |
| CN111047554B (en) | Composite insulator overheating defect detection method based on instance segmentation | |
| CN108986122B (en) | Intelligent reconstruction method for indoor parking guide map | |
| CN109190752A (en) | The image, semantic dividing method of global characteristics and local feature based on deep learning | |
| CN111222580A (en) | High-precision crack detection method | |
| CN110135296A (en) | Airfield runway FOD detection method based on convolutional neural networks | |
| CN110415233A (en) | Pavement crack rapid extracting method based on two step convolutional neural networks | |
| CN113240623B (en) | Pavement disease detection method and device | |
| Sarmiento | Pavement distress detection and segmentation using YOLOv4 and DeepLabv3 on pavements in the Philippines | |
| CN115546768A (en) | Pavement marking identification method and system based on multi-scale mechanism and attention mechanism | |
| CN116052110B (en) | Intelligent positioning method and system for pavement marking defects | |
| Dong et al. | Automatic inspection and evaluation system for pavement distress | |
| CN114299394A (en) | Intelligent interpretation method for remote sensing image | |
| CN116883650A (en) | Image-level weak supervision semantic segmentation method based on attention and local stitching | |
| Yang et al. | PDNet: Improved YOLOv5 nondeformable disease detection network for asphalt pavement | |
| CN113012167B (en) | Combined segmentation method for cell nucleus and cytoplasm | |
| CN118196028A (en) | Rural cement pavement disease extraction method with improved YOLOv8 | |
| Sun et al. | Roadway crack segmentation based on an encoder-decoder deep network with multi-scale convolutional blocks | |
| Li et al. | Learning to holistically detect bridges from large-size vhr remote sensing imagery | |
| Yin et al. | Road Damage Detection and Classification based on Multi-level Feature Pyramids. | |
| CN117726957A (en) | Novel pavement disease detection method | |
| Yin et al. | Promoting Automatic Detection of Road Damage: A High-Resolution Dataset, a New Approach, and a New Evaluation Criterion | |
| CN112364687A (en) | Improved Faster R-CNN gas station electrostatic sign identification method and system | |
| CN117058069A (en) | Automatic detection method for apparent diseases of pavement in panoramic image | |
| CN110889418A (en) | Gas contour identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |