CN116895008A - Crack identification model determination and crack identification method, device, equipment and medium - Google Patents
Crack identification model determination and crack identification method, device, equipment and medium Download PDFInfo
- Publication number
- CN116895008A CN116895008A CN202310879503.XA CN202310879503A CN116895008A CN 116895008 A CN116895008 A CN 116895008A CN 202310879503 A CN202310879503 A CN 202310879503A CN 116895008 A CN116895008 A CN 116895008A
- Authority
- CN
- China
- Prior art keywords
- network
- target
- crack
- detection
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000001514 detection method Methods 0.000 claims abstract description 159
- 238000000605 extraction Methods 0.000 claims abstract description 57
- 230000004927 fusion Effects 0.000 claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 29
- 230000015654 memory Effects 0.000 claims description 25
- 238000012549 training Methods 0.000 claims description 25
- 238000012360 testing method Methods 0.000 claims description 17
- 238000007499 fusion processing Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 30
- 230000000694 effects Effects 0.000 description 9
- 238000012512 characterization method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image recognition, and discloses a crack recognition model determining method, a crack recognition model determining device, a crack recognition device and a crack recognition medium, wherein the crack recognition model determining method comprises the following steps: acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model; adding a preset detection layer into the feature extraction network to obtain a target feature extraction network; determining a target detection network based on the target feature extraction network and the feature fusion network; and processing the target crack image data set through a main network, a target detection network and a regression detection network based on the crack image data set to obtain a target crack identification model. According to the invention, the preset detection layer is added in the feature extraction network, so that the feature extraction of the small-size target is increased, and the detection efficiency of the small target is improved. Further, the backbone network and the target detection network are utilized to process the crack image data set, so that the image detection accuracy is improved.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a crack recognition model determining and crack recognition method, device, equipment and medium.
Background
The image recognition technology has traditional machine learning and a deep learning method which is developed relatively hot at present, the image recognition technology based on the deep learning is developed from a two-stage algorithm to a one-stage algorithm, the recognition algorithm of the YOLO series is representative of the one-stage recognition algorithm at present, the application is relatively wide, and the recognition effect is improved continuously along with the continuous updating of the version.
At present, a model based on deep learning is adopted for crack image recognition, but the accuracy of a crack recognition result output by the model is poor due to the fact that the image feature extraction effect is poor.
Disclosure of Invention
In view of the above, the invention provides a method, a device, equipment and a medium for determining and identifying a crack identification model, which are used for solving the problem that the precision of a crack identification result output by the model is poor due to the fact that the image feature extraction effect is poor when the crack image identification is carried out by the model based on deep learning.
In a first aspect, the present invention provides a crack identification model determining method, including:
acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model, wherein the initial neural network comprises a main network, an initial detection network and a regression detection network, and the initial detection network comprises a feature extraction network and a feature fusion network; adding a preset detection layer into the feature extraction network to obtain a target feature extraction network; determining a target detection network based on the target feature extraction network and the feature fusion network; and processing the target crack image data set through a main network, a target detection network and a regression detection network based on the crack image data set to obtain a target crack identification model.
According to the crack identification model determining method provided by the invention, the feature extraction of the small-size target is increased and the detection efficiency of the small target is improved by adding the preset detection layer in the feature extraction network. Further, the backbone network and the target detection network are utilized to process the crack image data set, so that the image detection accuracy is improved.
In an alternative embodiment, the processing of the backbone network, the target detection network and the regression detection network based on the fracture image data set to obtain the target fracture identification model includes:
inputting the crack image data set into a main network, and after the crack image data set is processed by a target detection network, obtaining an initial detection result, wherein the output end of the main network is connected with the input end of the target detection network; determining a target neural network based on the target detection network, the backbone network and the regression detection network; and processing the target neural network based on the crack image data set and the initial detection result to obtain a target crack identification model.
According to the invention, the initial detection result is combined with the crack image data set, and the target crack identification model is obtained through target neural network training, so that the characteristics of the original image can be more maintained, the original characteristic expression effect is enhanced, excessive parameters are not introduced, and the image detection accuracy is improved.
In an alternative embodiment, inputting the fracture image dataset into a backbone network and obtaining an initial detection result after processing by a target detection network, including:
acquiring a first feature pyramid network and a second feature pyramid network of a target detection network; inputting the crack image data set into a backbone network to generate a characteristic image set; and carrying out feature extraction and feature fusion processing through the first feature pyramid network and the second feature pyramid network based on the feature image set to obtain an initial detection result.
The invention utilizes the first feature pyramid network and the second feature pyramid network to extract and fuse the image features of the crack image data set, can more keep the features of the original image, strengthens the original feature expression effect, does not introduce excessive parameters, improves the image detection accuracy, and further improves the crack identification accuracy.
In an alternative embodiment, the target fracture identification model is obtained through target neural network processing based on the fracture image dataset and the initial detection result, and the method comprises the following steps:
inputting the crack image data set into a target neural network for training to obtain an initial crack identification model; and optimizing the initial crack identification model by using the crack image dataset and the initial detection result to obtain the target crack identification model.
According to the invention, the target neural network is utilized to train the crack image data set, and the initial crack recognition model is optimized by utilizing the crack image data set and the initial detection result, so that the image detection accuracy is improved, and the model recognition precision is further improved.
In a second aspect, the present invention provides a crack identification method, the crack identification method comprising:
acquiring an image dataset of a crack to be identified;
inputting the image dataset into a target crack recognition model to obtain a target crack recognition result, wherein the target crack recognition model is obtained according to the crack recognition model determination method of the first aspect or any corresponding embodiment of the first aspect.
According to the crack identification method provided by the invention, the target crack identification model obtained by the crack identification model determination method provided by the first aspect of the invention is used for identification, so that the accuracy of the target crack identification result is improved.
In an alternative embodiment, inputting the image dataset into the target fracture recognition model to obtain the target fracture recognition result includes:
dividing the image dataset into an image training dataset and an image testing dataset; inputting the image training data set into a target crack identification model to obtain an initial crack identification result and a model parameter data set; and testing the initial crack identification result by using the model parameter data set and the image test data set to obtain a target crack identification result.
When training is carried out by utilizing the image training data set, the initial crack identification result is tested and verified by utilizing the image testing data set and the model parameter data set obtained by training, and the accuracy of the target crack identification result is improved.
In a third aspect, the present invention provides a crack recognition model determination apparatus including:
the first acquisition module is used for acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model, wherein the initial neural network comprises a main network, an initial detection network and a regression detection network, and the initial detection network comprises a feature extraction network and a feature fusion network; the adding module is used for adding a preset detection layer in the feature extraction network to obtain a target feature extraction network; the determining module is used for determining a target detection network based on the target feature extraction network and the feature fusion network; and the processing module is used for processing the main network, the target detection network and the regression detection network based on the crack image data set to obtain a target crack identification model.
In a fourth aspect, the present invention provides a crack recognition apparatus comprising:
the second acquisition module is used for acquiring an image data set of the crack to be identified; the input module is configured to input the image dataset into a target crack recognition model, to obtain a target crack recognition result, where the target crack recognition model is obtained according to the first aspect or any one of the corresponding embodiments of the crack recognition model determining method.
In a fifth aspect, the present invention provides a computer device comprising: the processor executes the computer instructions to perform the crack identification model determination method according to the first aspect or any embodiment thereof, or the crack identification method according to the second aspect or any embodiment thereof.
In a sixth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to execute the crack identification model determination method of the first aspect or any one of the embodiments corresponding thereto, or the crack identification method of the second aspect or any one of the embodiments corresponding thereto.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a crack recognition model determination method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an initial architecture of a feature extraction network neg according to an embodiment of the present invention;
FIG. 3 is a diagram of a multi-dimensional architecture of a feature extraction network neg layer according to an embodiment of the present invention;
FIG. 4 is a flow chart of another crack recognition model determination method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a multi-dimensional fusion structure according to an embodiment of the invention;
FIG. 6 is a flow chart of a crack identification method according to an embodiment of the present invention;
FIG. 7 is a flow chart of another crack identification method according to an embodiment of the present invention;
FIG. 8 is a block diagram showing the construction of a crack recognition model determination device according to an embodiment of the present invention;
fig. 9 is a block diagram of a crack recognition apparatus according to an embodiment of the present invention;
fig. 10 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention provides a crack identification model determining method, which achieves the effect of improving the detection efficiency of a small target by adding a preset detection layer in a feature extraction network.
According to an embodiment of the present invention, there is provided a crack identification model determination method and a crack identification method embodiment, it is to be noted that the steps shown in the flowcharts of the drawings may be performed in a computer system such as a set of computer-executable instructions, and that although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.
In this embodiment, a method for determining a crack identification model is provided, fig. 1 is a flowchart of a method for determining a crack identification model according to an embodiment of the present invention, and as shown in fig. 1, the flowchart includes the following steps:
step S101, acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model.
The preset target detection model is a YOLOv5 target detection model, and the initial neural network comprises a main network, an initial detection network and a regression detection network.
Further, the initial detection network may include a feature extraction network and a feature fusion network.
In a preferred embodiment, the initial neural network is composed of input, backbone, neck, head for performing image input, feature extraction, feature fusion and regression detection, respectively.
Step S102, adding a preset detection layer in the feature extraction network to obtain a target feature extraction network.
Specifically, the YOLOv5 target detection model often cannot detect a small target when the image resolution is lower than 64 pixels, so that a preset detection layer is added in the target feature extraction network, and the downsampling multiple can be reduced.
In a preferred embodiment, the feature extraction network neg adopts the structure of fpn+pan as shown in fig. 2.
The size of the input image is 608×608×3, and the feature map of 76×76×255, 38×38×255, and 19×19×255 is obtained after downsampling.
By adding one layer in the feature extraction network neg layer, the downsampling multiple is reduced, the downsampling is increased by 4 times, and the original three-scale feature map is changed into four-scale feature maps, as shown in fig. 3.
The three-size output is changed into four new-size output in the feature extraction network neg layer, so that feature extraction of small-size targets is increased, and detection efficiency of the small targets is improved.
Step S103, determining a target detection network based on the target feature extraction network and the feature fusion network.
Specifically, the target detection network is composed of a target feature extraction network and a feature fusion network after a preset detection layer is added.
Step S104, processing is carried out through a main network, a target detection network and a regression detection network based on the crack image data set, so as to obtain a target crack identification model.
Specifically, the training processing is performed on the fracture image dataset by using the trunk network and the target detection network respectively, so that a trained target fracture recognition model can be obtained.
According to the crack identification model determining method provided by the embodiment, the feature extraction of the small-size target is increased and the detection efficiency of the small target is improved by adding the preset detection layer in the feature extraction network. Further, the backbone network and the target detection network are utilized to process the crack image data set, so that the image detection accuracy is improved.
In this embodiment, a method for determining a crack identification model is provided, fig. 4 is a flowchart of a method for determining a crack identification model according to an embodiment of the present invention, and as shown in fig. 4, the flowchart includes the following steps:
step S401, acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model. Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S402, adding a preset detection layer in the feature extraction network to obtain a target feature extraction network. Please refer to step S102 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S403, determining a target detection network based on the target feature extraction network and the feature fusion network. Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
And step S404, processing the main network, the target detection network and the regression detection network based on the crack image data set to obtain a target crack identification model.
Specifically, the step S404 includes:
step S4041, inputting the crack image data set into a backbone network, and obtaining an initial detection result after processing by a target detection network.
The output end of the backbone network is connected with the input end of the target detection network.
Specifically, the crack image data set is firstly input into a backbone network, then is processed by the backbone network, generates different feature graphs, and then is input into a target detection network for feature extraction and fusion, so that a corresponding initial detection result can be obtained.
In step S4042, a target neural network is determined based on the target detection network, the backbone network, and the regression detection network.
Specifically, the target neural network is composed of a target detection network, a backbone network, and a regression detection network.
Step S4043, processing the target neural network based on the crack image data set and the initial detection result to obtain a target crack identification model.
Specifically, the crack image dataset and the initial detection result are input into a target neural network for training, and meanwhile, the initial detection result is utilized for optimization in the training process, so that a target crack identification model can be obtained.
In some alternative embodiments, step S4041 includes:
step a1, a first feature pyramid network and a second feature pyramid network of a target detection network are obtained.
And a2, inputting the crack image data set into a backbone network to generate a characteristic image set.
And a step a3, carrying out feature extraction and feature fusion processing through a first feature pyramid network and a second feature pyramid network based on the feature image set to obtain an initial detection result.
The object detection network comprises a first characteristic pyramid network (FPN) and a second characteristic pyramid network (PAN).
Specifically, after the fracture image dataset is input into the backbone network, different feature maps can be generated from bottom to top, and each feature map has lower resolution than the previous layer and higher semantic meaning.
Then, starting from the top of the feature map generated from bottom to top by using the FPN, doubling the resolution thereof through upsampling, and then adding the result to the next-layer feature map with lower resolution but higher semantics in the feature map sequence, thereby obtaining a new set of feature maps;
and finally, up-sampling the low-resolution feature map by using PAN, and then splicing the low-resolution feature map with the high-resolution feature map to obtain a richer feature map, namely an initial detection result after feature fusion.
In a preferred embodiment, as shown in fig. 3, the feature extraction and fusion process of the object detection network is:
(1) K5, up-sampling and fusing K4 to obtain a U4 feature map;
(2) The U4 is up-sampled and fused with the K3 to obtain a U3 characteristic diagram;
(3) The U3 is up-sampled and fused with the K2 to obtain a U2 feature map;
(4) D2, carrying out downsampling fusion on the U3 to obtain a D3 characteristic diagram;
(5) D3, downsampling and fusing U4 to obtain a D4 feature map;
(6) And D4, downsampling and fusing the U5 to obtain a D5 characteristic diagram.
Finally, four size results, namely initial detection results, are output from D2, D3, D4 and D5.
In some alternative embodiments, step S4043 includes:
and b1, inputting the crack image data set into a target neural network for training to obtain an initial crack identification model.
And b2, optimizing the initial crack identification model by using the crack image dataset and the initial detection result to obtain a target crack identification model.
Firstly, inputting a fracture image data set into a target neural network for training, and obtaining a trained initial fracture identification model.
Secondly, because the FPN is top-down characteristic expression, when the deep network is fused with the shallow network, information is lost, the FPN+PAN is fused in two directions, the characteristic expression effect is improved, but the defects exist, so that the initial crack recognition model is optimized by fusing the initial detection result with the characteristic diagram with the same resolution output by the main network, the characteristics of the original image can be more maintained, the original characteristic expression can be enhanced by the characteristic diagram output by the multi-size fusion structure, excessive parameters are not introduced, the image detection accuracy is improved, and the recognition accuracy of the target crack recognition model is further improved.
In a preferred embodiment, as shown in fig. 5, the upsampling structure of the optimized size fusion structure is unchanged, and the feature map of the K layers is fused during upsampling, and the fusion process is as follows:
(1) K5, up-sampling and fusing K4 to obtain a U4 feature map;
(2) The U4 is up-sampled and fused with the K3 to obtain a U3 characteristic diagram;
(3) The U3 is up-sampled and fused with the K2 to obtain a U2 feature map;
(4) D2, downsampling and fusing U3 and K3 of a backbone network to obtain a DD3 characteristic diagram;
(5) DD3 downsampling fusion U4 and K4 of backbone network to obtain DD4 characteristic diagram;
(6) DD4 downsamples and fuses U5 to obtain a D5 feature map.
Finally, four dimensional results are output from D2, DD3, DD4 and D5.
According to the crack identification model determining method, the first feature pyramid network and the second feature pyramid network are utilized to extract and fuse image features of the crack image data set, so that the features of an original image can be more maintained, the original feature expression effect is enhanced, excessive parameters are not introduced, the image detection accuracy is improved, the target neural network is utilized to train the crack image data set, the initial crack identification model is optimized by utilizing the crack image data set and the initial detection result, the image detection accuracy is improved, and the model identification accuracy is further improved.
In one example, a crack recognition method for multi-scale feature fusion is provided based on a YOLOv5 target recognition algorithm.
Specifically, the structure diagram consists of input, backbone, neck, head, and the input, feature extraction, feature fusion and detection of the picture are completed. The crack identification is generally aimed at a small target, the detail is strong, the characteristics of the shallow network are required to be subjected to enhanced fusion in the characteristic fusion, the receptive field of the shallow network is smaller, the geometric detail information characterization capability is strong, and the characteristic information of the crack can be better expressed.
Feature fusion is performed in the neg structure, and the fpn+pan structure diagram is adopted in the neg structure of YOLOv5, as shown in fig. 2.
The size of the input image is 608×608×3, after downsampling, feature maps of 76×76×255, 38×38×255, and 19×19×255 are obtained, when the resolution of YOLOv5 is lower than 64 pixels, small objects are often not detected, and in order to improve the detection accuracy of the small objects, model optimization is performed: one layer is added in the neg, the downsampling multiple is reduced, and the downsampling is increased by 4 times, so that the original three-scale characteristic diagram is changed into four-scale characteristic diagram, and the four-scale characteristic diagram is shown in fig. 3.
The three-size output is changed into four new-size output in the neg layer, so that the feature extraction of the small-size target is increased, and the detection efficiency of the small target is improved.
When the model of the multi-layer network is trained, the receptive field of the high-layer network is larger, the semantic information characterization capability is strong, but the resolution ratio of the feature map is low, and the space geometrical feature details are lacking. The receptive field of the low-level network is smaller, the geometric detail information characterization capability is strong, and the semantic information characterization capability is weak although the resolution ratio is high. The model is optimized for the defects of the high-level network and the bottom-level network.
The structure of the multi-size model is sampled from a back bone for feature extraction, and the fusion process is as follows: (1) K5 upsampling and fusing K4 to obtain a U4 characteristic diagram; (2) U4 upsampling fusion K3 to obtain a U3 feature map; (3) U3 upsampling and fusing K2 to obtain a U2 characteristic diagram; (4) D2 downsampling and fusing U3 to obtain a D3 characteristic diagram; (5) D3 downsampling and fusing U4 to obtain a D4 characteristic diagram; and (6) D4 downsampling and fusing U5 to obtain a D5 characteristic diagram. Finally, four dimensional results are output from D2, D3, D4 and D5.
Optimizing the multi-size model: the final output result is fused with the feature map with the same resolution in the backbone network, so that original information of the image can be more reserved, FPN is top-down feature expression, information is lost when the deep network is fused with the shallow network, FPN+PAN is two-way fusion, the feature expression effect is improved, but the defect still exists, a multi-size fusion method is improved for optimizing feature fusion, and a structural model is shown in figure 5.
The optimized size fusion structure is unchanged in the upsampling structure, the feature images of the K layers are fused in upsampling, and the fusion process is as follows: (1) K5 upsampling and fusing K4 to obtain a U4 characteristic diagram; (2) U4 upsampling fusion K3 to obtain a U3 feature map; (3) U3 upsampling and fusing K2 to obtain a U2 characteristic diagram; (4) D2, downsampling and fusing U3 and K3 of a backbone network to obtain a DD3 characteristic diagram; (5) DD3 downsampling fusion U4 and K4 of backbone network to obtain DD4 characteristic diagram; and (6) DD4 downsampling and fusing U5 to obtain a D5 characteristic diagram. Finally, four dimensional results are output from D2, DD3, DD4 and D5.
The multi-size fusion structure fuses the characteristic images which are not operated from top to bottom with the PAN structure before, so that the characteristics of the original image can be more maintained, the original characteristic expression can be enhanced through the characteristic images output by the multi-size fusion structure, excessive parameters are not introduced, and the image detection accuracy is improved.
In this embodiment, a crack identifying method is provided, fig. 6 is a flowchart of the crack identifying method according to an embodiment of the present invention, and as shown in fig. 6, the flowchart includes the following steps:
step S601, acquiring an image dataset of a crack to be identified.
Specifically, an image dataset of a crack to be identified is acquired.
Step S602, inputting the image data set into a target crack recognition model to obtain a target crack recognition result.
The target crack identification model is obtained according to the crack identification model determination method provided by the embodiment.
Specifically, the acquired image dataset of the crack to be identified is input into a trained target crack identification model, so that a target crack identification result of the crack to be identified can be obtained.
According to the crack identification method provided by the embodiment of the invention, the target crack identification model obtained by the crack identification model determination method provided by the first aspect of the invention is used for identification, so that the accuracy of the target crack identification result is improved.
In this embodiment, a crack identifying method is provided, fig. 7 is a flowchart of the crack identifying method according to an embodiment of the present invention, and as shown in fig. 7, the flowchart includes the following steps:
step S701, acquiring an image dataset of a crack to be identified. Please refer to step S601 in the embodiment shown in fig. 6 in detail, which is not described herein.
Step S702, inputting the image data set into a target crack recognition model to obtain a target crack recognition result.
Specifically, the step S702 includes:
step S7021, the image data set is divided into an image training data set and an image test data set.
Specifically, the image dataset is annotated and divided into an image training dataset and an image testing dataset.
Step S7022, inputting the image training data set into the target fracture recognition model to obtain an initial fracture recognition result and a model parameter data set.
Specifically, the image training data set is input into the target crack recognition model for training, so that a model parameter data set and a model output result of the target crack recognition model, namely an initial crack recognition result, can be obtained.
Step S7023, testing the initial crack recognition result by using the model parameter data set and the image test data set to obtain a target crack recognition result.
Specifically, the initial crack identification result is tested and verified by using the model parameter data set and the image test data set, and when the accuracy of the initial crack identification result meets the requirement, the target crack identification result is output.
According to the crack identification method provided by the embodiment, the initial crack identification result is tested and verified by using the image test data set and the model parameter data set obtained through training, so that the accuracy of the target crack identification result is improved.
The embodiment also provides a crack identification model determining device and a crack identification device, which are used for realizing the embodiment and the preferred implementation manner, and are not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a crack recognition model determination apparatus, as shown in fig. 8, including:
the first obtaining module 801 is configured to obtain a fracture image dataset, a preset target detection model, and an initial neural network of the preset target detection model, where the initial neural network includes a backbone network and an initial detection network, and the initial detection network includes a feature extraction network, a feature fusion network, and a regression detection network.
The adding module 802 is configured to add a preset detection layer to the feature extraction network, so as to obtain a target feature extraction network.
A determining module 803, configured to determine a target detection network based on the target feature extraction network, the feature fusion network, and the regression detection network.
The processing module 804 is configured to obtain a target fracture recognition model based on the fracture image dataset through processing of the backbone network and the target detection network.
In some alternative embodiments, the processing module 804 includes:
and the input and processing unit is used for inputting the crack image data set into the main network, obtaining an initial detection result after the crack image data set is processed by the target detection network, and connecting the output end of the main network with the input end of the target detection network.
And the determining unit is used for determining the target neural network based on the target detection network, the backbone network and the regression detection network.
And the processing unit is used for obtaining a target crack identification model through target neural network processing based on the crack image data set and the initial detection result.
In some alternative embodiments, the input and processing unit includes:
and the acquisition subunit is used for acquiring the first characteristic pyramid network and the second characteristic pyramid network of the target detection network.
And the generation subunit is used for inputting the crack image data set into the backbone network to generate the characteristic image set.
The processing subunit is used for carrying out feature extraction and feature fusion processing through the first feature pyramid network and the second feature pyramid network based on the feature image set to obtain an initial detection result.
In some alternative embodiments, the processing unit includes:
and the training subunit is used for inputting the crack image data set into the target neural network for training to obtain an initial crack identification model.
And the optimizing subunit is used for optimizing the initial crack identification model by utilizing the crack image dataset and the initial detection result to obtain the target crack identification model.
The crack recognition model determination means in this embodiment is presented in the form of functional units, here referred to as ASIC (Application Specific Integrated Circuit ) circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above described functions.
The present embodiment provides a crack identifying device, as shown in fig. 9, including:
a second acquisition module 901 for acquiring an image dataset of the crack to be identified.
The input module 902 is configured to input the image dataset into a target crack recognition model to obtain a target crack recognition result, where the target crack recognition model is obtained according to the crack recognition model determining method provided by the above embodiment of the present invention.
In some alternative embodiments, the input module 902 includes:
and the dividing unit is used for dividing the image data set into an image training data set and an image testing data set.
And the input unit is used for inputting the image training data set into the target crack identification model to obtain an initial crack identification result and a model parameter data set.
And the testing unit is used for testing the initial crack identification result by using the model parameter data set and the image testing data set to obtain a target crack identification result.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The crack recognition means in this embodiment are presented in the form of functional units, here referred to as ASIC (Application Specific Integrated Circuit ) circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above described functionality.
The embodiment of the invention also provides computer equipment, which is provided with the crack identification model determining device shown in the figure 8 and the crack identification device shown in the figure 9.
Referring to fig. 10, fig. 10 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 10, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 10.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (10)
1. A method for determining a crack recognition model, the method comprising:
acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model, wherein the initial neural network comprises a main network, an initial detection network and a regression detection network, and the initial detection network comprises a feature extraction network and a feature fusion network;
adding a preset detection layer into the feature extraction network to obtain a target feature extraction network;
determining a target detection network based on the target feature extraction network and the feature fusion network;
and processing the main network, the target detection network and the regression detection network based on the crack image data set to obtain a target crack identification model.
2. The method of claim 1, wherein processing through the backbone network, the target detection network, and the regression detection network based on the fracture image dataset results in a target fracture identification model comprising:
inputting the crack image data set into the main network, and after the crack image data set is processed by the target detection network, obtaining an initial detection result, wherein the output end of the main network is connected with the input end of the target detection network;
determining a target neural network based on the target detection network, the backbone network, and the regression detection network;
and processing the target neural network based on the crack image data set and the initial detection result to obtain the target crack identification model.
3. The method of claim 2, wherein inputting the fracture image dataset into the backbone network and processing by the target detection network results in an initial detection result, comprising:
acquiring a first feature pyramid network and a second feature pyramid network of the target detection network;
inputting the fracture image data set into the backbone network to generate a characteristic image set;
and carrying out feature extraction and feature fusion processing through the first feature pyramid network and the second feature pyramid network based on the feature image set to obtain the initial detection result.
4. The method of claim 2, wherein the target fracture identification model is obtained based on the fracture image dataset and the initial detection result via the target neural network processing, comprising:
inputting the crack image data set into the target neural network for training to obtain an initial crack identification model;
and optimizing the initial crack identification model by using the crack image data set and the initial detection result to obtain the target crack identification model.
5. A method of crack identification, the method comprising:
acquiring an image dataset of a crack to be identified;
inputting the image dataset into a target crack recognition model, obtaining a target crack recognition result, wherein the target crack recognition model is obtained according to the crack recognition model determining method as claimed in any one of claims 1 to 4.
6. The method of claim 5, wherein inputting the image dataset into a target fracture recognition model to obtain a target fracture recognition result comprises:
dividing the image dataset into an image training dataset and an image testing dataset;
inputting the image training data set into the target crack identification model to obtain an initial crack identification result and a model parameter data set;
and testing the initial crack identification result by using the model parameter data set and the image test data set to obtain the target crack identification result.
7. A crack recognition model determination apparatus, characterized in that the apparatus comprises:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a crack image data set, a preset target detection model and an initial neural network of the preset target detection model, the initial neural network comprises a main network and an initial detection network, and the initial detection network comprises a feature extraction network, a feature fusion network and a regression detection network;
the adding module is used for adding a preset detection layer into the feature extraction network to obtain a target feature extraction network;
the determining module is used for determining a target detection network based on the target feature extraction network, the feature fusion network and the regression detection network;
and the processing module is used for processing the main network and the target detection network based on the fracture image data set to obtain a target fracture identification model.
8. A crack identification device, the device comprising:
the second acquisition module is used for acquiring an image data set of the crack to be identified;
an input module for inputting the image dataset into a target crack recognition model, resulting in a target crack recognition result, the target crack recognition model being obtained according to the crack recognition model determination method as claimed in any one of claims 1 to 4.
9. A computer device, comprising:
a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the crack identification model determination method of any one of claims 1 to 4 or to perform the crack identification method of claim 5 or 6.
10. A computer-readable storage medium having stored thereon computer instructions for causing a computer to execute the crack identification model determination method according to any one of claims 1 to 4 or to execute the crack identification method according to claim 5 or 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310879503.XA CN116895008A (en) | 2023-07-17 | 2023-07-17 | Crack identification model determination and crack identification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310879503.XA CN116895008A (en) | 2023-07-17 | 2023-07-17 | Crack identification model determination and crack identification method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116895008A true CN116895008A (en) | 2023-10-17 |
Family
ID=88314630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310879503.XA Pending CN116895008A (en) | 2023-07-17 | 2023-07-17 | Crack identification model determination and crack identification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116895008A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117274817A (en) * | 2023-11-15 | 2023-12-22 | 深圳大学 | Automatic crack identification method and device, terminal equipment and storage medium |
-
2023
- 2023-07-17 CN CN202310879503.XA patent/CN116895008A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117274817A (en) * | 2023-11-15 | 2023-12-22 | 深圳大学 | Automatic crack identification method and device, terminal equipment and storage medium |
| CN117274817B (en) * | 2023-11-15 | 2024-03-12 | 深圳大学 | Automatic crack identification method and device, terminal equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3506161A1 (en) | Method and apparatus for recovering point cloud data | |
| US11328401B2 (en) | Stationary object detecting method, apparatus and electronic device | |
| US11810326B2 (en) | Determining camera parameters from a single digital image | |
| CN113139543B (en) | Training method of target object detection model, target object detection method and equipment | |
| CN111681273A (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| CN111914654B (en) | Text layout analysis method, device, equipment and medium | |
| CN110781980B (en) | Training method of target detection model, target detection method and device | |
| CN116895008A (en) | Crack identification model determination and crack identification method, device, equipment and medium | |
| CN111222454A (en) | Method and system for training multi-task target detection model and multi-task target detection | |
| CN113468946A (en) | Semantically consistent enhanced training data for traffic light detection | |
| CN112052907A (en) | Target detection method and device based on image edge information and storage medium | |
| CN111435445A (en) | Training method and device of character recognition model and character recognition method and device | |
| CN115375617A (en) | Defect detection and training method and device, storage medium and equipment | |
| CN117671509B (en) | Remote sensing target detection method and device, electronic equipment and storage medium | |
| CN111275126A (en) | Sample data set generation method, device, equipment and storage medium | |
| CN114693963A (en) | Recognition model training and recognition method and device based on electric power data feature extraction | |
| CN116484878B (en) | Semantic association method, device, equipment and storage medium of power heterogeneous data | |
| CN113297986A (en) | Handwritten character recognition method, device, medium and electronic equipment | |
| CN113743434A (en) | Training method of target detection network, image augmentation method and device | |
| CN116580232A (en) | Automatic image labeling method and system and electronic equipment | |
| CN115797291A (en) | Circuit terminal identification method and device, computer equipment and storage medium | |
| CN114241044A (en) | Loop detection method, device, electronic equipment and computer readable medium | |
| CN114219725A (en) | Image processing method, terminal equipment and computer readable storage medium | |
| CN112766068A (en) | Vehicle detection method and system based on gridding labeling | |
| CN113034432A (en) | Product defect detection method, system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |