CN114549446A

CN114549446A - Cylinder sleeve defect mark detection method based on deep learning

Info

Publication number: CN114549446A
Application number: CN202210146225.2A
Authority: CN
Inventors: 黄晓华; 邵秀燕; 郝飞; 刘迁
Original assignee: Nanjing Institute of Technology
Current assignee: Nanjing Institute of Technology
Priority date: 2022-02-17
Filing date: 2022-02-17
Publication date: 2022-05-27
Anticipated expiration: 2042-02-17
Also published as: CN114549446B

Abstract

The invention provides a cylinder sleeve defect mark detection method based on deep learning, which comprises the following steps: acquiring a cylinder sleeve image to construct an original data set, preprocessing and marking the image in the original data set, and acquiring a local defect map of a training set; modeling by using a Mask R-CNN algorithm based on a Swin converter to obtain a network model; detecting the image of the test set through the obtained network model; extracting an interest region to enhance the detection effect on the detection result through a mask mechanism; evaluating the detection performance of the network model; and detecting the defect marks of the acquired cylinder sleeve images through the obtained network model. According to the method, the influence of noise of the cylinder sleeve is reduced through image preprocessing, the original image is constructed into a local image to obtain a new batch of training data, the detection result of the original image is obtained through a method of mapping a small image to a large image, a noise area is filtered through a mask mechanism, and the small target detection precision is improved.

Description

Cylinder sleeve defect mark detection method based on deep learning

Technical Field

The invention relates to the field of image processing, in particular to a cylinder sleeve defect mark detection method based on deep learning.

Background

The performance of the cylinder liner as a core component of an internal combustion engine directly influences the overall performance of the internal combustion engine. The cylinder sleeve may have different defects such as sand holes, cracks, abrasion and the like due to uneven distribution of temperature, impurities and stress during processing. The detection of the surface defects of the steel plate becomes an essential link in the factory leaving process. The defect size of cylinder jacket is little, and the picture noise interference is big, according to traditional people eye detection efficiency low, and the precision is not high, and the easy appearance lou examines the phenomenon. At present, the requirement of enterprises cannot be met by eye detection, so deep learning is very important for detecting the defects of the cylinder sleeve.

The general detection methods are classified into the following 3 types:

traditional detection: the traditional defect detection mainly comprises manual visual detection, X-ray detection, sound wave detection, magnetic powder inspection, eddy current detection, laser scanning detection and the like.

And (3) machine learning detection: machine learning is mainly based on the manual qualitative design of features or statistical features of specific regions of an image.

Deep learning detection: in recent years, object detection using a deep learning method has been greatly advanced. Mainstream algorithms can be classified into two categories according to different flow. One is known as the first order method, such as YOLO, SSD, RetinaNet, etc. The first order method directly generates anchor points, and the main idea is to uniformly and densely sample different positions of an image. In the sampling process, different scales and aspect ratios are adopted, the features are extracted through a convolutional neural network, and then classification and regression are carried out. Only one step is needed in the whole process, so the detection speed is high. There is a problem in that it is more difficult to train the model due to the extreme imbalance between the positive and negative samples. Another class is called second order methods. The method comprises the steps of dividing a target detection task into two stages, firstly finding out a region in which a target to be detected possibly exists in an image, then carrying out classification judgment, and selecting a region with higher score by a method of setting a threshold value, thereby realizing target detection. The typical algorithms mainly comprise Fast R-CNN, Mask R-CNN and the like. Compared with the first-order network, the second-order network has more RPN layers, and meanwhile, the RPN layers generate candidate frames, and then anchor points are generated from the candidate frames. This has the effect of improving the accuracy of the first-order network to some extent, but the detection time becomes slow.

Disclosure of Invention

The technical problem to be solved is as follows: in order to solve the technical problems of the background technology, the invention provides a method for splitting a high-resolution image into small regions by region segmentation, combining a mask mechanism, carrying out target detection on the small regions, and returning to the high-resolution image by combining the small region detection, so that the precision of the small target detection on the high-resolution image is effectively improved; the invention has certain universality on other types of small target detection.

The technical scheme is as follows: 1. a cylinder sleeve defect mark detection method based on deep learning is characterized by comprising the following steps:

1) acquiring a cylinder sleeve image to construct an original data set, preprocessing and marking the image in the original data set, and dividing the original data set into a training set and a testing set according to a proportion;

2) acquiring a local defect map of a training set;

3) modeling by using a Mask R-CNN algorithm, and performing model training through a local defect map of a training set image to obtain a network model;

4) detecting the image of the test set through the obtained network model;

5) extracting the detection effect in the step 4) from the detection result through a mask mechanism;

6) evaluating the detection performance of the network model;

7) and 4) detecting the defect marks of the acquired cylinder sleeve images through the network model obtained in the step 4).

Further, a VOC format data set is manufactured, picture data in the data set are marked by using a LabelImg tool, an xml file is generated, the experimental data set is a cylinder liner defect picture, and the size of the picture is 2448 x 2048. The data set contains three categories, namely sand, scratch, week.

Further, image preprocessing. Because various noises exist in the actual image acquisition process, the target information of the defects is highlighted in order to improve the image quality. The invention adopts 5-by-5 Gaussian filtering to perform noise reduction treatment on the cylinder sleeve data set.

Further, a method of cutting out local graphs by original information is constructed, and the local graphs with the size of 256 × 256 are cut out by original marking information with the size of 2448 × 2048, and each local graph contains defects. These partial graphs were used for training on a Swin converter based Mask RCNN network.

Further, a small-image mapping large-image detection method is designed, original images of the test set are divided into 64 small images, a network trained before is selected for detection, and finally the original size is spliced.

Further, a mask mechanism method is provided, and a proper threshold value is selected for the cylinder sleeve mold image to carry out binarization. And removing the binary fine black points by using a morphological operation closed operation, ensuring that the binary region contains the whole interest region, and taking the whole interest region as the interest region. And (3) carrying out coordinate measurement on the upper end face image, determining the circle center, the maximum circle and the minimum circle radius according to the to-be-detected region of the upper end face image, drawing a circular ring binary image with the same size, finishing the manufacture of the upper end face image mold, and taking the upper end face image mold as the interest region of the upper end face image. In the last type of picture, we design a specific mask to extract the region of interest. The detection precision is further improved.

(III) advantageous effects

1. The method adopts a Mask RCNN network based on a Swin converter to train a local graph, wherein the Swin converter (Swin converter) is the latest visual converter network in the current computer vision field, is a coder decoder structure, has a hierarchical characteristic graph, can combine deep image blocks, and has linear calculation complexity.

2. The invention further improves the detection precision by adopting a mask mechanism, and extracts the interest region by adopting a mask. The interest area is extracted, so that the target area searching time is reduced, and the detection effect can be enhanced.

Drawings

FIG. 1 is a block diagram of the algorithm of the present invention;

figure 2 is a diagram of a Swin converter network of the present invention;

FIG. 3 is a block diagram of the algorithm detection of the present invention;

Detailed Description

The drawings are for illustrative purposes only and are not to be construed as limiting the patent;

for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.

Referring to fig. 1, firstly, the original high resolution training image (2448 × 2048) is subjected to noise reduction processing by a 5 × 5 gaussian filter, and then a local defect map is captured according to the position of defect calibration and normalized to a 256 × 256 size picture, as shown in (a) of fig. 1; secondly, a visual transformer is used as a network framework, a local defect map training network is adopted to extract the characteristics of a visual attention mechanism and is used for training a Mask-RCNN defect detection network, and the stage (b) is shown in FIG. 1; finally, aiming at the original high-resolution test image, the original image is divided into 64 small-area images, the 64 small-area images are detected, and then the influence of noise is shielded by using a mask mechanism with an interest area, so that the detection precision and the detection speed are improved, for example, in the detection stage (2) in fig. 1.

Constructing a local graph: the method comprises two key technologies, namely image denoising processing and local graph construction.

(1) And (3) image noise reduction processing: a large amount of noise can be generated due to signal interference of various temperatures, illumination and electronic components in the process of acquiring the cylinder sleeve image by a camera, the image quality is greatly reduced, the subsequent treatment is influenced, and even the detection result of the surface defect of the cylinder sleeve can be influenced. In consideration of reducing the complexity of a later deep learning algorithm and improving the quality of an original image, the invention preprocesses the acquired cylinder sleeve image, and data enhancement is an effective method for realizing the aim for establishing an effective deep learning model. The data enhancement comprises the technologies of noise removal, interest area extraction and the like, and the size and the quality of a training data set can be improved, so that a better deep learning model can be constructed, and the generalization capability of the model is improved. Common noises include salt and pepper noise, Gaussian noise and random noise, and various noises exist in the actual image acquisition process, so that the target information of defects is highlighted in order to improve the image quality. And performing noise reduction treatment on the cylinder sleeve picture, and performing simulation analysis by adopting Gaussian filtering. In order to ensure reliability, the image effect after noise reduction is analyzed by using PSNR (peak signal-to-noise ratio) and AG (sharpness) values. Generally, the larger the PSNR value, the better the image quality, and the formula is:

wherein I is a monochrome image composed of M × N, and K is a monochrome image composed of M × N; the micro detail contrast and the texture change characteristics in the AG image reflect the definition of the image, and the larger the value is, the better the value is, the formula is as follows:

wherein, the delta Ix and the delta Iy are respectively the convolution of Sobel horizontal and vertical direction edge detection operators at the pixel points (i, j), and M and N are the pixel point number.

The invention adopts 5-by-5 Gaussian filtering to reduce the noise of the collected cylinder sleeve images.

Local graph construction: the most difficult recognition of the surface defects of the cylinder sleeve is that no specific and accurate judgment standard exists for the sizes and types of the defects, the size of an industrially acquired picture is 2448 × 2048, the size of the picture is large, the defects are small, some defects are not easy to distinguish by human eyes, the detection is quite difficult, and on the other hand, the defects also have various noise interferences, and the effect is quite undesirable if the defects are directly detected. Therefore, an idea of "constructing a partial graph" is proposed. The principle is as follows: during training, on the training set and the test set of the original image, the defect positions of the cylinder sleeves are intercepted and obtained according to the marking information on the 2448 × 2048 large image, the intercepted images are artificially set, and the sizes are normalized to 256 × 256. The defects are more clearly visible than in the original.

As in fig. 2, the network is constructed: the Swin Transformer (Swin Transformer) is a latest visual Transformer network in the field of computer vision at present, and is a coder-decoder structure, has a hierarchical feature map, can merge deep image blocks, and has linear computational complexity.

The invention adopts a Swin converter-based Mask RCNN network to train the local graph.

The detection method of the small map mapping large map comprises the following steps: using the trained network weights, for the actual detected original image, the invention divides 2448 x 2048 original image into 64 small maps, wherein 64 small maps are detected, including 56 small maps of 256 x 256 and 8 small maps of 656 x 256. And (4) detecting through the trained weight file, and finally remapping 2448 x 2048 of the original picture to obtain a detection result aiming at the original picture.

Its advantages are high effectThe method is much better than the training and detection effect of directly using the original high-resolution image, and solves the difficulty of small target detection.

The masking mechanism: the detection precision is further improved, and the interested region is extracted by adopting a mask. The interest area is extracted, so that the target area searching time is reduced, and the detection effect can be enhanced.

(1) And (3) for the roughly-made region-of-interest mold of the upper end cover image, selecting a proper threshold value for the cylinder sleeve mold image to carry out binarization, wherein the mold comprises three types of defects. And removing the binary fine black points by using a morphological operation closed operation to ensure that the binary region contains the whole interest region. It is taken as the region of interest.

(2) For an upper end face image containing three types of defects, performing coordinate measurement on the upper end face image, and determining a circle center, a maximum circle and a minimum circle radius according to a to-be-detected area of the upper end face image; and drawing a circular ring binary image with the same size, finishing the manufacture of an upper end face image die, and taking the upper end face image die as an interest area of the upper end face image.

(3) For the cylinder liner inner wall image, which contains three types of defects, we find that the defects are all distributed in the upper half part, and in order to obtain the defect information better, we design a specific mask according to the outline of the upper half part. As for the region of interest extracted from the upper end face, performing coordinate measurement on the region of interest, and determining the contour information (approximate to a ring shape) of the upper half part according to the region to be detected; drawing a binarization image with the same shape to extract the interest area.

The invention carries out denoising on an image obtained by a method for detecting a small map and a large map by mask, and the principle is that the detection result of a non-interest area is removed and the detection performance is improved by setting the pixel of the interest area as 1 and the pixel of the non-interest area as 0, and the formula is as follows:

I_R＝I×M

(3)

wherein, I is the detected pixel point in the original image, M is the mask matrix array, I_RAnd the final detection pixel point is obtained.

There are many ways to evaluate the ability of deep learning methods. The invention utilizes a common MAP value to prove the detection performance of various deep learning algorithms. Wherein TP means a result of being correctly recognized. FP refers to the result of recognition error. FN refers to unidentified results, but the system identifies other defects.

The cylinder sleeve picture has the characteristics of high resolution, small defect target and the like. Aiming at the characteristics of the cylinder liner picture, the invention reduces the noise influence of the cylinder liner through image preprocessing, constructs a local picture for the original picture, obtains a new batch of training data, thereby training the network, and obtains the detection result of the original picture through a method of mapping a large picture with a small picture. And finally, filtering the noise area through a mask mechanism, and further improving the small target detection precision. Based on the method, the small target defects can be accurately identified, and the detection performance is improved.

It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A cylinder sleeve defect mark detection method based on deep learning is characterized by comprising the following steps:

2) acquiring a local defect map of a training set;

3) modeling by using a Mask R-CNN algorithm based on a Swin converter, and performing model training by using a local defect map of a training set image to obtain a network model;

4) detecting the image of the test set through the obtained network model;

6) and evaluating the detection performance of the network model.

2. The cylinder liner defect mark detection method based on deep learning of claim 1, wherein in step 1), a raw data set in a VOC file format is prepared from the acquired cylinder liner images, and simultaneously, a label img image labeling tool is used to label the images in the raw data set, so as to label the defect regions and types on the images in the raw data set, thereby generating an xml file, and the raw data set is divided into three types, namely, sand mark defects, scratch defects and wear defects; the preprocessing method comprises the steps of firstly, carrying out image random overturning, cutting, pixel normalization and image enhancement on an image in an original data set, and then carrying out 5-by-5 Gaussian filtering denoising processing on the image.

3. The method for detecting the cylinder liner defect mark based on the deep learning of claim 2, wherein the method for acquiring the local defect map in the step 2) is to artificially intercept the local defect map with the size of 256 × 256 by using the labeling information of the images in the training set, and each local defect map contains defects.

4. The cylinder liner defect mark detection method based on deep learning of claim 1, wherein the network model establishing method in step 3) is as follows:

301) inputting the local defect map in the training set into a Swin converter to obtain a corresponding feature map characteristic map;

302) setting a preset number of ROI (regions of interest) for each pixel point in the feature map to obtain a plurality of candidate ROIs;

303) sending the candidate ROI into a Region Proposal Network for processing, and performing Bounding box regression to obtain a corresponding Proposals Region to be detected;

304) and accurately extracting ROI features of the original image from the obtained Proposals to-be-detected region by using an ROI Align method, performing Classification Classification, Boundingbox regression and Mask generation on the ROI features, and performing continuous iterative training to obtain a final network model.

5. The cylinder liner defect mark detection method based on deep learning of claim 1, wherein the detection method in step 4) is as follows: dividing each image in the test set into 64 small images, inputting the 64 small images of each image into a trained network model for detection and outputting a result, and finally re-splicing the output 64 small images back to the size of each image to obtain the detection result of each image.

6. The cylinder liner defect mark detection method based on deep learning of claim 5, wherein the method for extracting the region of interest by the masking mechanism in step 5) comprises:

A. for the images of the upper end cover of the cylinder liner, firstly, selecting a proper threshold value for each image of the upper end cover to carry out binarization, removing fine binary black points by using morphological operation closed operation, ensuring that a binarization area contains the whole interest area, and taking the binarization area as the interest area;

B. for the upper end surface image of the cylinder sleeve, firstly, carrying out coordinate measurement on each image of the upper end surface, and determining the circle center, the maximum circle and the minimum circle radius according to the to-be-detected area of the upper end surface image; drawing a circular ring binary image with the same size, and taking the circular ring binary image as an interest area of the upper end face image;

C. for the image of the inner wall of the cylinder sleeve, the defects of the image are all distributed on the upper half part, in order to better obtain defect information, a specific mask is designed according to the outline of the upper half part, the same as the extraction of interest areas on the upper end surface is carried out, coordinate measurement is carried out on the image, and the outline information of the upper half part is determined according to the area to be detected; drawing a binarization image with the same shape to extract the interest area.

7. The cylinder liner defect mark detection method based on deep learning of claim 5, wherein the method of evaluation in step 6) is as follows: the average precision MAP value is used for proving the detection performance of the network model,

rate of accuracy

Recall rate

Average accuracy

Mean average accuracy

Wherein TP is a correctly identified result, FP is an incorrectly identified result, FN is an unidentified result, but the system identifies other defects, AP is an area formed by a P-R curve, and the effect is better when the area is larger. The abscissa is recall, the ordinate is accuracy, AP is the result of integrating the accuracy, n is the total number of defect types, and i is the second defect.