CN112345539A - Aluminum die casting surface defect detection method based on deep learning - Google Patents

Aluminum die casting surface defect detection method based on deep learning Download PDF

Info

Publication number
CN112345539A
CN112345539A CN202011225995.3A CN202011225995A CN112345539A CN 112345539 A CN112345539 A CN 112345539A CN 202011225995 A CN202011225995 A CN 202011225995A CN 112345539 A CN112345539 A CN 112345539A
Authority
CN
China
Prior art keywords
deep learning
aluminum die
defects
die casting
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011225995.3A
Other languages
Chinese (zh)
Inventor
曹彬
胡江洪
袁帅鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fitow Tianjin Detection Technology Co Ltd
Original Assignee
Fitow Tianjin Detection Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fitow Tianjin Detection Technology Co Ltd filed Critical Fitow Tianjin Detection Technology Co Ltd
Priority to CN202011225995.3A priority Critical patent/CN112345539A/en
Publication of CN112345539A publication Critical patent/CN112345539A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30136—Metal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Abstract

The invention discloses an aluminum die casting surface defect detection method based on deep learning, and relates to the technical field of digital image processing. The method comprises the following steps: by adopting a photometric stereo method, a plurality of pictures illuminated at different angles are combined into a format picture, so that the defects of material shortage, scratch and the like are more obvious; and (3) training a target detection model by applying a deep learning method, and detecting defects in the image. The invention adopts a common FA lens and a customized annular light source to finish the polishing of the photometry. The annular light source is averagely divided into four parts, the light on of each part of light source is sequentially controlled by using a control program, only one part of light is lightened each time, and images are collected simultaneously. And after the images are collected, synthesizing the four images into one tiff format image by using a photometric stereo method. As can be seen from the composite image, defects such as scratches are clearly present.

Description

Aluminum die casting surface defect detection method based on deep learning
Technical Field
The invention relates to the technical field of digital image processing, in particular to a method for detecting surface defects of an aluminum die casting based on deep learning.
Background
The aluminum die casting has important application in the production of automobile gearboxes, the rapid development of the automobile industry drives the mass production of the aluminum die casting, the aluminum die casting is used as a key part of the gearboxes, and the surface quality of the aluminum die casting directly influences the assembly and the use performance of parts. Because of the influence of factors such as production equipment, casting technology, manual technology, external environment and the like in the casting process, the surface of the aluminum die casting inevitably generates defects such as air holes, cracks, scratches and the like, and the properties such as the appearance, the sealing property, the wear resistance and the like of a finished product are influenced, even the normal use of the gearbox is influenced.
At present, the detection of surface defects of aluminum die castings is still considered to be the main subject for human. In long-term single repetitive work, workers are prone to fatigue, resulting in the influx of off-specification products into the application market. The operation skills of different workers are different in width and strict, and subjective inconsistency is generated in judgment of qualification of the limit piece. The surface of the aluminum die casting is complex in shape and various in defect types, and water stains, oil stains, machining textures and the like in the production process are easily identified as defects by mistake, so that the detection result is interfered. The Femandez.C develops an online surface detection vision system for continuous casting aluminum alloy castings, adopts an area array CCD to collect images, and utilizes a similarity algorithm and a texture algorithm to realize detection and classification identification of various surface defects. And Robles.L adopts an improved fuzzy pattern recognition algorithm to classify and recognize the defects on the surface of the casting, and has good classification effect on the defects such as cracks, holes and the like. Newan et al have designed a set of automatic visual inspection systems for casting surfaces, suitable for measuring a large number of metal castings with surface defects. But the defects such as air holes, cracks, scratches and the like cannot be accurately identified due to inconsistent product states. With the recent increasingly wide application and better effect of deep learning in the field of industrial detection, the invention provides an aluminum die casting surface defect detection algorithm based on deep learning.
In view of the above analysis, the prior art has problems that:
(1) in the process of detecting surface defects of aluminum die castings, which is mainly regarded as a manual work, workers are prone to fatigue, and unqualified products flow into the application market.
(2) The operation skills of different workers are different in width and strict, and subjective inconsistency is generated in judgment of qualification of the limit piece.
(3) The surface of the aluminum die casting is complex in shape and various in defect types, and water stains, oil stains, machining textures and the like in the production process are easily identified as defects by mistake, so that the detection result is interfered.
The difficulty in solving the technical problems is as follows:
the aluminum die casting has the advantages that the research results for detecting the surface defects of the aluminum die casting are few, the most common air hole defects are small in size and difficult to accurately divide the defect area, and water stains, oil stains and the like generated in the production process are easy to be mistakenly identified as defects, so that the detection result of the traditional mode is interfered.
The aluminum die castings have various forms, so that the algorithm is required to have certain universality for reducing the enterprise generation cost and increasing the effective detection capability of the algorithm, and the aluminum die castings are required to be suitable for products of different models by marking, training and identifying defects.
The significance of solving the technical problems is as follows:
in consideration of the traditional manual detection mode and the traditional visual detection mode, the detection method provided by the text can adapt to defects of different surface forms and different types, reduces the influence of water stains and oil stains on an algorithm, and improves the detection efficiency.
The same set of algorithm model is suitable for detecting the defects of different types of aluminum die casting products, one-time labeling training is realized, the method is suitable for detecting all similar defects, the operation cost of enterprises is reduced, and the research and development efficiency of later-stage equipment is improved.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiment of the invention provides a method for detecting surface defects of an aluminum die casting based on deep learning. The technical scheme is as follows:
the aluminum die casting surface defect detection method based on deep learning comprises the following steps:
step one, synthesizing a plurality of pictures illuminated at different angles into a format picture by a photometric stereo method, so that the defects of material shortage, scratch and the like are more obvious;
and secondly, training a target detection model by applying a deep learning method, and detecting defects in the image.
In the first step, after the images are synthesized, a YOLO-v3 target detection algorithm is adopted, and the synthesized images are loaded into a preset network framework for training.
As a preferred embodiment of the present invention, in the first step, the relative position of the camera lens module between the objects is kept unchanged, the illumination angle of the light source is changed, and a plurality of pictures are taken;
by utilizing a photometric stereo method, a plurality of pictures are combined into a picture in tiff format, and as can be seen from the combined picture, the defects are more obvious after the combination than before the combination.
As a preferred embodiment of the invention, the illumination angle of the light source is varied to at least three different angles.
As a preferred embodiment of the present invention, photometric stereo uses the equation of the brightness of each point in the image, i.e. the equation of the illumination, which is mathematically described as follows:
I(x，y)＝kd(x，y)S·N(x，y) (1)
where I is the brightness of the surface point, S is the light source vector, N is the surface normal vector, kdIs the surface reflection coefficient;
three luminosity stereo images I are given1、I2、I3From the luminance equation (1), three equations for the gradient of the object surface can be obtained:
Figure BDA0002763672550000021
here Ii(x, y) represents the brightness of the image at the (x, y) point, Ri(p (x, y), q (x, y)) represents the reflection equation under different light sources;
let three light sources in directions of
Figure BDA0002763672550000031
The vector form of the image is I ═ I (I)1，I2，I3)TWriting equation (2) in the form of a matrix:
Figure BDA0002763672550000032
wherein the content of the first and second substances,
Figure BDA0002763672550000033
if the three light source direction vectors are not coplanar
Figure BDA0002763672550000036
The matrix is full rank, i.e., it is invertible, and then the normal vector N of the object surface can be obtained by solving equation (3):
Figure BDA0002763672550000034
as a preferred embodiment of the present invention, in step two, mapping the gray value of the tiff format picture from 0-1 to 0-255 to obtain a png format picture;
classifying all the collected pictures, dividing the pictures into a training set and a testing set according to the proportion of 7: 3, manually marking the training set, and classifying labels according to defect types;
after the labeling is finished, importing the labeled pictures into a YOLOv3 target detection algorithm for training, and obtaining a training model after the training is finished;
and loading the model into a prediction program, and starting to perform target detection after the image is transmitted.
In the process of the YOLOv3 target detection algorithm, a weighted K-means algorithm is used to cluster the target samples.
As the preferred embodiment of the invention, the weighted K-means algorithm distributes the clustering center and sets the weight parameter of each sample, and then carries out clustering calculation; and adjusting the network structure of YOLOv3, and adding a residual error unit and a large-scale characteristic image layer output.
As a preferred embodiment of the invention, the weighted K-means algorithm is realized by the following steps:
(1) setting central points of 3 feature maps, wherein each 1 central point corresponds to 3 clustering centers;
(2) and (3) calculating the distance between each sample and the central point, wherein the distance calculation formula is shown as the formula (5):
Figure BDA0002763672550000035
wherein, boxi(1)、boxi(2) Is the abscissa and ordinate of the ith sample, Cl(1)、Cl(2) The horizontal and vertical coordinates of the first central point;
(3) solving a weight matrix, namely the weight of each sample corresponding to each clustering center, wherein the clustering centers belonging to the same central point share the same weight; the weight is the opposite number of the sample after the distance from the sample to the clustering center is standardized by z-score, the weight of the sample with a longer distance is directly set to be 0, and the interference of the data unbalanced sample is reduced; equation (6) is the average value of the distances from all samples to a certain central point, and equation (7) is the calculation formula of the weight:
Figure BDA0002763672550000041
Figure BDA0002763672550000042
wherein l is the central point, dilIs the distance of the sample i from the center point l,
Figure BDA0002763672550000043
the average distance of all samples from the center point l;
(4) and iteratively updating all the clustering centers until the clustering centers are not changed any more.
As a preferred embodiment of the invention, on the basis of a YOLOv3 network, the output of the last detection layer in the FPN is up-sampled, fused with the shallow layer output added with a residual error unit, and convolved to form a new feature map.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
(1) the invention adopts a common FA lens and a customized annular light source to finish the polishing of the photometry. The annular light source is averagely divided into four parts, the light on of each part of light source is sequentially controlled by using a control program, only one part of light is lightened each time, and images are collected simultaneously. And after the images are collected, synthesizing the four images into one tiff format image by using a photometric stereo method. The composite image shows that the defects such as scratch are obviously shown, which indicates that the improved photometric polishing mode effect reaches the ideal expectation, and the detection capacity is up to 99%.
(2) And after the images are synthesized, loading the synthesized images into a preset network framework for training. After the training is completed, the obtained model is used for online defect detection.
(3) The invention uses a weighted K-means algorithm to replace the original K-means algorithm to cluster the target samples: firstly, a clustering center is distributed, a weight parameter of each sample is set, clustering calculation is carried out, the influence of unbalanced defect size on a clustering result is avoided, and the matching degree of a prior frame and a feature map layer is improved; meanwhile, the network structure of YOLOv3 is adjusted, a residual error unit and a large-scale characteristic layer are added for output, the detection capability of the algorithm on the surface defects of the aluminum die casting is improved, and the detection precision is improved.
(4) The same set of algorithm model has the advantages that the aluminum die casting surface defect utilization rate of different products is up to 98%, the algorithm research and development period is shortened and the production efficiency of enterprises is improved according to the research and development of new equipment.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow chart of a method for detecting surface defects of an aluminum die casting based on deep learning provided by the invention.
FIG. 2 is a schematic view of an aluminum die cast part of the invention provided in an embodiment of the invention;
FIG. 3 is a schematic view of a lighting environment according to the invention provided in an embodiment of the present invention;
FIG. 4 is a diagram of illumination at various angles involved in the invention provided in an embodiment of the invention;
wherein, a, 0 degree diagram; b. a 90 ° view; c. 180 degree graph; d. 270 degree graph;
FIG. 5 is a synthetic diagram relating to photometry in the invention provided in an embodiment of the present invention;
FIG. 6 is a diagram of a process involved in tagging in the invention provided in an embodiment of the present invention;
fig. 7 is a diagram of a network structure related to improved YOLOv3 in the invention provided in the embodiment of the present invention;
FIG. 8 is a plot of loss associated with the training process of the invention provided in an embodiment of the present invention;
FIG. 9 is a model involved in the invention provided in an embodiment of the invention;
FIG. 10 is a diagram of a deep learning object detection map of the invention provided in an embodiment of the present invention;
FIG. 11 is an enlarged view of a portion of a deep learning object detection map of the invention as provided in an embodiment of the present invention;
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather should be construed as broadly as the present invention is capable of modification in various respects, all without departing from the spirit and scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. As used herein, the terms "vertical," "horizontal," "left," "right," and the like are for purposes of illustration only and are not intended to represent the only embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The embodiment of the invention provides an aluminum die casting surface defect detection method based on deep learning, and aims to solve the technical problems of low accuracy and low precision of aluminum die casting defect detection in the prior art. The invention fully utilizes an industrial light source and an industrial camera, adopts a machine vision detection method, realizes the quick and accurate detection of the surface defects of the aluminum die casting, and processes data in real time. The technical scheme of the invention is as follows:
the method comprises the following steps: by adopting a photometric stereo method, a plurality of pictures illuminated at different angles are combined into a picture with a special format, so that the defects of material shortage, scratch and the like are more obvious; and (3) training a target detection model by applying a deep learning method, and detecting defects in the image.
In the method, a telecentric lens and a telecentric light source are adopted in the traditional photometric stereo method, and because the area of the shooting object in the invention is large (120mm x 120mm), correspondingly, the telecentric lens with a large shooting visual field range and the telecentric light source with a large light irradiation range need to be used. However, such telecentric lenses and telecentric light sources not only occupy space and are expensive, but also further increase the mechanical cost, resulting in a sudden increase in the total cost. The improved polishing mode not only ensures the defective shooting effect, but also saves the space and the cost.
And after the images are synthesized, loading the synthesized images into a preset network framework for training. The improved YOLO-v3 target detection algorithm is adopted, and the detection precision is improved.
The embodiment provides a machine vision-based aluminum die casting defect detection method, and please refer to fig. 1, the method includes:
step S101: by the photometric stereo method, a plurality of pictures illuminated at different angles are combined into a picture with a special format, so that the defects of material shortage, scratch and the like are more obvious.
Step S102: and (3) training a target detection model by applying a deep learning method, and detecting defects in the image.
The machine vision-based aluminum die casting defect detection method provided by the application is described in detail below with reference to fig. 1:
firstly, step S101 is executed, a plurality of pictures illuminated at different angles are combined into a picture with a special format through a photometric stereo method, and defects such as material shortage and scratches are more obvious.
Specifically, the above-mentioned pair of pictures illuminated at different angles are combined into a picture of a special format by a photometric stereo method, so that defects such as material shortage and scratches are more obvious, and the method specifically includes:
the relative position of the camera lens module between the shot objects is kept unchanged, the illumination angle of the light source is changed (at least three different angles, preferably four to six angles), and a plurality of pictures are shot. As shown in fig. 4.
By utilizing a photometric stereo method, a plurality of pictures are combined into a picture with a special format (tiff format), and as can be seen from the combined picture, the defects are more obvious after the combination than before the combination. As shown in fig. 5.
The core of the photometric stereo method is the brightness equation of each point in the image, namely the irradiation equation, and the mathematical description is as follows:
I(x，y)＝kd(x，y)S·N(x，y) (1)
where I is the brightness of the surface point, S is the light source vector, N is the surface normal vector, kdIs the surface reflection coefficient.
Three luminosity stereo images I are given1、I2、I3From the luminance equation (1), three equations for the gradient of the object surface can be obtained:
Figure BDA0002763672550000061
here Ii(x, y) represents the brightness of the image at the (x, y) point, Ri(p (x, y), q (x, y)) represents the reflection equation under different light sources.
Let three light sources in directions of
Figure BDA0002763672550000071
The vector form of the image is I ═ I (I)1，I2，I3)T. Writing equation (2) in the form of a matrix:
Figure BDA0002763672550000072
wherein the content of the first and second substances,
Figure BDA0002763672550000073
if the three light source direction vectors are not coplanar
Figure BDA0002763672550000074
The matrix is full rank, i.e., it is invertible, and then the normal vector N of the object surface can be obtained by solving equation (3):
Figure BDA0002763672550000075
in summary, if three photometric stereo images of the surface of an object can be obtained, the normal vector field N (x, y) of the surface can be directly obtained.
Then, step S102 is executed: and (3) training a target detection model by applying a deep learning method, and detecting defects in the image.
In a specific implementation process, since the picture in the tiff format cannot be directly applied to labeling and training of deep learning, the picture in the tiff format needs to be converted into the png format. The specific operation is that the gray value of the tiff format picture is mapped to 0-255 from 0-1 to obtain the png format picture.
Classifying all the collected pictures, dividing the pictures into a training set and a testing set according to the proportion of 7: 3, manually marking the training set, and classifying labels according to defect types. As shown in fig. 6.
After the labeling is finished, the labeled picture is imported into a Yolov3 target detection algorithm for training, and a loss curve in the training process is shown in FIG. 7.
After the training is completed, a training model is obtained, as shown in fig. 8. The model is loaded into the prediction program, and after the image is transmitted, the target detection is started, and the detection result is shown in fig. 9.
In order to improve the detection accuracy of YOLOv3, the algorithm is improved by the invention. And (3) clustering the target samples by using a weighted K-means algorithm to replace the original K-means algorithm: firstly, a clustering center is distributed, a weight parameter of each sample is set, clustering calculation is carried out, the influence of unbalanced defect size on a clustering result is avoided, and the matching degree of a prior frame and a feature map layer is improved; meanwhile, the network structure of YOLOv3 is adjusted, a residual error unit and a large-scale feature layer are added for output, and the detection capability of the algorithm on the surface defects of the aluminum die casting is improved.
The specific implementation steps of the weighted K-means algorithm are as follows:
(1) and setting central points of 3 feature maps, wherein each 1 central point corresponds to 3 clustering centers.
(2) And (3) calculating the distance between each sample and the central point, wherein the distance calculation formula is shown as the formula (5):
Figure BDA0002763672550000081
wherein, boxi(1)、boxi(2) Is the abscissa and ordinate of the ith sample, Cl(1)、Cl(2) The abscissa and ordinate of the ith central point.
(3) And (4) solving a weight matrix, namely the weight of each sample corresponding to each clustering center, wherein the clustering centers belonging to the same central point share the same weight. The weight is the inverse number of the sample after the distance from the clustering center to the clustering center is standardized by z-score, and the weight of the sample with a longer distance is directly set to be 0, so that the interference of data unbalanced samples is reduced. Equation (6) is the average value of the distances from all samples to a certain central point, and equation (7) is the calculation formula of the weight:
Figure BDA0002763672550000082
Figure BDA0002763672550000083
wherein l is the central point, dilIs the distance of the sample i from the center point l,
Figure BDA0002763672550000084
the average distance from the center point/is taken for all samples.
(4) And iteratively updating all the clustering centers until the clustering centers are not changed any more.
In the problem of surface defect detection of aluminum die castings, different types of defects are greatly different in form, size and the like, and if the original Yolov3 network is used for detection, the detection effect of tiny defects is poor. Taking a picture with a size of 256x256 as an example, the final grid sizes of the original YOLOv3 network are 32x32, 16x16 and 8x8 respectively, the maximum number of preselection frames that can be obtained is only 1344, and the detection requirement is difficult to meet.
The structure of the improved YOLOv3 network is shown in fig. 7. The CONV + BN + LeakyRelu is the integration of the convolution layer, batch normalization and the activation function layer LeakyRelu, and batch normalization is added after all convolution layers, so that the training process is accelerated, the performance is improved, and the problem of gradient disappearance can be relieved. Res refers to a residual unit, and the structure is helpful for solving the problems of gradient explosion and gradient disappearance after the number of network layers is increased. Upsampling is an Upsampling layer, the picture is enlarged by using an interpolation method, and all Upsampling layers in the picture enlarge the picture to 2 times of the original picture. Out is the finally obtained characteristic image layer.
On the basis of an original YOLOv3 network, the invention adjusts the network structure, samples the output of the last detection layer in the FPN, fuses with the shallow layer output added with a residual error unit, and forms a new characteristic diagram after convolution, wherein an improved part is arranged in a dotted line frame in figure 7. The new feature map not only inherits deep features output by the original YOLOv3 network, but also contains shallow features, and the feature extraction capability is stronger. The width and the height of the newly obtained characteristic scale are 1/4 of the original image respectively, the number of the networks is 64x64, the size of the grid is only 4x4, and the detection capability of the improved network on small targets is stronger. Meanwhile, keeping the number of the prior boxes on each feature map to be 3, the total number of the prior boxes is increased from 9 to 12. Compared with the original network structure, the number of feature maps obtained by using the improved network structure is increased from 3 to 4, and the maximum pre-selected frame number on each picture can reach 5440. The increase of the number of the characteristic graphs can better divide the defect size layer and enhance the detection capability of the network to the defects with different sizes. And the improvement of the number of the preselection frames on each picture can increase the detection density, prevent the occurrence of defects and missing detection and influence on the detection precision.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure should be limited only by the attached claims.

Claims (10)

1. The method for detecting the surface defects of the aluminum die castings based on deep learning is characterized by comprising the following steps of:
step one, synthesizing a plurality of pictures illuminated at different angles into a format picture by a photometric stereo method, so that the defects of material shortage and scratching are more obvious;
and secondly, training a target detection model by applying a deep learning method, and detecting defects in the image.
2. The method for detecting the surface defects of the aluminum die castings based on the deep learning of claim 1, wherein in the step one, after the images are synthesized, the synthesized images are loaded into a preset network framework for training by using a YOLO-v3 target detection algorithm.
3. The method for detecting the surface defects of the aluminum die casting based on the deep learning as claimed in claim 1, wherein in the first step, the relative position of a camera lens module and a shot object is kept unchanged, the illumination angle of a light source is changed, and a plurality of pictures are taken;
and (3) synthesizing a plurality of pictures into a picture in tiff format by using a photometric stereo method.
4. The aluminum die casting surface defect detection method based on deep learning of claim 3, wherein the light source is changed to at least three different illumination angles.
5. The aluminum die casting surface defect detection method based on deep learning of claim 3, wherein the photometric stereo method adopts a brightness equation, namely an irradiation equation, of each point in the image, and the mathematical description is as follows:
I(x，y)＝kd(x，y)S·N(x，y) (1)
where I is the brightness of the surface point, S is the light source vector, N is the surface normal vector, kdIs the surface reflection coefficient;
three luminosity stereo images I are given1、I2、I3From the luminance equation (1), three equations for the gradient of the object surface can be obtained:
Figure FDA0002763672540000011
wherein Ii(x, y) represents the brightness of the image at the (x, y) point, Ri(p (x, y), q (x, y)) represents the reflection equation under different light sources;
let three light sources in directions of
Figure FDA0002763672540000012
The vector form of the image is I ═ I (I)1，I2，I3)TWriting equation (2) in the form of a matrix:
Figure FDA0002763672540000021
wherein the content of the first and second substances,
Figure FDA0002763672540000022
if the three light source direction vectors are not coplanar
Figure FDA0002763672540000023
The matrix is full rank, i.e. it is invertible, when solved(3) The normal vector N of the surface of the obtained object is as follows:
Figure FDA0002763672540000024
6. the aluminum die casting surface defect detection method based on deep learning of claim 1, wherein in the second step, the gray value of the tiff format picture is mapped from 0-1 to 0-255, and a png format picture is obtained;
classifying all collected pictures according to the following steps of 7: 3, dividing the proportion into a training set and a test set, manually labeling the training set, and classifying the labels according to the defect types;
after the labeling is finished, importing the labeled pictures into a YOLOv3 target detection algorithm for training, and obtaining a training model after the training is finished;
and loading the model into a prediction program, and starting to perform target detection after the image is transmitted.
7. The aluminum die casting surface defect detection method based on deep learning of claim 6, wherein in the processing process of the YOLOv3 target detection algorithm, a weighted K-means algorithm is used for clustering target samples.
8. The aluminum die casting surface defect detection method based on deep learning of claim 7, wherein a weighted K-means algorithm is used for distributing clustering centers, setting weight parameters of each sample and then carrying out clustering calculation; and adjusting the network structure of YOLOv3, and adding a residual error unit and a large-scale characteristic image layer output.
9. The aluminum die casting surface defect detection method based on deep learning of claim 7, wherein the weighted K-means algorithm is realized by the following steps:
(1) setting central points of 3 feature maps, wherein each 1 central point corresponds to 3 clustering centers;
(2) and (3) calculating the distance between each sample and the central point, wherein the distance calculation formula is shown as the formula (5):
Figure FDA0002763672540000025
wherein, boxi(1)、boxi(2) Is the abscissa and ordinate of the ith sample, Cl(1)、Cl(2) The horizontal and vertical coordinates of the first central point;
(3) solving a weight matrix, namely the weight of each sample corresponding to each clustering center, wherein the clustering centers belonging to the same central point share the same weight; the weight is the opposite number of the sample after the distance from the sample to the clustering center is standardized by z-score, the weight of the sample with a longer distance is directly set to be 0, and the interference of the data unbalanced sample is reduced; equation (6) is the average value of the distances from all samples to a certain central point, and equation (7) is the calculation formula of the weight:
Figure FDA0002763672540000031
Figure FDA0002763672540000032
wherein l is the central point, dilIs the distance of the sample i from the center point l,
Figure FDA0002763672540000033
the average distance of all samples from the center point l;
(4) and iteratively updating all the clustering centers until the clustering centers are not changed any more.
10. The method for detecting the surface defects of the aluminum die castings based on the deep learning of claim 6, wherein the output of the last detection layer in the FPN is up-sampled on the basis of a YOLOv3 network, is fused with the shallow output added with the residual error unit, and is convolved to form a new feature map.
CN202011225995.3A 2020-11-05 2020-11-05 Aluminum die casting surface defect detection method based on deep learning Pending CN112345539A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011225995.3A CN112345539A (en) 2020-11-05 2020-11-05 Aluminum die casting surface defect detection method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011225995.3A CN112345539A (en) 2020-11-05 2020-11-05 Aluminum die casting surface defect detection method based on deep learning

Publications (1)

Publication Number Publication Date
CN112345539A true CN112345539A (en) 2021-02-09

Family

ID=74429864

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011225995.3A Pending CN112345539A (en) 2020-11-05 2020-11-05 Aluminum die casting surface defect detection method based on deep learning

Country Status (1)

Country Link
CN (1) CN112345539A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113111828A (en) * 2021-04-23 2021-07-13 中国科学院宁波材料技术与工程研究所 Three-dimensional defect detection method and system for bearing
CN113155852A (en) * 2021-04-08 2021-07-23 煤炭科学研究总院 Transmission band detection method and device and electronic equipment
CN113252567A (en) * 2021-06-08 2021-08-13 菲特（天津）检测技术有限公司 Method, system, medium and terminal for rapidly detecting multiple defects on surface of aluminum valve plate
CN113414784A (en) * 2021-06-17 2021-09-21 东莞理工学院 Robot micro-assembly grabbing system based on deep reinforcement learning
CN115170804A (en) * 2022-07-26 2022-10-11 无锡九霄科技有限公司 Surface defect detection method, device, system and medium based on deep learning
WO2022222120A1 (en) * 2021-04-23 2022-10-27 中国科学院宁波材料技术与工程研究所 Bearing three-dimensional defect detection method and system
CN115656189A (en) * 2022-12-12 2023-01-31 易斯德(福建)智能科技有限公司 Defect detection method and device based on luminosity stereo and deep learning algorithm
CN116091506A (en) * 2023-04-12 2023-05-09 湖北工业大学 Machine vision defect quality inspection method based on YOLOV5
CN116930196A (en) * 2023-09-18 2023-10-24 山东卓越精工集团有限公司 Machine vision-based aluminum profile production defect analysis processing method
CN116977341A (en) * 2023-09-25 2023-10-31 腾讯科技（深圳）有限公司 Dimension measurement method and related device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107378780A (en) * 2017-07-19 2017-11-24 江苏大学 A kind of robot casting grinding adaptive approach of view-based access control model system
CN110222681A (en) * 2019-05-31 2019-09-10 华中科技大学 A kind of casting defect recognition methods based on convolutional neural networks
CN110660052A (en) * 2019-09-23 2020-01-07 武汉科技大学 Hot-rolled strip steel surface defect detection method based on deep learning
CN111223088A (en) * 2020-01-16 2020-06-02 东南大学 Casting surface defect identification method based on deep convolutional neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107378780A (en) * 2017-07-19 2017-11-24 江苏大学 A kind of robot casting grinding adaptive approach of view-based access control model system
CN110222681A (en) * 2019-05-31 2019-09-10 华中科技大学 A kind of casting defect recognition methods based on convolutional neural networks
CN110660052A (en) * 2019-09-23 2020-01-07 武汉科技大学 Hot-rolled strip steel surface defect detection method based on deep learning
CN111223088A (en) * 2020-01-16 2020-06-02 东南大学 Casting surface defect identification method based on deep convolutional neural network

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
李维刚: ""基于改进 YOLOv3 算法的带钢表面缺陷检测"", 《电子学报》 *
杨敬安: "《三维计算机视觉》", 31 August 1994 *
林慕清: ""基于光度立体法的三维重建技术的研究与实现"", 《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》 *
王磊: ""基于光度立体的金属板带表面缺陷三维检测方法"", 《中国优秀博硕士学位论文全文数据库(博士) 工程科技Ⅰ辑》 *
石月晴: ""基于图像处理的沥青路面抗滑性能研究"", 《中国优秀博硕士学位论文全文数据库(硕士) 工程科技Ⅱ辑》 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113155852A (en) * 2021-04-08 2021-07-23 煤炭科学研究总院 Transmission band detection method and device and electronic equipment
WO2022222120A1 (en) * 2021-04-23 2022-10-27 中国科学院宁波材料技术与工程研究所 Bearing three-dimensional defect detection method and system
CN113111828A (en) * 2021-04-23 2021-07-13 中国科学院宁波材料技术与工程研究所 Three-dimensional defect detection method and system for bearing
CN113252567A (en) * 2021-06-08 2021-08-13 菲特（天津）检测技术有限公司 Method, system, medium and terminal for rapidly detecting multiple defects on surface of aluminum valve plate
CN113414784A (en) * 2021-06-17 2021-09-21 东莞理工学院 Robot micro-assembly grabbing system based on deep reinforcement learning
CN115170804B (en) * 2022-07-26 2024-01-26 无锡九霄科技有限公司 Surface defect detection method, device, system and medium based on deep learning
CN115170804A (en) * 2022-07-26 2022-10-11 无锡九霄科技有限公司 Surface defect detection method, device, system and medium based on deep learning
CN115656189A (en) * 2022-12-12 2023-01-31 易斯德(福建)智能科技有限公司 Defect detection method and device based on luminosity stereo and deep learning algorithm
CN116091506A (en) * 2023-04-12 2023-05-09 湖北工业大学 Machine vision defect quality inspection method based on YOLOV5
CN116930196A (en) * 2023-09-18 2023-10-24 山东卓越精工集团有限公司 Machine vision-based aluminum profile production defect analysis processing method
CN116930196B (en) * 2023-09-18 2023-12-22 山东卓越精工集团有限公司 Machine vision-based aluminum profile production defect analysis processing method
CN116977341A (en) * 2023-09-25 2023-10-31 腾讯科技（深圳）有限公司 Dimension measurement method and related device
CN116977341B (en) * 2023-09-25 2024-01-09 腾讯科技（深圳）有限公司 Dimension measurement method and related device

Similar Documents

Publication Publication Date Title
CN112345539A (en) Aluminum die casting surface defect detection method based on deep learning
CN113450307B (en) Product edge defect detection method
CN106875373B (en) Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm
CN111612737B (en) Artificial board surface flaw detection device and detection method
CN111007073A (en) Method and system for online detection of part defects in additive manufacturing process
CN107677216B (en) A kind of multiple abrasive grain three-dimensional appearance synchronous obtaining methods based on photometric stereo vision
CN112308832A (en) Bearing quality detection method based on machine vision
JP2021515885A (en) Methods, devices, systems and programs for setting lighting conditions and storage media
CN108985337A (en) A kind of product surface scratch detection method based on picture depth study
CN110110131B (en) Airplane cable support identification and parameter acquisition method based on deep learning and binocular stereo vision
CN115035082B (en) Method for detecting defects of transparent parts of aircraft based on YOLOv4 improved algorithm
CN114119591A (en) Display screen picture quality detection method
CN113327243B (en) PAD light guide plate defect visual detection method based on Ayolov3-Tiny new framework
CN116071315A (en) Product visual defect detection method and system based on machine vision
CN112381751A (en) Online intelligent detection system and method based on image processing algorithm
CN113706496B (en) Aircraft structure crack detection method based on deep learning model
CN114841992A (en) Defect detection method based on cyclic generation countermeasure network and structural similarity
CN114372980A (en) Industrial defect detection method and system
CN110458231B (en) Ceramic product detection method, device and equipment
CN116843618A (en) Method for detecting shallow apparent appearance defects of metal parts
CN117169247A (en) Metal surface defect multi-dimensional detection method and system based on machine vision
CN111161227A (en) Target positioning method and system based on deep neural network
CN212646436U (en) Artificial board surface flaw detection device
CN116543247A (en) Data set manufacturing method and verification system based on photometric stereo surface reconstruction
CN116051808A (en) YOLOv 5-based lightweight part identification and positioning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210209