CN111815555A - Metal additive manufacturing image detection method and device combining anti-neural network with local binary - Google Patents
Metal additive manufacturing image detection method and device combining anti-neural network with local binary Download PDFInfo
- Publication number
- CN111815555A CN111815555A CN202010442941.6A CN202010442941A CN111815555A CN 111815555 A CN111815555 A CN 111815555A CN 202010442941 A CN202010442941 A CN 202010442941A CN 111815555 A CN111815555 A CN 111815555A
- Authority
- CN
- China
- Prior art keywords
- image
- molten pool
- sputtering
- repaired
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 63
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 63
- 239000000654 additive Substances 0.000 title claims abstract description 46
- 230000000996 additive effect Effects 0.000 title claims abstract description 46
- 239000002184 metal Substances 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 title claims description 7
- 238000004544 sputter deposition Methods 0.000 claims abstract description 145
- 230000007547 defect Effects 0.000 claims abstract description 141
- 238000000034 method Methods 0.000 claims abstract description 40
- 230000003042 antagnostic effect Effects 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 19
- 238000001914 filtration Methods 0.000 claims description 16
- 230000011218 segmentation Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 239000000463 material Substances 0.000 claims description 2
- 230000008569 process Effects 0.000 description 11
- 230000002950 deficient Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000003754 machining Methods 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000009659 non-destructive testing Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B22—CASTING; POWDER METALLURGY
- B22F—WORKING METALLIC POWDER; MANUFACTURE OF ARTICLES FROM METALLIC POWDER; MAKING METALLIC POWDER; APPARATUS OR DEVICES SPECIALLY ADAPTED FOR METALLIC POWDER
- B22F3/00—Manufacture of workpieces or articles from metallic powder characterised by the manner of compacting or sintering; Apparatus specially adapted therefor ; Presses and furnaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B33—ADDITIVE MANUFACTURING TECHNOLOGY
- B33Y—ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3-D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3-D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING
- B33Y50/00—Data acquisition or data processing for additive manufacturing
-
- G06T5/77—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]
Abstract
The invention provides a method and a device for detecting a metal additive manufacturing image by combining an anti-neural network with local binary, wherein the method comprises the following steps: acquiring a molten pool and a sputtering image for metal additive manufacturing, and preprocessing the molten pool and the sputtering image; repairing the defects in the pretreated molten pool and the sputtering image by using the trained generative antagonistic neural network to obtain a repaired molten pool and a repaired sputtering image; and acquiring LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image by using a local binary pattern algorithm, identifying the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image according to the numerical difference of the LBP values of the molten pool and the original sputtering image, wherein the image of the difference part is the image of the defect area, and accurately positioning and identifying the defect area according to the image of the defect area. The invention can position and identify the defects in the molten pool and the sputtering image in the metal additive manufacturing process, thereby adjusting the manufacturing process parameters in real time and improving the yield of part manufacturing.
Description
Technical Field
The invention relates to the field of metal additive manufacturing missing image detection, in particular to a method and a device for detecting a metal additive manufacturing image by combining an anti-neural network with local binary.
Background
The metal additive manufacturing technology is increasingly widely applied to the high-end manufacturing fields of aerospace manufacturing industry, medical instruments and the like, and the nondestructive testing technology for monitoring the quality of the formed parts is increasingly required to be high in accuracy, instantaneity, operability and the like. In recent years, many researches on additive manufacturing defect detection have been progressed, and most of the researches are to obtain defect information through real-time sensing and measurement of information of a part machining area by various sensors and through a series of data processing flows, so that the defect information is used for assisting in quality detection of manufactured parts to remove unqualified parts or is directly fed back to a machining center of an additive manufacturing system to regulate and control process parameters. The yield of the current metal additive manufacturing product is about 70%, and currently in the aerospace field, the time consumption is about several days to several months, because the device is a large-size component. Therefore, the reliability of the quality is particularly important, and it is needed to monitor the molten pool, sputtering and the like in the metal additive manufacturing process in a deep learning manner, find out the unqualified area in time, and further improve the additive manufacturing yield. Therefore, many research institutes have studied this in recent years both at home and abroad. At present, the pennsylvania state university mechanical engineering system, the aachen industry university image and computer vision research institute, the canaimeron university mechanical engineering system, and the like have studied the dimensional deviation of different metal additive manufacturing defects corresponding to different machine learning algorithms. The university of toronto, the university of virginia, the university of south florida and the like have correspondingly studied different mechanical learning algorithms for the same metal additive manufacturing defect identification running time. The Huazhong science and technology university, the Singapore national university, the Qinghua university and the like perform relevant research on different metal additive manufacturing defect classifications by different mechanical learning algorithms. However, the existing mechanical learning program is difficult to consider defect positioning, area measurement and calculation and stable and efficient identification speed in the metal additive manufacturing process, and finally influences identification efficiency and identification precision.
Disclosure of Invention
In order to solve the technical problems in the prior art, the invention provides a method and a device for detecting a metal additive manufacturing image by combining an anti-neural network with local binary.
The invention is realized by the following steps:
in one aspect, the invention provides a method for detecting a metal additive manufacturing image by combining an anti-neural network with local binary, which comprises the following steps:
s1, acquiring a molten pool and a sputtering image for metal additive manufacturing, and preprocessing the molten pool and the sputtering image;
s2, repairing the defects in the pretreated molten pool and the sputtering image by using the trained generative antagonistic neural network to obtain a repaired molten pool and a repaired sputtering image;
s3, LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image are obtained through a local binary pattern algorithm, the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image is recognized according to the numerical difference of the LBP values of the molten pool and the original sputtering image, the image of the difference portion is the image of the defect area, and the defect area is accurately positioned and recognized according to the image of the defect area.
Further, the preprocessing of the molten pool and the sputtering image in the step S1 specifically includes:
and carrying out preprocessing operations of graying and smooth filtering on the molten pool and the sputtering image.
Further, the repairing the defects in the preprocessed molten pool and the preprocessed sputtering image by using the trained generative countermeasure neural network in the step S2 specifically includes:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
Further, the step S3 of precisely locating and identifying the defect area according to the image of the defect area specifically includes:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library to obtain the defect type.
Further, the method also comprises the step of measuring the size of the molten pool, and the specific method is as follows:
matching the template image, establishing a measuring frame ROI, and processing and identifying picture features in the measuring frame; for a molten pool image to be measured, firstly, drawing a gray level histogram along the length direction in an ROI (region of interest) and smoothing the gray level histogram by utilizing Gaussian filtering, then judging the edge of the molten pool according to the gray level change condition on the gray level histogram and establishing a measurement edge pair, and meanwhile, calculating the size of the molten pool according to the number of pixels between adjacent edges in the edge pair.
Further, the method also comprises the step of counting the number of the sputtered materials, and the specific method comprises the following steps:
and generating a confidence density map required by the sputtering image to be counted by using a generating type antagonistic neural network, performing antagonistic generation network training, comparing the confidence density map with the sputtering images in the image library, and reading the sputtering number prestored in the corresponding image in the image library after comparison, namely the sputtering number of the sputtering image to be counted.
In another aspect, the present invention further provides a metal additive manufacturing image processing apparatus for resisting neural network combining local binary, including:
the image preprocessing module is used for acquiring a molten pool and a sputtering image for metal additive manufacturing and preprocessing the molten pool and the sputtering image;
the generating type countermeasure neural network module is used for repairing the defects in the pretreated molten pool and the sputtering image by utilizing the trained generating type countermeasure neural network to obtain the repaired molten pool and the repaired sputtering image;
and the local binary pattern algorithm module is used for acquiring LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image by using a local binary pattern algorithm, identifying the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image according to the numerical difference of the LBP values of the molten pool and the original sputtering image, wherein the image of the difference part is the image of the defect area, and accurately positioning and identifying the defect area according to the image of the defect area.
Further, the image preprocessing module is specifically configured to:
and carrying out preprocessing operations of graying and smooth filtering on the molten pool and the sputtering image.
Further, the repairing of the defects in the pretreated molten pool and the sputtering image by the generated antagonistic neural network module using the trained generated antagonistic neural network specifically comprises:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
Further, the step of accurately positioning and identifying the defect region by the local binary pattern algorithm module according to the image of the defect region specifically includes:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library to obtain the defect type.
Compared with the prior art, the invention has the following beneficial effects:
the method and the device for detecting the metal additive manufacturing image by combining the antagonistic neural network with the local binary can position and identify the defects in a molten pool and a sputtering image in the metal additive manufacturing process, so that the result of segmentation and identification according to the defect example is fed back to a manufacturing process parameter control system in real time in the additive manufacturing process, the manufacturing process parameters are adjusted in real time, the yield of part manufacturing is improved, and parts which do not meet the process quality requirements of the parts can be stopped in time to reduce the manufacturing cost. The recognition perception information of the defects can be used as environmental data for regulating and controlling the process parameters by a reinforcement learning method, so that an intelligent agent (a decision system of the process parameters) makes an optimal decision, and integrated manufacturing from perception to decision is realized.
Drawings
Fig. 1 is a flowchart of a method for detecting a metal additive manufacturing image by combining an anti-neural network with local binarization according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of image processing provided by an embodiment of the present invention;
FIG. 3 is a pre-processed weld puddle and sputter image provided by an embodiment of the present invention;
FIG. 4 is an image of a molten pool region after a generative countering neural network segmentation provided by an embodiment of the present invention;
FIG. 5 is a schematic drawing of a weld puddle profile extraction provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram of sputter profile extraction provided by an embodiment of the present invention;
fig. 7 is a block diagram of a metal additive manufacturing image detection apparatus combining a neural network with local binarization according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for detecting a metal additive manufacturing image by combining an anti-neural network with local binarization, including the following steps:
s1, acquiring a molten pool and a sputtering image for metal additive manufacturing, and preprocessing the molten pool and the sputtering image;
s2, repairing the defects in the pretreated molten pool and the sputtering image by using a trained generative adaptive neural network (GAN) to obtain a repaired molten pool and a repaired sputtering image;
s3, obtaining LBP values of the molten pool and the original sputter image (as shown in fig. 2(a)) and the repaired molten pool and the repaired sputtering image (as shown in fig. 2(b)) by using a Local Binary Pattern (LBP) algorithm, identifying a difference between the molten pool and the original sputter image and the repaired molten pool and the repaired sputtering image according to a difference between the LBP values of the molten pool and the repaired molten pool, wherein an image of the difference portion (as shown in fig. 2(c)) is an image of a defect region, and accurately locating and identifying the defect region according to the image of the defect region.
The method and the device for detecting the metal additive manufacturing image by combining the anti-neural network with the local binary can position and identify the defects in a molten pool and a sputtering image in the metal additive manufacturing process, so that the result of segmentation and identification according to the defect example is fed back to a manufacturing process parameter control system in real time in the additive manufacturing process, the manufacturing process parameters are adjusted in real time, the yield of part manufacturing is improved, and parts which do not meet the process quality requirements of the parts can be stopped in time to reduce the manufacturing cost. The recognition perception information of the defects can be used as environmental data for regulating and controlling the process parameters by a reinforcement learning method, so that an intelligent agent (a decision system of the process parameters) makes an optimal decision, and integrated manufacturing from perception to decision is realized.
The above steps are explained in detail below.
The step S1 of preprocessing the molten pool and the sputtered image specifically includes: and carrying out image preprocessing operations such as graying, smooth filtering and the like on the molten pool and the sputtering image. The gray processing converts a color picture into a gray picture, and the storage information of a single pixel of the picture is converted into a single gray value (0-255) through the organic combination of red (R), green (G) and blue (B), so that the subsequent further processing of image information is facilitated, and the storage space of the picture is reduced to one third of the original storage space. The commonly used graying processing mainly includes three processing modes of average value, maximum value and weighted graying. Smoothing filtering is used to suppress or eliminate the interference in the image, separate out useful information, obtain a smoother curve, and enable the identified image to be more accurate after filtering. The commonly used filtering methods are linear filtering and median filtering, and the median filtering is preferably used in this embodiment. The pretreated puddle and sputter image are shown in FIG. 3.
In step S2, a generative countering neural network is first trained by using a training set composed of a defective molten pool and a defective sputtered image and a non-defective molten pool and a defective sputtered image to obtain a generative countering neural network with defect repairing capability, where the generative countering neural network includes a generative network G and a discrimination network D, and the training process includes:
inputting the defective molten pool and sputtering image X in the training set into a generating network G to generate a repaired molten pool and sputtering image G (X);
inputting a molten pool and sputtering image Y without defects in the training set and the repaired molten pool and sputtering image G (X) into the judgment network D to obtain judgment results D (Y) and D (G (X));
calculating and generating a loss function G _ loss of the network and a loss function D _ loss of the discrimination network according to the discrimination results D (Y) and D (G (X));
and respectively updating the generating network G and the judging network D according to the loss function G _ loss of the generating network and the loss function D _ loss of the judging network until the training is finished.
In the training process, the aim of generating the network G is to generate a real picture as much as possible to deceive the discrimination network D, and the aim of discriminating the network D is to separate the picture generated by the network G and the real picture as much as possible, so that the network G and the network D form a dynamic game process. Through a certain amount of sample training, the generated network G learns to obtain the defect repairing capability.
Further, the repairing the defects in the preprocessed molten pool and the preprocessed sputtering image by using the trained generative countermeasure neural network in the step S2 specifically includes:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region; the image of the puddle region after the generated antagonistic neural network segmentation is shown in FIG. 3;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
In the embodiment, the molten pool area and the sputtering area are classified through image semantic segmentation, so that the molten pool area and the sputtering area can be conveniently and respectively processed in a follow-up manner.
In step S3, the principle of the LBP algorithm is: the original LBP operator is defined as that in a window of 3 × 3, the central pixel of the window is used as a threshold value, the gray values of the adjacent 8 pixels are compared with the central pixel, if the values of the surrounding pixels are greater than the value of the central pixel, the position of the pixel is marked as 1, otherwise, the position is 0. Thus, 8 points in the 3 × 3 neighborhood can generate 8-bit binary numbers (usually converted into decimal numbers, i.e. LBP codes, 256 types in total) by comparison, that is, the LBP value of the pixel point in the center of the window is obtained, and the LBP value is used to reflect the texture information of the region. The embodiment adopts the principle, the LBP values of the molten pool, the original sputtering image and the repaired molten pool and the repaired sputtering image are obtained by using a local binary pattern algorithm, the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image is identified according to the numerical difference of the LBP values of the molten pool and the original sputtering image, the image of the difference part is the image of the defect area, and the position and the form of the defect area are displayed, so that the defect area is accurately positioned and identified according to the image of the defect area.
The step S3 of precisely locating and identifying the defect area according to the image of the defect area specifically includes:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library, wherein the defect type of the defect image which is closest to the defect image in the picture library is the defect type corresponding to the image of the defect area to be identified. In general, blow hole defects, spheroidization defects, inclusion defects, and non-fusion defects are the more common defects of the molten pool. Sputtering may induce porosity and spheroidization defects.
The defect example segmentation is realized through the method, and the manufacturing process parameter control system can adjust the manufacturing process parameters in a targeted manner according to the segmentation result of the defect example.
Preferably, the method also comprises the step of measuring the size of the molten pool, and the specific method is as follows:
matching the template image, establishing a measuring frame ROI, and processing and identifying picture features in the measuring frame; for a molten pool image to be measured, firstly, drawing a gray level histogram along the length direction in an ROI (region of interest) and smoothing the gray level histogram by utilizing Gaussian filtering, then judging the edge of the molten pool according to the gray level change condition on the gray level histogram and establishing a measurement edge pair, and meanwhile, calculating the size of the molten pool according to the number of pixels between adjacent edges in the edge pair. The manufacturing process parameters are adjusted by comparing the measured size of the molten pool with the preset size, and the generation of machining errors is avoided.
Further preferably, the method further comprises the step of counting the number of the sputtered particles, and the specific method comprises the following steps:
and generating a confidence density map required by the sputtering image to be counted by using a generating type antagonistic neural network, performing antagonistic generation network training, comparing the confidence density map with the sputtering images in the image library, and reading the sputtering number prestored in the corresponding image in the image library after comparison, namely the sputtering number of the sputtering image to be counted. The statistics of the sputtering number has very important significance for establishing the optimal selection of the subsequent processing energy.
Based on the same inventive concept, the embodiment of the present invention further provides a metal additive manufacturing image processing apparatus combining an anti-neural network with local binary, and since the principle of the apparatus for solving the technical problem is similar to that of the above method, repeated descriptions are omitted.
As shown in fig. 7, an apparatus for processing a metal additive manufacturing image by combining a neural network with local binary is provided in an embodiment of the present invention, and the apparatus includes:
the image preprocessing module is used for acquiring a molten pool and a sputtering image for metal additive manufacturing and preprocessing the molten pool and the sputtering image;
the generating type countermeasure neural network module is used for repairing the defects in the pretreated molten pool and the sputtering image by utilizing the trained generating type countermeasure neural network to obtain the repaired molten pool and the repaired sputtering image;
and the local binary pattern algorithm module is used for acquiring images with difference parts between the original molten pool and sputtering images and the repaired molten pool and sputtering images by using a local binary pattern algorithm, wherein the images with the difference parts are images of the defect area, and the defect area is accurately positioned and identified according to the images of the defect area.
Further, the image preprocessing module is specifically configured to:
and carrying out preprocessing operations of graying and smooth filtering on the molten pool and the sputtering image.
Further, the repairing of the defects in the pretreated molten pool and the sputtering image by the generated antagonistic neural network module using the trained generated antagonistic neural network specifically comprises:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
Further, the step of accurately positioning and identifying the defect region by the local binary pattern algorithm module according to the image of the defect region specifically includes:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library to obtain the defect type.
Preferably, the device further comprises a molten pool size measuring module, which is used for:
matching the template image, establishing a measuring frame ROI, and processing and identifying picture features in the measuring frame; for a molten pool image to be measured, firstly, drawing a gray level histogram along the length direction in an ROI (region of interest) and smoothing the gray level histogram by utilizing Gaussian filtering, then judging the edge of the molten pool according to the gray level change condition on the gray level histogram and establishing a measurement edge pair, and meanwhile, calculating the size of the molten pool according to the number of pixels between adjacent edges in the edge pair.
Preferably, the sputtering system further comprises a sputtering number statistic module, specifically configured to:
and generating a confidence density map required by the sputtering image to be counted by using a generating type antagonistic neural network, performing antagonistic generation network training, comparing the confidence density map with the sputtering images in the image library, and reading the sputtering number prestored in the corresponding image in the image library after comparison, namely the sputtering number of the sputtering image to be counted.
Each module of the antagonistic neural network of the embodiment can be flexibly modified and optimized, and end-to-end training and prediction can be realized. The local binary pattern algorithm module obtains the segmentation results of different examples through segmentation among defect classes, filters out the defect examples within a threshold value through setting a threshold value, and timely adjusts process parameters or stops continuous processing of parts, so that the processing cost is avoided or reduced. The anti-neural network is combined with local binary continuous deep learning, the model training accuracy rate reaches 99.89%, the verification accuracy rate reaches 97.3%, and the recognition speed is better under the same hardware condition.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A metal additive manufacturing image detection method combining an anti-neural network and local binary is characterized by comprising the following steps:
s1, acquiring a molten pool and a sputtering image for metal additive manufacturing, and preprocessing the molten pool and the sputtering image;
s2, repairing the defects in the pretreated molten pool and the sputtering image by using the trained generative antagonistic neural network to obtain a repaired molten pool and a repaired sputtering image;
s3, LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image are obtained through a local binary pattern algorithm, the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image is recognized according to the numerical difference of the LBP values of the molten pool and the original sputtering image, the image of the difference portion is the image of the defect area, and the defect area is accurately positioned and recognized according to the image of the defect area.
2. The method for detecting metal additive manufacturing image of the anti-neural network combined with local binary as claimed in claim 1, wherein the preprocessing of the molten pool and the sputtering image in the step S1 specifically comprises:
and carrying out preprocessing operations of graying and smooth filtering on the molten pool and the sputtering image.
3. The method for detecting metal additive manufacturing image of claim 1, wherein the repairing the defects in the preprocessed molten pool and sputtered image by using the trained generative countering neural network in step S2 specifically comprises:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
4. The method for detecting a metal additive manufacturing image of an anti-neural network combined with local binarization as claimed in claim 1, wherein: the step S3 of precisely locating and identifying the defect area according to the image of the defect area specifically includes:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library to obtain the defect type.
5. The method for detecting the metal additive manufacturing image of the anti-neural network combined with the local binary value as claimed in claim 1, wherein the method further comprises the following steps of measuring the size of a molten pool:
matching the template image, establishing a measuring frame ROI, and processing and identifying picture features in the measuring frame; for a molten pool image to be measured, firstly, drawing a gray level histogram along the length direction in an ROI (region of interest) and smoothing the gray level histogram by utilizing Gaussian filtering, then judging the edge of the molten pool according to the gray level change condition on the gray level histogram and establishing a measurement edge pair, and meanwhile, calculating the size of the molten pool according to the number of pixels between adjacent edges in the edge pair.
6. The method for detecting the metal additive manufacturing image of the anti-neural network combined with the local binary as claimed in claim 1, wherein the method further comprises the following steps of counting the number of sputtered materials:
and generating a confidence density map required by the sputtering image to be counted by using a generating type antagonistic neural network, performing antagonistic generation network training, comparing the confidence density map with the sputtering images in the image library, and reading the sputtering number prestored in the corresponding image in the image library after comparison, namely the sputtering number of the sputtering image to be counted.
7. A metal additive manufacturing image processing apparatus incorporating local binarization to an anti-neural network, comprising:
the image preprocessing module is used for acquiring a molten pool and a sputtering image for metal additive manufacturing and preprocessing the molten pool and the sputtering image;
the generating type countermeasure neural network module is used for repairing the defects in the pretreated molten pool and the sputtering image by utilizing the trained generating type countermeasure neural network to obtain the repaired molten pool and the repaired sputtering image;
and the local binary pattern algorithm module is used for acquiring LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image by using a local binary pattern algorithm, identifying the difference between the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image according to the numerical difference of the LBP values of the molten pool and the original sputtering image, wherein the image of the difference part is the image of the defect area, and accurately positioning and identifying the defect area according to the image of the defect area.
8. The apparatus of claim 7, wherein the image pre-processing module is specifically configured to:
and carrying out preprocessing operations of graying and smooth filtering on the molten pool and the sputtering image.
9. The metal additive manufacturing image processing device of claim 7, wherein the generating type antagonistic neural network module repairing the defects in the preprocessed molten pool and sputtering image by using the trained generating type antagonistic neural network specifically comprises:
the generating type countermeasure neural network firstly carries out size transformation on a molten pool and a sputtering image to be repaired, transforms the molten pool and the sputtering image to the input size of the adaptive defect feature extraction network, carries out defect feature extraction through the defect feature extraction network, outputs a convolution feature map of an original image, carries out image semantic segmentation according to the convolution feature map, and further generates a network output through a candidate region through the convolution feature map to obtain a molten pool region and a sputtering region;
and repairing the defects in the molten pool area and the sputtering area respectively by using the trained generative antagonistic neural network.
10. The apparatus as claimed in claim 7, wherein the means for processing the metal additive manufacturing image with the neural network combined with the local binary, wherein the means for performing the local binary pattern algorithm to precisely locate and identify the defect area according to the image of the defect area specifically comprises:
positioning the defect area according to the numerical difference of LBP values of the molten pool and the original sputtering image and the repaired molten pool and the repaired sputtering image to obtain the position of a target frame of the defect area; and comparing the image of the defect area with the defect image of the known defect type in the picture library to obtain the defect type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442941.6A CN111815555A (en) | 2020-05-22 | 2020-05-22 | Metal additive manufacturing image detection method and device combining anti-neural network with local binary |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442941.6A CN111815555A (en) | 2020-05-22 | 2020-05-22 | Metal additive manufacturing image detection method and device combining anti-neural network with local binary |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111815555A true CN111815555A (en) | 2020-10-23 |
Family
ID=72847726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010442941.6A Pending CN111815555A (en) | 2020-05-22 | 2020-05-22 | Metal additive manufacturing image detection method and device combining anti-neural network with local binary |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111815555A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112967267A (en) * | 2021-03-23 | 2021-06-15 | 武汉大学 | Laser directional energy deposition sputtering counting method of full convolution neural network |
| CN112991304A (en) * | 2021-03-23 | 2021-06-18 | 武汉大学 | Molten pool sputtering detection method based on laser directional energy deposition monitoring system |
| CN113066060A (en) * | 2021-03-23 | 2021-07-02 | 武汉大学 | Laser directional energy deposition image identification method of void convolution neural network |
| CN113066049A (en) * | 2021-03-10 | 2021-07-02 | 武汉大学 | MEMS sensor defect type identification method and system |
| CN113240668A (en) * | 2021-06-08 | 2021-08-10 | 南京师范大学 | Weld pool image quality evaluation method based on image digital feature distribution |
| CN113570549A (en) * | 2021-06-30 | 2021-10-29 | 青岛海尔科技有限公司 | Defect detection method and device for reflective surface |
| CN113674176A (en) * | 2021-08-23 | 2021-11-19 | 北京市商汤科技开发有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN113763320A (en) * | 2021-07-29 | 2021-12-07 | 国网江苏省电力有限公司电力科学研究院 | Cable accessory lead sealing construction defect detection method and system |
| CN117340280A (en) * | 2023-12-05 | 2024-01-05 | 成都斐正能达科技有限责任公司 | LPBF additive manufacturing process monitoring method |
-
2020
- 2020-05-22 CN CN202010442941.6A patent/CN111815555A/en active Pending
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113066049A (en) * | 2021-03-10 | 2021-07-02 | 武汉大学 | MEMS sensor defect type identification method and system |
| CN112967267B (en) * | 2021-03-23 | 2024-01-23 | 湖南珞佳智能科技有限公司 | Laser directional energy deposition sputtering counting method of full convolution neural network |
| CN112991304A (en) * | 2021-03-23 | 2021-06-18 | 武汉大学 | Molten pool sputtering detection method based on laser directional energy deposition monitoring system |
| CN113066060A (en) * | 2021-03-23 | 2021-07-02 | 武汉大学 | Laser directional energy deposition image identification method of void convolution neural network |
| CN112967267A (en) * | 2021-03-23 | 2021-06-15 | 武汉大学 | Laser directional energy deposition sputtering counting method of full convolution neural network |
| CN113066060B (en) * | 2021-03-23 | 2023-12-19 | 湖南珞佳智能科技有限公司 | Laser directional energy precipitation image identification method of cavity convolutional neural network |
| CN113240668A (en) * | 2021-06-08 | 2021-08-10 | 南京师范大学 | Weld pool image quality evaluation method based on image digital feature distribution |
| CN113240668B (en) * | 2021-06-08 | 2024-04-16 | 南京师范大学 | Image digital feature distribution-based generated molten pool image quality evaluation method |
| CN113570549A (en) * | 2021-06-30 | 2021-10-29 | 青岛海尔科技有限公司 | Defect detection method and device for reflective surface |
| CN113763320A (en) * | 2021-07-29 | 2021-12-07 | 国网江苏省电力有限公司电力科学研究院 | Cable accessory lead sealing construction defect detection method and system |
| CN113674176A (en) * | 2021-08-23 | 2021-11-19 | 北京市商汤科技开发有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN113674176B (en) * | 2021-08-23 | 2024-04-16 | 北京市商汤科技开发有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN117340280B (en) * | 2023-12-05 | 2024-02-13 | 成都斐正能达科技有限责任公司 | LPBF additive manufacturing process monitoring method |
| CN117340280A (en) * | 2023-12-05 | 2024-01-05 | 成都斐正能达科技有限责任公司 | LPBF additive manufacturing process monitoring method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111815555A (en) | Metal additive manufacturing image detection method and device combining anti-neural network with local binary | |
| CN109685760B (en) | MATLAB-based SLM powder bed powder laying image convex hull depression defect detection method | |
| CN102496161B (en) | Method for extracting contour of image of printed circuit board (PCB) | |
| CN113724231B (en) | Industrial defect detection method based on semantic segmentation and target detection fusion model | |
| CN111539927B (en) | Detection method of automobile plastic assembly fastening buckle missing detection device | |
| CN115311629B (en) | Abnormal bending precision monitoring system of bending machine | |
| CN114897908B (en) | Machine vision-based method and system for analyzing defects of selective laser powder spreading sintering surface | |
| CN116188459B (en) | Line laser rapid identification method and system for belt tearing detection | |
| CN112907561A (en) | Notebook appearance flaw detection method based on deep learning | |
| CN113177924A (en) | Industrial production line product flaw detection method | |
| CN115035092A (en) | Image-based bottle detection method, device, equipment and storage medium | |
| CN116228651A (en) | Cloth defect detection method, system, equipment and medium | |
| CN114119595A (en) | GMAW welding quality on-line monitoring and evaluating method based on integrated deep learning | |
| CN117253024B (en) | Industrial salt quality inspection control method and system based on machine vision | |
| CN113077423B (en) | Laser selective melting pool image analysis system based on convolutional neural network | |
| CN116563277B (en) | Machine vision-based copper-clad aluminum surface defect detection method and system | |
| CN117115147B (en) | Textile detection method and system based on machine vision | |
| CN116110006B (en) | Scenic spot tourist abnormal behavior identification method for intelligent tourism system | |
| CN115866502A (en) | Microphone part surface defect online detection process | |
| CN113919396A (en) | Vibration signal and image characteristic machine tool cutter wear state monitoring method based on semi-supervised learning | |
| CN110147818B (en) | Sparse representation-based laser welding forming defect prediction classification method | |
| Chang et al. | Research on PCB solder joint defect detection method based on machine vision | |
| Jiang et al. | Visual inspection system for CNC turning tool wear based on transfer learning | |
| CN117269168B (en) | New energy automobile precision part surface defect detection device and detection method | |
| CN112581472B (en) | Target surface defect detection method facing human-computer interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |