CN116342996A - Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion - Google Patents
Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion Download PDFInfo
- Publication number
- CN116342996A CN116342996A CN202310407868.2A CN202310407868A CN116342996A CN 116342996 A CN116342996 A CN 116342996A CN 202310407868 A CN202310407868 A CN 202310407868A CN 116342996 A CN116342996 A CN 116342996A
- Authority
- CN
- China
- Prior art keywords
- image
- composite coating
- laser
- fusion
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 230000004927 fusion Effects 0.000 title claims abstract description 74
- 238000000576 coating method Methods 0.000 title claims abstract description 66
- 239000011248 coating agent Substances 0.000 title claims abstract description 60
- 239000002131 composite material Substances 0.000 title claims abstract description 46
- 238000002347 injection Methods 0.000 title claims abstract description 28
- 239000007924 injection Substances 0.000 title claims abstract description 28
- 230000008018 melting Effects 0.000 claims abstract description 36
- 230000008569 process Effects 0.000 claims abstract description 34
- 238000002844 melting Methods 0.000 claims abstract description 33
- 239000000919 ceramic Substances 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 26
- 239000002184 metal Substances 0.000 claims abstract description 16
- 239000011156 metal matrix composite Substances 0.000 claims abstract description 16
- 238000005266 casting Methods 0.000 claims abstract description 7
- 238000010309 melting process Methods 0.000 claims abstract description 3
- 238000002156 mixing Methods 0.000 claims abstract description 3
- 239000002245 particle Substances 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 12
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 239000008188 pellet Substances 0.000 claims description 4
- 239000006002 Pepper Substances 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000005336 cracking Methods 0.000 claims description 3
- 238000004090 dissolution Methods 0.000 claims description 3
- 238000007500 overflow downdraw method Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 230000003746 surface roughness Effects 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 2
- 238000011049 filling Methods 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 abstract description 4
- 230000006872 improvement Effects 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000000463 material Substances 0.000 description 6
- 239000000843 powder Substances 0.000 description 6
- 239000000758 substrate Substances 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 239000000243 solution Substances 0.000 description 4
- 229910000831 Steel Inorganic materials 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000010959 steel Substances 0.000 description 3
- XKRFYHLGVUSROY-UHFFFAOYSA-N Argon Chemical compound [Ar] XKRFYHLGVUSROY-UHFFFAOYSA-N 0.000 description 2
- 239000000956 alloy Substances 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000011365 complex material Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000004308 accommodation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000007605 air drying Methods 0.000 description 1
- 229910045601 alloy Inorganic materials 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 229910052786 argon Inorganic materials 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910010293 ceramic material Inorganic materials 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000005238 degreasing Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000001802 infusion Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004372 laser cladding Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005488 sandblasting Methods 0.000 description 1
- 238000007711 solidification Methods 0.000 description 1
- 230000008023 solidification Effects 0.000 description 1
- 238000005979 thermal decomposition reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P10/00—Technologies related to metal processing
- Y02P10/25—Process efficiency
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Tourism & Hospitality (AREA)
- Development Economics (AREA)
- Artificial Intelligence (AREA)
- General Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Software Systems (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Game Theory and Decision Science (AREA)
- Educational Administration (AREA)
- Manufacturing & Machinery (AREA)
- Primary Health Care (AREA)
- Laser Beam Processing (AREA)
Abstract
The invention relates to a quality prediction method of a laser melt-injection composite coating based on multistage mixing fusion. The prediction method comprises the steps of firstly establishing a laser melting and injecting process map of the ceramic reinforced metal matrix composite coating, simultaneously collecting paired image data of an infrared camera image and a high-speed camera image of the composite coating under a plurality of groups of laser melting and injecting process parameters, grouping the corresponding paired image data according to the process map, then constructing a laser melting and injecting multistage hybrid fusion frame with a data layer and a characteristic layer, finally training a laser melting and injecting quality prediction model based on the paired image data and the multistage hybrid fusion frame, and predicting the state of the ceramic reinforced metal matrix composite coating in the laser melting and casting process in real time by utilizing the trained quality prediction model. The prediction method can integrate and fuse the complementary molten pool and splash characteristics to realize the quality prediction of the laser-injected ceramic reinforced metal-based composite coating, and remarkably improve the prediction precision of the model.
Description
Technical Field
The invention relates to the technical field of material manufacturing monitoring and prediction, in particular to a quality prediction method of a laser melt-injection composite coating based on multistage hybrid fusion and a system applying the quality prediction method.
Background
Since conventional subtractive manufacturing is limited to processing monolithic alloy products, achieving an improvement in the functional performance of localized areas of the product while maintaining plasticity and relatively low cost is a difficult task. However, with the development of high energy beam technology, laser melt injection provides a new approach to the manufacture of multi-material products with locally tailored physical properties. As the demand for complex materials having special functions increases, ceramic materials corresponding to the demand are selected and injected into the surface of the metal part, and ceramic reinforced metal matrix composite coatings can be formed. In the field of aerospace, space exploration, energy development, biological medicine industry and other industries with huge demands, the local ceramic reinforced metal matrix composite coating becomes particularly important. However, these industries are slow in accepting and applying laser-infused ceramic-reinforced metal matrix composite coatings, and current process reliability, stability and repeatability are major obstacles for further application of laser-infused composite coatings.
From a material perspective, the preparation of the laser-infused composite coating requires controlling the temperature of the molten pool to avoid thermal decomposition of the ceramic particles and ensure melting of the metal matrix, and the ceramic-reinforced metal-based composite coating with a more complex material system has greater uncertainty than the high-energy beam processing of a single alloy material. From a processing perspective, the change of processing boundary conditions, uneven powder feeding and inconsistent laser processing equipment caused by the structural design of the part can influence the state of ceramic particles and the solidification process of a molten pool. The disadvantages of part quality and part performance caused by these uncertainties add considerable additional cost and material wastage in post-process inspection. Therefore, the real-time monitoring and predictive evaluation of the state of the laser melt injection manufacturing quality is of great significance in promoting the large-scale application of the laser melt injection composite coating.
With the improvement of computer information processing technology and the rising of intelligent manufacturing technology, the application of multi-source sensor and deep learning in combination to the field of high-energy beam processing becomes a new research hotspot. The combination of deep learning and multi-source monitoring data not only can improve the robustness of the monitoring system, but also can capture complementary information from different modalities. Deep-learning multi-layer networks provide the flexibility to achieve multi-source monitoring data fusion, but still lack a unified data fusion architecture and what fusion architecture is currently chosen, often based on intuition, which makes choosing the best fusion architecture more challenging on a particular problem. At present, an effective solution to the problem of quality defect monitoring of a laser-infused ceramic reinforced metal-based composite coating is lacking, and how to effectively utilize multi-source sensor data and select a corresponding fusion structure to highlight complementarity and robustness of a fusion framework is a key problem of monitoring multi-mode fusion in a laser infusion process.
Disclosure of Invention
Based on the above, the invention provides a quality prediction method of a laser-injected composite coating based on multistage hybrid fusion, which is necessary to solve the technical problem that the quality defect monitoring problem of the laser-injected ceramic reinforced metal-based composite coating in the prior art lacks an effective solution.
The invention discloses a quality prediction method of a laser melt-injection composite coating based on multistage mixing fusion, which is used for predicting the quality of a ceramic reinforced metal matrix composite coating in real time. The prediction method comprises the following steps:
s1, establishing a laser melting and injecting process map of the ceramic reinforced metal matrix composite coating.
S2, simultaneously acquiring paired image data of an infrared camera image and a high-speed camera image of the composite coating under a plurality of groups of laser melting and injecting process parameters, and grouping the corresponding paired image data according to a process map.
S3, constructing a laser melt injection multi-stage hybrid fusion frame with a data layer and a feature layer combined, wherein the construction method comprises the following steps:
s31, respectively extracting a molten pool characteristic region in the infrared camera image and a splash characteristic region in the high-speed camera image by a characteristic extraction processing method.
S32, fusing the characteristic region image of the molten pool and the corresponding splash characteristic region image to obtain a fused image.
S33, carrying out data enhancement on the molten pool characteristic region image, the splash characteristic region image and the fusion image, respectively taking the three types of images as input, and carrying out model training and recognition through a plurality of advanced characteristic extractors, thereby determining a preferred characteristic extractor corresponding to the input of the three types of images.
S34, simultaneously taking the characteristic region image of the molten pool, the characteristic region image of splashing and the fusion image as inputs to realize fusion of the data layers, respectively passing through corresponding preferential characteristic extractors, and carrying out fusion of the characteristic layers before entering the pooling layer and the full-connection layer, thereby obtaining the laser melt injection multistage hybrid fusion frame.
S4, training a laser melting and casting quality prediction model based on the paired image data and the multi-stage hybrid fusion frame, and predicting the state of the ceramic reinforced metal matrix composite coating in the laser melting and casting process in real time by using the trained quality prediction model.
As a further improvement of the above scheme, in S1, the method for establishing the laser melt injection process map includes the following steps:
the forming quality state of the laser melt-injected composite coating is divided into four types of particle adhesion, pellet and crack, normal melting and excessive decomposition of particles by analyzing the dissolution state, mesoscopic surface roughness and macroscopic cracking condition of ceramic particles in the coating.
As a further improvement of the above solution, in S31, the method for extracting the molten pool features from the infrared camera image includes the steps of:
(1) And extracting the region of interest by taking the coordinate corresponding to the highest temperature in the infrared camera image as the center.
(2) Non-local mean filtering is performed on the infrared camera image of the extracted region of interest to eliminate isolated noise caused by splatter.
(3) The extracted molten pool area image is amplified by a bilinear difference algorithm.
(4) Correcting the emissivity by using a temperature gradient method, repairing the temperature image, and further obtaining a characteristic region of the molten pool.
As a further improvement of the above-described aspect, in S31, the method of performing splash feature extraction on the high-speed camera image includes the steps of:
(1) And taking the arc centroid in the high-speed camera image as a center to extract the region of interest.
(2) And carrying out median filtering denoising on the high-speed camera image of the extracted region of interest.
(3) The gray scale image is converted into a binarized image by an Otsu global threshold method.
(4) And extracting an arc light region and a laser head reflection region by an area search method, and extracting the rest part in the image as a splash characteristic region.
As a further improvement of the above scheme, in S32, the fusion process is performed by a pixel weighted fusion method.
As a further improvement of the above-described scheme, in S33, the data enhancement performs the same operation on the image data acquired at the same time, including the steps of:
(1) Half of the images were randomly selected to add 3% salt-and-pepper noise.
(2) Half of the images were randomly selected for horizontal flipping.
(3) Each picture was randomly rotated ±20 degrees, after which the rotated picture was cut and filled to the original size.
(4) The images were sequentially added with a width random offset of 20% and a height random offset of 20%.
As a further improvement of the above-described scheme, in S33, the types of advanced feature extractors include: resNet18, VGG16, inceptionV3 and MobileNet V3.
As a further improvement of the above solution, in S33, the determining the preferred feature extractor by the average prediction time and the model prediction efficiency specifically includes the following steps:
(1) And eliminating the feature extractor with the average prediction time higher than the acquisition frequency of the camera.
(2) Calculating model prediction efficiency eta of each feature extractor, wherein a calculation formula is as follows:
where m represents the total number of exercises. t represents the picture average processing time. n is n i The total sample size for the ith training. n is n TPi And n TFi The sample sizes of true positives and false positives in the ith training, respectively.
(3) And respectively selecting the feature extractor with highest model prediction efficiency as the corresponding preferred feature extractor.
As a further improvement of the above-described scheme, in S2, the acquisition frequency adjustment of the infrared camera and the high-speed camera is also kept consistent before the acquisition of the infrared camera image and the high-speed camera image.
The invention also discloses a quality prediction system of the laser melt-injection composite coating based on multistage hybrid fusion, which applies the quality prediction method. The quality prediction system comprises: and the image acquisition module and the data processing module.
The image acquisition module comprises an infrared camera and a high-speed camera which are respectively used for acquiring infrared images and high-speed images of the composite coating under a plurality of groups of laser melting and injecting process parameters and forming paired image data.
The data processing module is used for establishing a laser melting injection process map of the ceramic reinforced metal matrix composite coating, grouping corresponding paired image data according to the process map, training a laser melting injection quality prediction model by utilizing the paired image data and the constructed laser melting injection multistage hybrid fusion frame, and predicting the state of the ceramic reinforced metal matrix composite coating in the laser melting casting process in real time by utilizing the trained quality prediction model.
Compared with the prior art, the technical scheme disclosed by the invention has the following beneficial effects:
the invention provides a laser-injected ceramic reinforced metal-based composite coating quality prediction method based on an infrared camera image and a high-speed camera image. The method comprises the steps of establishing a multi-stage hybrid fusion framework of data-level fusion and feature-level fusion, wherein three types of input data of the proposed fusion framework are complementary and redundant, the complementary data can enable a feature extraction network to consider global information between a molten pool and splashing, and the redundant data can enhance the credibility of extracted features and predicted results.
In addition, the proposed multi-stage hybrid fusion framework does not require a specific feature extraction network, hybrid fusion allows the use of different feature extraction networks for each channel, and a more suitable feature extraction network can be selected as input data and device conditions change, thereby achieving greater flexibility.
Drawings
FIG. 1 is a schematic view of a laser melting and injecting experiment platform according to a preferred embodiment of the present invention;
FIG. 2 is a flow chart of a method for predicting the quality of a laser-infused composite coating based on multistage hybrid fusion in accordance with the present invention;
FIG. 3 is a typical macro-micro state diagram of four types of laser-injected WC-reinforced metal-based composite coatings in an embodiment of the invention;
FIG. 4 is a schematic diagram of a laser-infused WC-reinforced 316L metal-based composite coating process in an embodiment of the invention;
FIG. 5 is a schematic diagram of a process for molten pool feature extraction of an infrared camera image in an embodiment of the invention;
FIG. 6 is a schematic diagram of a process for splash feature extraction of high-speed camera images in an embodiment of the present invention;
FIG. 7 is a schematic diagram of a process for fusing a high speed camera image with an infrared camera image;
fig. 8 is a schematic diagram of a laser-infused multi-stage hybrid fusion framework in an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It is noted that when an element is referred to as being "mounted to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "secured to" another element, it can be directly secured to the other element or intervening elements may also be present.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "or/and" as used herein includes any and all combinations of one or more of the associated listed items.
The embodiment provides a quality prediction method of a laser melt-injection composite coating based on multistage hybrid fusion, which is used for predicting the quality of a ceramic reinforced metal matrix composite coating in real time.
In the embodiment, WC particles are laser injected on the surface of a 316L steel substrate, and quality prediction is carried out on the formed WC particle reinforced 316L metal-based composite coating. Referring to fig. 1, the hardware platform used in the present embodiment includes a laser generator 1, an argon gas cylinder 2, a powder feeder 3, an environmental chamber 4, a laser beam 5, a laser head 6, a processing platform 7, a substrate 8, a ceramic reinforced metal matrix composite coating 9, and a quality prediction system applying the quality prediction method, where the prediction system includes an image processing module and a data processing module. The image processing module may include an infrared camera 11 and a high-speed camera 10, and the data processing module may include a data collector 12 and a computer 13 installed with prediction software.
Referring to fig. 2, the prediction method includes the following steps S1 to S4.
S1, establishing a laser melting and injecting process map of the ceramic reinforced metal matrix composite coating.
S2, simultaneously acquiring paired image data of an infrared camera image and a high-speed camera image of the composite coating under a plurality of groups of laser melting and injecting process parameters, and grouping the corresponding paired image data according to a process map.
In the embodiment, 506R type large-scale environmental box laser cladding equipment produced by the Condui company is selected, the surface of a 316L steel substrate is subjected to sand blasting, polishing, degreasing, cleaning and cold air drying, casting WC powder with the granularity of 50-150 meshes is filled into a Condui T2 double-cylinder automatic powder feeder 3, and a ceramic insulating plate and the 316L steel substrate are sequentially placed on a workbench.
And carrying out laser melting and injecting experiments of the WC reinforced metal-based composite coating under a plurality of groups of different laser process parameters, wherein the selected laser process parameters are shown in table 1. The forming quality and quality states of the laser melt-injected composite coating are divided into four types of particle adhesion, pellet and crack, normal melting and excessive decomposition of particles by analyzing the dissolution state, mesoscopic surface roughness and macroscopic cracking condition of ceramic particles in the composite coating, and the typical macroscopic and microscopic states of the coating are shown in figure 3. Specifically characterized by (1) particle adhesion: the substrate cannot form a continuous molten pool due to the lack of energy input in the laser melting and injection process, and only a small amount of particles adhere to the surface of the substrate; (2) pellets and cracks: when the powder feeding amount of the reinforced particles exceeds the accommodation limit of a molten pool formed by the base material, the particles are clustered, the surface of the coating is rough, and a plurality of transverse penetrating cracks are accompanied; (3) normal melting: the laser energy input is matched with the powder feeding amount, the surface of the material is smooth and has no cracks, and the internal reinforced particles are uniformly distributed; (4) excessive decomposition of particles: under the condition of excessive energy input, the mechanical properties of the laser melt injection coating are adversely affected by the decomposition of ceramic particles. A laser-infused WC-reinforced 316L metal-based composite coating process map was established according to the process experiments and analysis described above, as shown in fig. 4.
TABLE 1 laser melt injection process parameters
Corresponding infrared camera images and high-speed camera images are acquired in the laser melting and injecting process, wherein the types and parameters of the selected infrared camera and high-speed camera are shown in table 2. Since the exposure frequency of the high-speed camera is high, the same acquisition frequency (0.02 s) is maintained as with the IR camera using downsampling. Each set of process parameters collects data pairs of 1000 sets of high-speed cameras and infrared cameras, 70% of images collected from the experiments are used for training the classification model, and the rest are used for verification, namely a training part data set and a verification part data set respectively comprise 8400 sets of data pairs and 3600 sets of data pairs.
Table 2 camera model and acquisition parameters
S3, constructing a laser melt injection multi-stage hybrid fusion frame with a data layer and a feature layer combined, wherein the construction method comprises the following steps:
s31, respectively extracting a molten pool characteristic region in the infrared camera image and a splash characteristic region in the high-speed camera image by a characteristic extraction processing method.
As shown in fig. 5, the method for extracting the molten pool characteristics from the infrared camera image comprises the following steps:
(1) A region of interest (ROI) (80×40 pixels) is extracted centering on coordinates corresponding to the highest temperature in the infrared camera image.
(2) Non-local mean filtering is performed on the infrared camera image of the extracted region of interest to eliminate isolated noise caused by splatter.
(3) The extracted molten pool area image is magnified 10 times by a bilinear difference algorithm.
(4) Correcting the emissivity by using a temperature gradient method, repairing the temperature image, and further obtaining a characteristic region of the molten pool.
As shown in fig. 6, the method for splash feature extraction for a high-speed camera image includes the steps of:
(1) The region of interest (650 x 400 pixels) is extracted centered on the arc centroid in the high speed camera image.
(2) The high-speed camera image of the extracted region of interest is median filtered (3 x 3) to remove noise.
(3) The gray scale image is converted into a binarized image by an Otsu global threshold method.
(4) And extracting an arc light region and a laser head reflection region by an area search method, and extracting the rest part in the image as a splash characteristic region.
S32, fusing the characteristic region image of the molten pool and the corresponding splash characteristic region image to obtain a fused image. The preprocessed high-speed camera image and the preprocessed infrared camera image are normalized and are adjusted in size, a fusion image is formed through a pixel weighting fusion method, and the fusion process is shown in fig. 7.
S33, carrying out data enhancement on the molten pool characteristic region image, the splash characteristic region image and the fusion image, respectively taking the three types of images as input, and carrying out model training and recognition through a plurality of advanced characteristic extractors, thereby determining a preferred characteristic extractor corresponding to the input of the three types of images.
The method comprises the steps of carrying out data enhancement on a molten pool characteristic region image, a splash characteristic region image and a fusion image, wherein the data enhancement step comprises the following steps: (1) Randomly selecting half of the images, and adding 3% of salt-pepper noise; (2) randomly selecting half of the images to horizontally overturn; (3) Randomly rotating the picture by +/-20 degrees, and then cutting and filling the rotated picture into an original size; (4) a random width offset of 20%; (5) random height offset 20%.
In this embodiment, three types of images are respectively used as input, model training and recognition are respectively performed through four feature extractors of ResNet18, VGG16, inceptionV3 and MobileNetV3, and through 10 cross validation training experiments, it is proved that the average processing time of the images of the four feature extractors is smaller than the image acquisition frequency (0.02 s), and further the optimal feature extractor corresponding to the three types of image input is determined according to the model prediction efficiency, and the result is shown in table 3, so that the optimized feature extractors of the molten pool feature area image, the splash feature area image and the fusion image are respectively InceptionV3, mobileNetV3 and InceptionV3.
Table 3 model efficiency based on different sensor images
In this embodiment, the preferred feature extractor is determined by the average prediction time and the model prediction efficiency, and specifically includes the following steps:
(1) And eliminating the feature extractor with the average prediction time higher than the acquisition frequency of the camera.
(2) Calculating model prediction efficiency eta of each feature extractor, wherein a calculation formula is as follows:
where m represents the total number of exercises. t represents the picture average processing time. n is n i The total sample size for the ith training. n is n TPi And n TFi The sample sizes of true positives and false positives in the ith training, respectively.
(3) And respectively selecting the feature extractor with highest model prediction efficiency as the corresponding preferred feature extractor.
S34, simultaneously taking a molten pool characteristic region image, a splash characteristic region image and a fusion image as inputs to realize fusion of a data layer, respectively passing through corresponding preferential characteristic extractors InceptionV3, mobileNet V3 and InceptionV3, and fusing characteristic layers before entering a pooling layer and a full-connection layer to determine a laser fusion multi-stage hybrid fusion frame combining the data layer and the characteristic layers, as shown in FIG. 8.
S4, training a laser melting and injecting quality prediction model based on the collected data and the determined multistage hybrid fusion frame, and predicting the WC reinforced 316L metal-based composite coating quality state of the metal-based composite material in the laser melting and injecting process in real time through the quality prediction model.
In order to embody the superiority of the proposed multi-stage hybrid fusion framework, the performance of the two types of single-sensor training models, the data layer fusion training model (based on fusion images), the feature layer fusion training model and the multi-stage hybrid fusion training model on a real-time data set is shown in table 4, wherein the data layer fusion training model, the feature layer fusion training model and the multi-stage hybrid fusion select a preferred feature extractor. The observation shows that under the index of four accuracy rates, the accuracy rate of the multi-level hybrid fusion model is improved compared with that of the feature layer fusion and the data layer fusion.
TABLE 4 accuracy index for different fusion models
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent embodiments of the invention, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of the invention should be assessed as that of the appended claims.
Claims (10)
1. The quality prediction method of the laser melt-injection composite coating based on multistage mixing fusion is characterized by being used for predicting the quality of the ceramic reinforced metal matrix composite coating in real time; the prediction method comprises the following steps:
s1, establishing a laser melting and injecting process map of the ceramic reinforced metal matrix composite coating;
s2, simultaneously acquiring paired image data of an infrared camera image and a high-speed camera image of the composite coating under a plurality of groups of laser melting and injecting process parameters, and grouping the corresponding paired image data according to the process map;
s3, constructing a laser melt injection multi-stage hybrid fusion frame with a data layer and a feature layer combined, wherein the construction method comprises the following steps:
s31, respectively extracting a molten pool characteristic region in the infrared camera image and a splash characteristic region in the high-speed camera image by a characteristic extraction processing method;
s32, fusing the characteristic region image of the molten pool and the corresponding splash characteristic region image to obtain a fused image;
s33, carrying out data enhancement on the molten pool characteristic region image, the splash characteristic region image and the fusion image, taking the three types of images as input respectively, and carrying out model training and recognition through a plurality of advanced characteristic extractors to further determine a preferred characteristic extractor corresponding to the input of the three types of images;
s34, taking the characteristic region image of the molten pool, the characteristic region image of splashing and the fusion image as inputs to realize fusion of data layers, and respectively carrying out fusion of characteristic layers before entering a pooling layer and a full-connection layer through corresponding preferential characteristic extractors so as to obtain the laser melt injection multistage hybrid fusion frame;
s4, training a laser melting and injecting quality prediction model based on the paired image data and the multistage hybrid fusion frame, and predicting the state of the ceramic reinforced metal matrix composite coating in the laser melting and casting process in real time by using the trained quality prediction model.
2. The quality prediction method of the laser melt injection composite coating based on the multistage hybrid fusion according to claim 1, wherein in S1, the establishment method of the laser melt injection process map comprises the following steps:
the forming quality state of the laser melt-injected composite coating is divided into four types of particle adhesion, pellet and crack, normal melting and excessive decomposition of particles by analyzing the dissolution state, mesoscopic surface roughness and macroscopic cracking condition of ceramic particles in the coating.
3. The quality prediction method of the laser melt-injected composite coating based on the multistage hybrid fusion according to claim 1, wherein in S31, the method for extracting molten pool characteristics from the infrared camera image comprises the following steps:
(1) Extracting a region of interest by taking a coordinate corresponding to the highest temperature in the infrared camera image as a center;
(2) Non-local mean filtering is carried out on the infrared camera image of the extracted region of interest so as to eliminate isolated noise caused by splashing;
(3) Amplifying the extracted molten pool area image by a bilinear difference algorithm;
(4) Correcting the emissivity by using a temperature gradient method, repairing the temperature image, and further obtaining the characteristic region of the molten pool.
4. The quality prediction method of a laser-infused composite coating based on multistage hybrid fusion according to claim 1, wherein in S31, the method of performing splash feature extraction on the high-speed camera image comprises the following steps:
(1) Extracting a region of interest with an arc centroid in the high-speed camera image as a center;
(2) Carrying out median filtering denoising on the high-speed camera image of the extracted region of interest;
(3) Converting the gray image into a binarized image by an Otsu global threshold method;
(4) And extracting an arc light region and a laser head reflection region by an area search method, and extracting the rest part in the image as the splash characteristic region.
5. The quality prediction method of the laser melt-injected composite coating based on multistage hybrid fusion according to claim 1, wherein in S32, the fusion treatment is performed by a pixel weighted fusion method.
6. The method for predicting the quality of a laser-infused composite coating based on multistage hybrid fusion according to claim 1, wherein in S33, the data enhancement performs the same operation on the image data acquired at the same time, comprising the steps of:
(1) Randomly selecting half of the images and adding 3% of salt-pepper noise;
(2) Randomly selecting half of the images to horizontally overturn;
(3) Randomly rotating each picture by +/-20 degrees, and then cutting and filling the rotated picture into an original size;
(4) The images were sequentially added with a width random offset of 20% and a height random offset of 20%.
7. The method for predicting quality of a laser-infused composite coating based on multistage hybrid fusion of claim 1, wherein in S33, the categories of advanced feature extractors comprise: resNet18, VGG16, inceptionV3 and MobileNet V3.
8. The method for predicting the quality of a laser-infused composite coating based on multistage hybrid fusion according to claim 1, wherein in S33, the preferred feature extractor is determined by an average prediction time and a model prediction efficiency, and specifically comprises the following steps:
(1) Eliminating a feature extractor with average prediction time higher than the acquisition frequency of the camera;
(2) Calculating model prediction efficiency eta of each feature extractor, wherein a calculation formula is as follows:
wherein m represents the total training times; t represents average processing time of the picture; n is n i Total sample size for the ith training; n is n TPi And n TFi Sample sizes of true positive and false positive in the ith training respectively;
(3) And respectively selecting the feature extractor with highest model prediction efficiency as the corresponding preferred feature extractor.
9. The method for predicting quality of laser-infused composite coating based on multistage hybrid fusion of claim 1, wherein in S2, the acquisition frequency adjustment of the infrared camera and the high-speed camera is also kept consistent before the acquisition of the infrared camera image and the high-speed camera image.
10. A quality prediction system of a laser-infused composite coating based on multistage hybrid fusion, characterized in that it applies the quality prediction method according to any one of claims 1 to 9; the quality prediction system includes:
the image acquisition module comprises an infrared camera and a high-speed camera, and is used for acquiring infrared images and high-speed images of the composite coating under a plurality of groups of laser melting and injecting process parameters respectively and forming paired image data;
the data processing module is used for establishing a laser melting injection process map of the ceramic reinforced metal-based composite coating, grouping corresponding paired image data according to the process map, training a laser melting injection quality prediction model by utilizing the paired image data and the constructed laser melting injection multistage hybrid fusion frame, and predicting the state of the ceramic reinforced metal-based composite coating in the laser melting casting process in real time by utilizing the trained quality prediction model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310407868.2A CN116342996A (en) | 2023-04-17 | 2023-04-17 | Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310407868.2A CN116342996A (en) | 2023-04-17 | 2023-04-17 | Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116342996A true CN116342996A (en) | 2023-06-27 |
Family
ID=86882399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310407868.2A Pending CN116342996A (en) | 2023-04-17 | 2023-04-17 | Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116342996A (en) |
-
2023
- 2023-04-17 CN CN202310407868.2A patent/CN116342996A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ren et al. | Thermal field prediction for laser scanning paths in laser aided additive manufacturing by physics-based machine learning | |
| Chen et al. | Defect inspection technologies for additive manufacturing | |
| Bartlett et al. | Prediction of microstructural defects in additive manufacturing from powder bed quality using digital image correlation | |
| Cho et al. | Real-time anomaly detection using convolutional neural network in wire arc additive manufacturing: molybdenum material | |
| Cai et al. | A review of in-situ monitoring and process control system in metal-based laser additive manufacturing | |
| Xiao et al. | Detection of powder bed defects in selective laser sintering using convolutional neural network | |
| Khanzadeh et al. | A methodology for predicting porosity from thermal imaging of melt pools in additive manufacturing thin wall sections | |
| Riaz et al. | An inspection approach for casting defects detection using image segmentation | |
| Nalajam et al. | A hybrid deep learning model for layer-wise melt pool temperature forecasting in wire-arc additive manufacturing process | |
| Chen et al. | Multisensor fusion-based digital twin for localized quality prediction in robotic laser-directed energy deposition | |
| Yuan et al. | A method for melt pool state monitoring in laser-based direct energy deposition based on DenseNet | |
| Wang et al. | Deep learning characterization of surface defects in the selective laser melting process | |
| Chen et al. | Multisensor fusion-based digital twin in additive manufacturing for in-situ quality monitoring and defect correction | |
| Zhu et al. | Surface morphology inspection for directed energy deposition using small dataset with transfer learning | |
| Zhang et al. | Research of surface oxidation defects in copper alloy wire arc additive manufacturing based on time-frequency analysis and deep learning method | |
| Shin et al. | Material-adaptive anomaly detection using property-concatenated transfer learning in wire arc additive manufacturing | |
| CN116342996A (en) | Quality prediction method of laser melt-injection composite coating based on multistage hybrid fusion | |
| Ye et al. | A review of the parameter-signature-quality correlations through in situ sensing in laser metal additive manufacturing | |
| CN116070484A (en) | Method for predicting component state on line in real time based on deep learning | |
| Behnke et al. | Comparison of early stopping neural network and random forest for in-situ quality prediction in laser based additive manufacturing | |
| Chen et al. | Multimodal sensor fusion for real-time location-dependent defect detection in laser-directed energy deposition | |
| Xu et al. | In situ monitoring in laser melt injection based on fusion of infrared thermal and high-speed camera images | |
| CN112967267B (en) | Laser directional energy deposition sputtering counting method of full convolution neural network | |
| Guo et al. | Innovative product design based on customer requirement weight calculation model | |
| CN115609110A (en) | Electric arc composite additive melting depth prediction method based on multimode fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |