CN116977652B - Workpiece surface morphology generation method and device based on multi-mode image generation - Google Patents
Workpiece surface morphology generation method and device based on multi-mode image generation Download PDFInfo
- Publication number
- CN116977652B CN116977652B CN202311227139.5A CN202311227139A CN116977652B CN 116977652 B CN116977652 B CN 116977652B CN 202311227139 A CN202311227139 A CN 202311227139A CN 116977652 B CN116977652 B CN 116977652B
- Authority
- CN
- China
- Prior art keywords
- target
- vector
- low
- noise
- surface topography
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 239000013598 vector Substances 0.000 claims abstract description 78
- 238000009792 diffusion process Methods 0.000 claims abstract description 73
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000011156 evaluation Methods 0.000 claims abstract description 15
- 238000010586 diagram Methods 0.000 claims abstract description 10
- 238000013441 quality evaluation Methods 0.000 claims abstract description 5
- 238000012876 topography Methods 0.000 claims description 66
- 238000001228 spectrum Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 14
- 238000012512 characterization method Methods 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000003672 processing method Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000007477 logistic regression Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000004927 fusion Effects 0.000 claims description 2
- 238000003754 machining Methods 0.000 abstract description 13
- 238000013507 mapping Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 14
- 238000004088 simulation Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000005520 cutting process Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007516 diamond turning Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000003746 surface roughness Effects 0.000 description 1
- 238000007514 turning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for generating a workpiece surface morphology based on multi-mode image generation, belonging to the technical field of machining data processing, and comprising the following steps: constructing a guide vector based on multi-mode information of historical data of different processing modes; based on a diffusion model, adding noise to the low-dimensional representation of the surface appearance gray level diagram to obtain a noise vector, inputting the guide vector, the time step and the noise vector into a reverse diffusion process, reducing noise layer by layer to restore the low-dimensional representation, and realizing the training of the model; extracting target multi-mode information to construct a target guide vector, inputting a random noise hidden variable and the target guide vector into a trained diffusion model to obtain a target low-dimensional representation, and obtaining a target surface morphology gray scale map through a decoder; and adopting an image quality comprehensive evaluation module to perform quality evaluation. The invention adopts a diffusion model, realizes the accurate mapping from the multi-mode information to the surface morphology image, has the characteristics of quick generation and high fidelity, and has great potential for the real-time surface morphology prediction.
Description
Technical Field
The invention belongs to the technical field of machining data processing, and particularly relates to a workpiece surface morphology generation method and device based on multi-mode image generation.
Background
In the field of machining, a machining process is the basis, and different machining parameters influence the surface quality of a part, including the surface roughness, surface shape errors and the like of the part, and relate to the mechanical property, the optical property and the like of the surface of the part. Because different processing parameters correspond to different processing morphologies, in an actual experiment, a lot of processing parameters and corresponding morphology measurement historical data exist, and how to use the existing data as a priori provides experience knowledge for subsequent processing parameters is one of key factors for realizing intelligent manufacturing.
The existing conventional technology generally needs to use expensive detection equipment to measure the surface morphology of a machined part offline or in place, and needs to continuously carry out a processing-detecting-reprocessing process, so that the processing precision of a target can be converged by multiple iterations, a great deal of manpower and money are consumed, the processing efficiency is low, therefore, the existing surface morphology simulation method has the advantages of two types through part surface morphology simulation and surface morphology gray level image generation methods, the three-dimensional surface morphology of a workpiece is solved through an analytical model, and the three-dimensional surface morphology is generated in recent years by a learning method, but the method for solving the surface morphology image based on the surface morphology simulation is complex in modeling, large in calculation amount and long in time.
Patent document with publication number of CN112387995A discloses a surface morphology prediction method after ultraprecise turning of a free-form surface, comprising the following steps: by combining two research directions of tool path planning and surface morphology simulation, a region L needing surface morphology simulation is formed according to the tool path planned based on active control machining precision x ×L y According to resolution d x And d y Dividing into m multiplied by n grids, calculating coordinate data of all grid points in a simulation area according to the geometric position relation of each grid point and a knife contact point on a planned tool path, and then performing curved surface reconstruction by using the calculated coordinate data to realize surface morphology simulation modeling of curved surface single-point diamond turning, wherein the obtained simulation modeling is realizedThe true model can predict the machining error by removing the shape component of the curved surface. However, the method for solving the surface morphology based on analytical modeling has the problems of complex modeling, large calculated amount and long time consumption.
Patent document with publication number CN116012480a discloses a data-driven method for generating a grey-scale image of a machined surface topography, which adopts a generator and a discriminator in a generating countermeasure network model based on a neural network to directly convert machining signal spectrogram and tool rigidity data into grey-scale image data of the machined surface topography, but the precision of the generating countermeasure network model adopted by the invention is insufficient, the generated data cannot well simulate the distribution of actual data, and the generalization performance is poor.
Disclosure of Invention
The invention aims to provide a workpiece surface morphology generating method and device based on multi-mode image generation, which adopts a diffusion model to realize accurate mapping from workpiece multi-mode information to workpiece surface morphology images, has stronger capability in the utilization of historical data, can realize rapid and high-fidelity surface morphology gray level image generation on one hand, and can accurately predict current surface morphology information through real-time processing information on the other hand.
In order to achieve the above purpose, the technical scheme provided by the invention is as follows:
in a first aspect, an embodiment of the present invention provides a method for generating a workpiece surface topography based on multi-mode image generation, including the following steps:
step 1: collecting surface morphology images and processing signal spectrograms of different processing modes, marking to obtain multi-mode information, and processing the multi-mode information to obtain a guide vector;
step 2: compressing a gray level image corresponding to a surface morphology image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, adding noise to the first low-dimensional representation layer by layer through a forward diffusion process to obtain a noise vector, reducing noise to restore a second low-dimensional representation layer by layer through a reverse diffusion process based on a guide vector, a time step and the noise vector, and training the diffusion model;
step 3: extracting target multi-mode information during application to construct a target guide vector, inputting a hidden variable of Gaussian noise, a time step and the target guide vector which are randomly generated into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and obtaining a target surface morphology gray scale image through a decoder by the target low-dimensional representation;
step 4: and inputting the target surface topography gray scale map into an image quality comprehensive evaluation module for evaluating the fidelity of the target surface topography gray scale map.
The invention adopts a diffusion model, firstly collects multi-mode information of different processing modes based on the history data of the machining process, and constructs a guide vector; in the model training process, a gray level image corresponding to a surface morphology image is compressed into a first low-dimensional representation through an encoder, the first low-dimensional representation is noised layer by utilizing a forward diffusion process of a diffusion model to obtain a noise vector, then the noise vector is reduced layer by utilizing a reverse diffusion process of the diffusion model to restore the noise vector into a second low-dimensional representation, and the diffusion model is trained; in the actual application process, a target guide vector is constructed according to target multi-mode information, a two-dimensional Gaussian noise hidden variable, a time step and the target guide vector which are randomly generated are input into a trained diffusion model to obtain a target low-dimensional representation, and a decoder is adopted to convert the target low-dimensional representation into a surface morphology gray scale map; and finally, evaluating the fidelity of the generated image by adopting an image quality comprehensive evaluation module. The method provided by the invention realizes the accurate mapping from the multi-mode information to the surface morphology image, makes full use of the historical data, can realize the rapid and high-fidelity surface morphology gray scale image generation, and is also very suitable for the surface morphology image prediction in a real-time processing scene.
Further, in step 1, labeling the surface feature image and the processing signal spectrogram to obtain multi-mode information, including:
marking the surface topography image to obtain corresponding text information, wherein the text information comprises a processing method, a feeding amount, a workpiece material, a geometric shape of a cutter and vibration between the cutter and the workpiece;
labeling the processing signal spectrogram to obtain a corresponding processing frequency spectrum signal;
the multimodal information includes text information and a processed spectral signal.
Further, in step 1, the processing the multimodal information to obtain a guide vector includes:
and converting the text information and the processed spectrum signals into a representation form through a text encoder and a spectrum signal encoder respectively, and cascading the representation form to obtain an embedded feature vector serving as a guide vector, wherein the text encoder and the spectrum signal encoder adopt a contrast language-image pre-training model CLIP.
Further, the encoder and the decoder form a variable self-encoder.
Further, in step 2, the inverse diffusion process employs a uiet noise estimation network based on a cross-attention mechanism, where the uiet noise estimation network is used to generate estimated noise, and the estimated noise is used to reduce noise per time step.
Further, in step 4, the image quality comprehensive evaluation module includes a high-dimensional semantic feature extractor, a low-dimensional deformation feature extractor and a regression model;
the target surface morphology gray level map is respectively subjected to a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor to extract semantic features and deformation features, the semantic features and the deformation features are input into a regression model after feature fusion, and a quality score is predicted through logistic regression of the regression model and is used for evaluating the fidelity of the target surface morphology gray level map.
Further, in step 4, the high-dimensional semantic feature extractor includes a pre-trained afflicientnetv 2 network, and the low-dimensional deformation feature extractor includes a pre-trained VGG16 network.
In order to achieve the above object, the embodiment of the present invention further provides a workpiece surface topography generating device based on multi-mode image generation, which includes a guide vector constructing unit, a model training unit, a model application unit, and a quality evaluating unit;
the guiding vector construction unit is used for acquiring surface morphology images and processing signal spectrograms of different processing modes, marking the surface morphology images and the processing signal spectrograms to obtain multi-mode information, and processing the multi-mode information to obtain guiding vectors;
the model training unit is used for compressing a gray level diagram corresponding to a surface morphology image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, carrying out layer-by-layer noise addition on the first low-dimensional representation through a forward diffusion process to obtain a noise vector, carrying out layer-by-layer noise reduction on the noise vector through a reverse diffusion process based on a guide vector, a time step and a time step to restore a second low-dimensional representation, and training the diffusion model;
the model application unit is used for extracting target multi-mode information during application to construct a target guide vector, inputting a hidden variable of Gaussian noise generated randomly, a time step and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and obtaining a target surface topography gray level diagram through a decoder by the target low-dimensional representation;
the quality evaluation unit is used for inputting the target surface topography gray scale map into an image quality comprehensive evaluation module for evaluating the fidelity of the target surface topography gray scale map.
In order to achieve the above object, an embodiment of the present invention further provides a workpiece surface topography generating device based on multi-mode image generation, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to implement, when executing the computer program, the workpiece surface topography generating method based on multi-mode image generation provided by the embodiment of the present invention in the first aspect.
In a fourth aspect, in order to achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the storage medium, and when the computer program is used in a computer, the method for generating a workpiece surface topography based on multi-mode image generation provided in the first aspect is implemented.
The beneficial effects of the invention are as follows:
(1) The method provided by the invention realizes accurate mapping from multi-mode information to the surface morphology gray level image, and compared with the traditional detection equipment and surface morphology simulation method, the method provided by the invention greatly reduces the data acquisition time and has stronger capability in the utilization of historical data;
(2) Compared with the prior art comprising two parts of training of a generator and a discriminator, and the two networks need to be converged, the method provided by the invention greatly reduces the image generation time and simultaneously avoids the problems of mode collapse, poor generalization performance and the like of the generation countermeasure network in the training process;
(3) The method provided by the invention adopts randomly generated noise as the input of the diffusion model in practical application, and has randomness and diversity, so that the method provided by the invention can generate various surface morphology images with high quality and diversity and conforming to the real situation;
(4) The method provided by the invention can accurately predict the current surface morphology information through real-time processing information, thereby helping a processor acquire the surface morphology image in the processing process in time, being beneficial to the fast judgment of the product quality of the processed part by the worker and the timely adjustment of the processing parameters.
Drawings
FIG. 1 is a flowchart of a method for generating a workpiece surface topography based on multi-modal image generation provided by an embodiment of the invention;
FIG. 2 is a schematic flow chart of a cross-attention mechanism based back diffusion process provided by an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a practical application process based on a diffusion model according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an image quality comprehensive evaluation module according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a workpiece surface topography generating device based on multi-modal image generation according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a workpiece surface topography generating apparatus based on multi-modal image generation according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the detailed description is presented by way of example only and is not intended to limit the scope of the invention.
The invention is characterized in that: aiming at the problems that the modeling is complex, the calculated amount is large, the time consumption is long, the generated data cannot simulate the distribution of actual data well and the generalization performance is poor based on the insufficient precision of a generated countermeasure network model in the prior art, the invention provides a workpiece surface morphology generating method and device based on multi-modal image generation. Compared with the method for sampling the surface morphology by directly adopting the detection equipment and calculating the surface morphology based on the analytical modeling method, the method greatly reduces the data sampling time and the image generation time; compared with the method for generating the countermeasure network, the training process of the method is relatively stable, the situation of unstable training is not easy to occur, samples which are high in quality, diversity and accord with the actual situation can be generated, the method has the characteristics of quick generation and high fidelity, and the method has great potential for predicting the real-time surface morphology.
The diffusion model is a wide mathematical model and is widely used in the fields of probability theory, statistics and correlation. In particular, a diffusion model describes the process of evolution or diffusion of a phenomenon from an initial state to another state over a period of time. The diffusion model can be considered as a form of hidden variable model that attempts to learn the noise distribution under the data distribution, using a markov chain to gradually add noise to the data, and in the process learn the posterior probability distribution of the data, the diffusion model can provide an effective solution when dealing with complex data distributions, such as multi-modal information, of the workpiece.
Fig. 1 is a flowchart of a method for generating a workpiece surface topography based on multi-modal image generation according to an embodiment of the present invention. As shown in fig. 1, an embodiment provides a workpiece surface topography generating method based on multi-mode image generation, including the following steps:
s110, collecting surface morphology images and processing signal spectrograms of different processing modes, marking to obtain multi-mode information, and processing the multi-mode information to obtain guide vectors.
In this embodiment, taking the cutting process as an example, a three-dimensional scanner is used to collect surface data of an actual machined part in the cutting process, where the surface data includes a surface topography image and a machining signal spectrogram, and the surface topography image is projected into an image space and converted into a surface topography gray scale image, which is used as corresponding target data for supervising the generated data.
Marking text information corresponding to the surface morphology gray scale map, wherein the text information comprises a processing method, a feeding amount, a workpiece material, a geometric shape of a cutter and vibration between the cutter and the workpiece; and labeling the processing frequency spectrum signal, text information and the processing frequency spectrum signal of the processing signal frequency spectrum chart as multi-mode information. Text encoder and spectrum signal encoder based on contrast language-image pre-training model CLIP are adopted to convert text information and processed spectrum signals into representation forms and are cascaded to obtain embedded feature vectorsMapping the obtained embedded feature vector into a joint space with the surface topography gray level map, and establishing a semantic relation between the surface topography gray level map and multi-mode information, wherein the embedded feature vector is used as a guide vector for providing surface topography gray level map generation conditions, and guiding a noise estimation network of a diffusion model to carry out a reverse diffusion process on the noise vector.
The training stage mainly adopts historical processing parameters and part surface morphology data measured by a morphology detector and a three-dimensional scanner, so that the accuracy of the model is ensured, and the actual use stage predicts according to the actual processing parameters.
S120, compressing a gray level diagram corresponding to the surface morphology image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, adding noise to the first low-dimensional representation layer by layer through a forward diffusion process to obtain a noise vector, reducing noise to restore a second low-dimensional representation layer by layer through a reverse diffusion process based on a guide vector, a time step and the noise vector, and training the diffusion model.
The pre-training-based variational self-encoder comprises an encoder and a decoder, wherein the encoder is used for compressing the surface topography gray scale map in S110 into a first low-dimensional representationCharacterizing the first low-dimensional +.>And inputting a diffusion model. The diffusion model specifically comprises a forward diffusion process and a reverse diffusion process, the training stage comprises a forward diffusion process training process and a reverse diffusion optimizing process, and the trained reverse diffusion process is used in the practical application stage. During the training phase, two-dimensional Gaussian noise based on random generation +>The forward diffusion process will input the first low-dimensional representation +.>Performing layer-by-layer noise adding processing to obtain noise vector +.>. The reverse diffusion process adopts a multi-head-based diffusion process>Structural Unet noise estimation network, said multiple head +.>The structure belongs to a cross-attention mechanism, as shown in figure 2, noise vector +.>Time step T and embedding feature vector +.>Input to the multi-head based->A structural Unet noise estimation network obtains estimated noise +.>Will estimate noise->Two-dimensional Gaussian noise generated randomly>Comparing, establishing estimated noise->Two-dimensional Gaussian noise generated randomly>A loss function therebetween, the loss function being formulated as:
(1)
wherein,representation ofθIs used for the differentiation of the (c) and (d),θrepresenting network super-parameters, minimizing a loss function based on an optimization iterative algorithm, and obtaining estimated noise meeting convergence conditions>Obtaining a second low-dimensional representation through multiple rounds of denoising, wherein when the second low-dimensional representation is infinitely close to the first low-dimensional representationAnd obtaining a trained diffusion model. Said multiple head->The structure is expressed as:
(2)
wherein,representing query (query), is->Representing a key->Representation value (value), +.>，/>，，/>, />、/>And->Respectively indicate->、/>、/>The corresponding weight matrix is used for the weight matrix,a normalization function is shown to be performed,drepresentation->、/>、/>Mainly for reducing +.>And->Ensures +.>Gradient stability of the function.
S130, extracting target multi-mode information during application to construct a target guide vector, inputting a hidden variable of Gaussian noise generated randomly, a time step and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and obtaining a target surface morphology gray scale image through a decoder by the target low-dimensional representation.
In the practical application process, extracting target multi-mode information, including text information (cutting machining, feeding amount, workpiece material, geometric shape of a cutter and vibration between the cutter and the workpiece) and machining spectrum signals, respectively converting the text information and the machining spectrum signals into characterization forms through a text encoder and a spectrum signal encoder, and cascading the characterization forms to obtain an embedded characterization vector serving as a target guide vector. Converting the randomly generated two-dimensional Gaussian noise signal into a characterization vector of a latent space, specifically, performing convolution processing on the two-dimensional Gaussian noise signal to obtain a binary vector with the size ofInputting the hidden variables and the target guide vector into a trained diffusion model, and performing a reverse diffusion process to obtain processed +.>And (3) denoising the condition hidden variables of the size for multiple times through an optimization iterative algorithm to obtain a target low-dimensional representation, sending the target low-dimensional representation to a decoder part in a pre-trained variational self-encoder, and restoring the implicit target low-dimensional representation into image information by the decoder as shown in fig. 3 so as to generate a target surface topography gray level map.
And S140, inputting the target surface topography gray scale map into an image quality comprehensive evaluation module for evaluating the fidelity of the target surface topography gray scale map.
In this embodiment, the decoder of the pre-trained variational self-encoder is used to restore the target low-dimensional representation to 3 gray images with different views of the target surface topography gray image, and the generated multi-view gray image is input into an image quality comprehensive evaluation module for feature extraction and distortion evaluation, so as to evaluate whether the generated target surface topography gray image meets the requirements in terms of semantic content and fidelity.
As shown in fig. 4, the image quality comprehensive evaluation module includes a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor, wherein the high-dimensional semantic feature extractor adopts the last 4 layers of the pretrained afflicientnetv 2 network to extract high-dimensional features as high-dimensional semantic distortion features, and the high-dimensional semantic distortion features include semantic features such as content information, physical characteristics, space-time relations among the content and the like; the low-dimensional deformation feature extractor adopts the first four layers of the pretrained VGG16 network, extracts low-dimensional features as low-dimensional deformation distortion features, wherein the low-dimensional deformation distortion features comprise compression, noise, blurring, overexposure, or overdrising, chromatic aberration, sharpness and blockiness effects. And after the high-dimensional semantic distortion characteristics and the low-dimensional deformation distortion characteristics are fused through the characteristics, inputting the characteristics into an image distortion quality regression model formed by three full-connection layers, and obtaining corresponding quality scores. The image distortion quality regression model is used for evaluating the quality of a generated image, and combines subjective and objective evaluation indexes, and takes high-dimensional characteristics (physical characteristics of objects, space-time relations among the objects, content information of the objects and the like) and low-dimensional characteristics (compression, noise, blurring, overexposure or darkness, chromatic aberration, sharpness, block effect and the like) as distortion indexes. Taking the distortion processing of high-quality images obtained in a real scene as an example, the distortion index of the high-quality images is set to be 0, the distortion processing of different degrees is carried out on the high-quality images, and scoring between 0 and 1 is carried out, wherein the distortion degree is higher as the score is closer to 1, so that the quality score obtained by logistic regression of an image distortion quality regression model is lower after a series of processing is carried out on the surface topography gray scale image generated by the method, and the higher the quality is, the better the fidelity is.
Based on the same inventive concept, the embodiment of the invention also provides a workpiece surface topography generating device 500 based on multi-mode image generation, as shown in fig. 5, which comprises a guide vector constructing unit 510, a model training unit 520, a model applying unit 530 and a quality evaluating unit 540;
the guiding vector constructing unit 510 is configured to collect surface topography images and processing signal spectrograms of different processing modes, label the surface topography images and the processing signal spectrograms to obtain multi-mode information, and process the multi-mode information to obtain guiding vectors;
the model training unit 520 is configured to compress a gray level map corresponding to a surface topography image into a first low-dimensional representation through an encoder, input the first low-dimensional representation into a diffusion model, perform layer-by-layer noise addition on the first low-dimensional representation through a forward diffusion process to obtain a noise vector, perform layer-by-layer noise reduction on the noise vector through a reverse diffusion process based on a guide vector, a time step and a time step to restore a second low-dimensional representation, and train the diffusion model;
the model application unit 530 is configured to extract target multi-mode information during application to construct a target guide vector, input a hidden variable of gaussian noise generated randomly, a time step and the target guide vector into a trained diffusion model, obtain a target low-dimensional representation through a reverse diffusion process, and obtain a target surface topography gray scale map through a decoder;
the quality evaluation unit 540 is configured to input the target surface topography gray scale map into an image quality comprehensive evaluation module, so as to evaluate the fidelity of the target surface topography gray scale map.
For the workpiece surface topography generating device based on multi-mode image generation provided by the embodiment of the invention, the device basically corresponds to the method embodiment, so relevant parts refer to part of the description of the method embodiment. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Based on the same inventive concept, an embodiment further provides a workpiece surface topography generating device based on multi-modal image generation, as shown in fig. 6, including a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for implementing the workpiece surface topography generating method based on multi-modal image generation when executing the computer program.
The workpiece surface topography generating device based on multi-mode image generation provided by the embodiment of the invention can be a device such as a computer. The device embodiments can be implemented by software, or by hardware or a combination of hardware and software. Taking software implementation as an example, the method is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory through a processor of any equipment with data processing capability. From a hardware level, fig. 6 is a schematic structural diagram of a workpiece surface topography generating device based on multi-mode image generation according to an embodiment of the present invention, and besides a processor, a memory, a network interface, and a nonvolatile memory shown in fig. 6, the workpiece surface topography generating device based on multi-mode image generation according to an embodiment of the present invention generally includes other hardware according to an actual function of the device with data processing capability, which is not described herein.
Based on the same inventive concept, the embodiment also provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is used by a computer, the workpiece surface topography generation method based on the multi-mode image generation is realized.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any of the data processing enabled devices described in any of the previous embodiments. The computer readable storage medium may also be an external storage device of the wind turbine generator, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), etc. provided on the device. Further, the computer readable storage medium may include both internal storage units and external storage devices of any device having data processing capabilities. The computer readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing apparatus, and may also be used for temporarily storing data that has been output or is to be output.
It should be noted that, the workpiece surface topography generating device based on the multi-mode image generation, and the computer-readable storage medium provided in the foregoing embodiments all belong to the same concept as the workpiece surface topography generating method embodiment based on the multi-mode image generation, and detailed implementation processes of the method embodiment are shown in the workpiece surface topography generating method embodiment based on the multi-mode image generation, which is not described herein again.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the foregoing detailed description of the invention has been provided, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing examples, and that certain features may be substituted for those illustrated and described herein. Modifications, equivalents, and alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. The workpiece surface morphology generating method based on multi-mode image generation is characterized by comprising the following steps of:
step 1: collecting surface morphology images and processing signal spectrograms of different processing modes, marking to obtain multi-mode information, and processing the multi-mode information to obtain a guide vector, wherein the method comprises the following steps: marking a surface topography image to obtain text information, wherein the text information comprises a processing method, a feeding amount, a workpiece material, a geometric shape of a cutter and vibration between the cutter and the workpiece, a processing signal spectrogram is marked to obtain a processing frequency spectrum signal, the text information and the processing frequency spectrum signal are used as multi-mode information, the text information and the processing frequency spectrum signal are converted into a characterization form through a comparison language-image pre-training model CLIP and are cascaded to obtain an embedded feature vector, the embedded feature vector is mapped into a joint space with a surface topography gray scale map and used for establishing a semantic relation between the multi-mode information and the surface topography gray scale map, and the mapped embedded feature vector is used as a guide vector;
step 2: compressing a gray level image corresponding to a surface morphology image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, adding noise to the first low-dimensional representation layer by layer through a forward diffusion process to obtain a noise vector, reducing noise to restore a second low-dimensional representation layer by layer through a reverse diffusion process based on a guide vector, a time step and the noise vector, and training the diffusion model;
step 3: extracting target multi-mode information during application to construct a target guide vector, inputting a hidden variable of Gaussian noise, a time step and the target guide vector which are randomly generated into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and obtaining a target surface morphology gray scale image through a decoder by the target low-dimensional representation;
step 4: and inputting the target surface topography gray scale map into an image quality comprehensive evaluation module for evaluating the fidelity of the target surface topography gray scale map.
2. The method of generating a workpiece surface topography based on multi-modal image generation of claim 1, wherein the encoder and the decoder constitute a variational self-encoder.
3. The method for generating workpiece surface topography based on multi-modal image generation according to claim 1, wherein the back diffusion process employs a cross-attention mechanism based nnet noise estimation network for generating estimated noise for noise reduction per time step.
4. The method for generating the surface topography of the workpiece based on the multi-modal image generation according to claim 1, wherein the image quality comprehensive evaluation module comprises a high-dimensional semantic feature extractor, a low-dimensional deformation feature extractor and a regression model;
the target surface morphology gray level map is respectively subjected to a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor to extract semantic features and deformation features, the semantic features and the deformation features are input into a regression model after feature fusion, and a quality score is predicted through logistic regression of the regression model and used for evaluating the fidelity of the target surface morphology gray level map.
5. The multi-modal image generation-based workpiece surface topography generation method of claim 4, wherein the high-dimensional semantic feature extractor comprises a pre-trained EfficientNetV2 network and the low-dimensional deformation feature extractor comprises a pre-trained VGG16 network.
6. The workpiece surface morphology generating device based on multi-mode image generation is characterized by comprising a guide vector constructing unit, a model training unit, a model application unit and a quality evaluation unit;
the guiding vector construction unit is used for collecting surface morphology images and processing signal spectrograms of different processing modes, labeling the surface morphology images and the processing signal spectrograms to obtain multi-mode information, processing the multi-mode information to obtain guiding vectors, and comprises the following steps: marking a surface topography image to obtain text information, wherein the text information comprises a processing method, a feeding amount, a workpiece material, a geometric shape of a cutter and vibration between the cutter and the workpiece, a processing signal spectrogram is marked to obtain a processing frequency spectrum signal, the text information and the processing frequency spectrum signal are used as multi-mode information, the text information and the processing frequency spectrum signal are converted into a characterization form through a comparison language-image pre-training model CLIP and are cascaded to obtain an embedded feature vector, the embedded feature vector is mapped into a joint space with a surface topography gray scale map and used for establishing a semantic relation between the multi-mode information and the surface topography gray scale map, and the mapped embedded feature vector is used as a guide vector;
the model training unit is used for compressing a gray level diagram corresponding to a surface morphology image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, carrying out layer-by-layer noise addition on the first low-dimensional representation through a forward diffusion process to obtain a noise vector, carrying out layer-by-layer noise reduction on the noise vector through a reverse diffusion process based on a guide vector, a time step and a time step to restore a second low-dimensional representation, and training the diffusion model;
the model application unit is used for extracting target multi-mode information during application to construct a target guide vector, inputting a hidden variable of Gaussian noise generated randomly, a time step and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and obtaining a target surface topography gray level diagram through a decoder by the target low-dimensional representation;
the quality evaluation unit is used for inputting the target surface topography gray scale map into an image quality comprehensive evaluation module for evaluating the fidelity of the target surface topography gray scale map.
7. A multi-modal image generation-based workpiece surface topography generation device comprising a memory for storing a computer program and a processor, wherein the processor is configured to implement the multi-modal image generation-based workpiece surface topography generation method of any one of claims 1-5 when the computer program is executed.
8. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when used with a computer, implements the method for generating a workpiece surface topography based on multimodal image generation of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311227139.5A CN116977652B (en) | 2023-09-22 | 2023-09-22 | Workpiece surface morphology generation method and device based on multi-mode image generation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311227139.5A CN116977652B (en) | 2023-09-22 | 2023-09-22 | Workpiece surface morphology generation method and device based on multi-mode image generation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116977652A CN116977652A (en) | 2023-10-31 |
| CN116977652B true CN116977652B (en) | 2023-12-22 |
Family
ID=88479960
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311227139.5A Active CN116977652B (en) | 2023-09-22 | 2023-09-22 | Workpiece surface morphology generation method and device based on multi-mode image generation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977652B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102490081A (en) * | 2011-11-14 | 2012-06-13 | 华中科技大学 | Workpiece three-dimensional surface topography simulating method based on ball head milling |
| CN112387995A (en) * | 2020-10-30 | 2021-02-23 | 吉林大学 | Surface morphology prediction method after ultra-precise turning of free-form surface |
| CN113313147A (en) * | 2021-05-12 | 2021-08-27 | 北京大学 | Image matching method based on deep semantic alignment network model |
| CN114023300A (en) * | 2021-11-03 | 2022-02-08 | 四川大学 | Chinese speech synthesis method based on diffusion probability model |
| CN116012480A (en) * | 2023-01-13 | 2023-04-25 | 重庆大学 | Data-driven cutting surface morphology gray scale image generation method |
| CN116450873A (en) * | 2023-02-20 | 2023-07-18 | 阿里巴巴达摩院(杭州)科技有限公司 | Image generation and diffusion model training method, electronic device and storage medium |
| CN116503513A (en) * | 2023-03-20 | 2023-07-28 | 浙江大学 | Sketch guided image editing method based on conditional diffusion model |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102019003921B4 (en) * | 2019-06-05 | 2021-05-06 | Hufschmied Zerspanungssysteme Gmbh | Workpiece inspection method and workpiece inspection system |
| US20230118966A1 (en) * | 2022-12-16 | 2023-04-20 | Lemon Inc. | Generation of story videos corresponding to user input using generative models |
-
2023
- 2023-09-22 CN CN202311227139.5A patent/CN116977652B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102490081A (en) * | 2011-11-14 | 2012-06-13 | 华中科技大学 | Workpiece three-dimensional surface topography simulating method based on ball head milling |
| CN112387995A (en) * | 2020-10-30 | 2021-02-23 | 吉林大学 | Surface morphology prediction method after ultra-precise turning of free-form surface |
| CN113313147A (en) * | 2021-05-12 | 2021-08-27 | 北京大学 | Image matching method based on deep semantic alignment network model |
| CN114023300A (en) * | 2021-11-03 | 2022-02-08 | 四川大学 | Chinese speech synthesis method based on diffusion probability model |
| CN116012480A (en) * | 2023-01-13 | 2023-04-25 | 重庆大学 | Data-driven cutting surface morphology gray scale image generation method |
| CN116450873A (en) * | 2023-02-20 | 2023-07-18 | 阿里巴巴达摩院(杭州)科技有限公司 | Image generation and diffusion model training method, electronic device and storage medium |
| CN116503513A (en) * | 2023-03-20 | 2023-07-28 | 浙江大学 | Sketch guided image editing method based on conditional diffusion model |
Non-Patent Citations (4)
| Title |
|---|
| Deep diffusion models for seismic processing;Ricard Durall等;《Computers & Geosciences》;第177卷;第1-11页 * |
| Theoretical and experimental investigation on modeling of surface topography influenced by the tool-workpiece vibration in the cutting direction and feeding direction in single-point diamond turning;Fujing Tian等;《The International Journal of Advanced Manufacturing Technology》;第86卷;第2433-2439页 * |
| 基于刀具振动的超精密车削表面粗糙度预测技术;李世豪;《工程科技Ⅰ辑》(第03期);第13-63页 * |
| 基于扩散模型的指纹图像生成方法;杨光锴;《河北省科学院学报》;第40卷(第1期);第13-18、66页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116977652A (en) | 2023-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860982A (en) | Wind power plant short-term wind power prediction method based on VMD-FCM-GRU | |
| CN110569901A (en) | Channel selection-based countermeasure elimination weak supervision target detection method | |
| CN115080801B (en) | Cross-modal retrieval method and system based on federal learning and data binary representation | |
| CN112365511B (en) | Point cloud segmentation method based on overlapped region retrieval and alignment | |
| CN109543693A (en) | Weak labeling data noise reduction method based on regularization label propagation | |
| CN115457492A (en) | Target detection method and device, computer equipment and storage medium | |
| CN117974693B (en) | Image segmentation method, device, computer equipment and storage medium | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN116051382A (en) | Data enhancement method based on deep reinforcement learning generation type antagonistic neural network and super-resolution reconstruction | |
| CN113723294B (en) | Data processing method and device and object recognition method and device | |
| Zhang et al. | Sonar image quality evaluation using deep neural network | |
| CN116977652B (en) | Workpiece surface morphology generation method and device based on multi-mode image generation | |
| Li et al. | Towards communication-efficient digital twin via AI-powered transmission and reconstruction | |
| CN117115474A (en) | End-to-end single target tracking method based on multi-stage feature extraction | |
| CN115604131B (en) | Link flow prediction method, system, electronic device and medium | |
| CN109887012B (en) | Point cloud registration method combined with self-adaptive search point set | |
| CN116662307A (en) | Intelligent early warning method, system and equipment based on multi-source data fusion | |
| CN116363469A (en) | Method, device and system for detecting infrared target with few samples | |
| CN113723468B (en) | Object detection method of three-dimensional point cloud | |
| CN112446345B (en) | Low-quality three-dimensional face recognition method, system, equipment and storage medium | |
| CN111914751B (en) | Image crowd density identification detection method and system | |
| Cao et al. | Understanding 3D point cloud deep neural networks by visualization techniques | |
| CN112991402A (en) | Cultural relic point cloud registration method and system based on improved differential evolution algorithm | |
| CN118470577B (en) | Inspection scene identification method and system based on big data | |
| Mohr et al. | GAFAR: Graph-Attention Feature-Augmentation for Registration A Fast and Light-weight Point Set Registration Algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |