CN117952983A

CN117952983A - Intelligent manufacturing production process monitoring method and system based on artificial intelligence

Info

Publication number: CN117952983A
Application number: CN202410354357.3A
Authority: CN
Inventors: 余楷; 严梦琪; 申林; 夏道勋; 武晓; 梁正华; 潘文杰; 冯夫健; 余正涛; 秦舒浩; 郭红建; 邹蕾; 赵林畅; 黄于欣; 肖书芹; 谢真强; 董厚泽; 代杨; 苑建坤; 孙丽娟
Original assignee: CETC Big Data Research Institute Co Ltd
Current assignee: CETC Big Data Research Institute Co Ltd
Priority date: 2024-03-27
Filing date: 2024-03-27
Publication date: 2024-04-30

Abstract

The application discloses an intelligent manufacturing production process monitoring method and system based on artificial intelligence, which are used for improving the efficiency of a product manufacturing production process. The application comprises the following steps: acquiring a real-time image of product processing and a monitoring model of a deep learning production process; inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability; when the product defect probability indicates that the target product has defects, acquiring a real-time image of a cutter, an initial image of the cutter and a defect image of the cutter, which correspond to the used cutter; inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set; inputting the cutter defect feature set serving as a defect enhancement tag and a cutter real-time image into a first generator for defect feature fusion to generate a reconstructed image; inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability; and generating a cutter state result according to the cutter defect distribution probability.

Description

Intelligent manufacturing production process monitoring method and system based on artificial intelligence

Technical Field

The embodiment of the application relates to the field of intelligent manufacturing, in particular to an intelligent manufacturing production process monitoring method and system based on artificial intelligence.

Background

With the continuous development of the mechanical field, more and more automation technologies are applied to production and manufacturing links of mechanical equipment, such as welding among mechanical components, transporting the mechanical components, grabbing the mechanical components and the like, and after the automation technologies are used in the links, the requirements on precision are met, and the efficiency is far higher than that of manpower.

In particular, in the field of manufacturing machine parts, such machine parts are generally required to be machined from a single piece of pretreated raw material, and various types of tools such as cutters and friction stir bars are used for machining, and the present application is mainly directed to manufacturing machine parts machined using the cutters. Machining of raw materials by means of numerical control tools generally requires setting of various machining parameters, such as position parameters of the tool, time node parameters, change instructions, position parameters of the raw materials, rotation parameters and pressure parameters, etc. The intelligent automatic production and manufacturing enable the accuracy and efficiency of the numerical control cutter to process raw materials far exceeding those of manual processing.

However, as the degree of automation of the numerical control machine tool is continuously increased, it becomes necessary to monitor the cutting tool and the raw material processing step. If the monitoring of the cutter and the processing link of the raw materials is not designed, the condition that the cutter is damaged and not perceived in time is likely to occur, and if the processing is continued, a large number of unqualified products can be produced, and the cutter is further damaged when serious, so that other parts affecting the numerical control machine tool are damaged. The existing method for monitoring the numerical control machine tool in the manufacturing production process has the advantages that manual real-time inspection is carried out mainly on finished products or semi-finished products, whether defects exist on the cutter or not is judged by observing the states of the finished products and the semi-finished products, the method is effective for producing a small amount of products, fatigue is easily caused once a large amount of products are required to be produced, labor cost is increased, and meanwhile, the defects generated on the precise products cannot be accurately judged by manual observation on the fine damage defects of the cutter. In order to ensure the accuracy of the monitoring, it is necessary for the inspector to increase the inspection time.

In order to solve the above problems, people start to monitor the production and manufacturing process by using artificial intelligence, and perform defect analysis on the appearance of the product through a deep learning model so as to judge whether the cutter has defects. But this approach is very suitable for manufacturing processes for single tool machining. However, in order to improve the manufacturing efficiency, a plurality of numerical control cutters often process raw materials, and the processing areas of different numerical control cutters are overlapped, and after the processing is completed by the plurality of numerical control cutters, image acquisition can not be performed, which one of the cutters with defects is not determined, and the type of the defects belongs to the defects of the blocks, the single cracks, the multiple cracks, the abrasion or the like, and the simultaneous existence of the defects of the plurality of numerical control cutters is possible. Therefore, only the numerical control cutter can be independently checked in a manual detection mode, the replacement efficiency is reduced, and the efficiency of the product manufacturing and production process is further reduced.

Disclosure of Invention

The application discloses an intelligent manufacturing production process monitoring method and system based on artificial intelligence, which are used for improving the efficiency of a product manufacturing production process.

The first aspect of the application discloses an intelligent manufacturing production process monitoring method based on artificial intelligence, which comprises the following steps:

Acquiring a product processing real-time image and a deep learning production process monitoring model, wherein the product processing real-time image is a shooting image of a target product processed by at least two cutters, and the deep learning production process monitoring model comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model;

Inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability;

When the product defect probability indicates that the target product has defects, acquiring a real-time image of a cutter, an initial image of the cutter and a defect image of the cutter, which correspond to the used cutter;

inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set;

inputting the cutter defect feature set serving as a defect enhancement tag and a cutter real-time image into a first generator for defect feature fusion to generate a reconstructed image;

Inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability;

and generating a cutter state result according to the cutter defect distribution probability.

Optionally, the deep learning production process monitoring model further comprises a second generator;

after the product processing real-time image and the deep learning production process monitoring model are obtained, the product processing real-time image is input into a product defect detection convolution model, and before the product defect probability is generated, the intelligent manufacturing production process monitoring method further comprises the following steps:

obtaining defect characteristics of a product;

and inputting the product defect characteristics serving as defect enhancement tags and the product processing real-time images into a second generator for defect characteristic fusion.

Optionally, the deep learning production process monitoring model further comprises a second defect feature extraction convolution model;

Obtaining product defect characteristics, including:

acquiring a product defect image corresponding to a target product when a used cutter has defects;

acquiring a raw material defect image of a target product;

Inputting the product defect image into a second defect feature extraction convolution model to generate a first product defect feature, wherein the product defect feature corresponds to the cutter defect;

inputting the raw material defect image into a second defect feature extraction convolution model to generate a second product defect feature;

And superposing the first product defect characteristic and the second product defect characteristic in a characteristic channel to generate the product defect characteristic.

Optionally, inputting the product defect feature as a defect enhancement tag and the product processing real-time image into a second generator for defect feature fusion, including:

dividing the real-time image for product processing into areas, and generating processing areas corresponding to all cutters;

And inputting the product defect characteristics serving as defect enhancement labels and real-time images of product processing into a second generator, and carrying out defect characteristic fusion according to the processing area.

Optionally, the product defect probability is a product defect distribution probability, and the product defect distribution probability comprises probabilities of at least two defects;

After inputting the tool defect image into the first defect feature extraction convolution model to generate the tool defect feature set, the intelligent manufacturing production process monitoring method further comprises the following steps:

and screening the cutter defect characteristics in the cutter defect characteristic set according to the product defect distribution probability.

Optionally, screening the cutter defect feature in the cutter defect feature set according to the product defect distribution probability, including:

determining the product defect type larger than a preset threshold according to the product defect distribution probability;

reversely deducing the defect distribution probability of the cutter according to the defect types of the product larger than a preset threshold value;

And screening the cutter defect characteristics in the cutter defect characteristic set according to the cutter defect distribution probability.

Optionally, inputting the tool defect feature set as a defect enhancement tag and the tool real-time image into a first generator for performing defect feature fusion, and generating a reconstructed image, including:

carrying out 1*1 convolution processing on the first cutter defect characteristic to generate a defect convolution characteristic, and carrying out channel superposition processing on the cutter defect characteristic and the defect convolution characteristic;

performing 1*1 convolution operation on the real-time image of the cutter to generate real-time convolution characteristics of the cutter;

performing regional pixel attention generation processing and channel multiplication processing on the defect convolution characteristic to generate a first processing characteristic;

performing 1*1 convolution processing on the first processing characteristic to generate convolution data;

Performing channel superposition on the convolution data and the first processing feature to generate a second processing feature;

vector calculation is carried out on the characteristic channels of the second processing characteristic, a channel vector set is generated, and one-dimensional channel vectors are output according to the channel vector set;

Correspondingly multiplying the second processing features according to the one-dimensional channel vector and generating third processing features;

Residual extraction and residual fusion processing are carried out on the third processing characteristics, and fusion residual is generated;

Carrying out channel superposition on the fusion residual error, the real-time convolution characteristic of the cutter and the defect convolution characteristic to generate a fourth processing characteristic;

Performing edge reconstruction on the fourth processing feature to generate a fifth processing feature;

Distributing attention to each neuron corresponding to the fifth processing feature, and screening out neurons with the attention smaller than a first preset threshold value to generate a sixth processing feature;

Performing edge reconstruction on the sixth processing feature to generate an enhancement parameter;

And restoring and outputting the enhancement parameters to generate a reconstructed image.

The second aspect of the application discloses an intelligent manufacturing production process monitoring system based on artificial intelligence, which comprises the following components:

The system comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a product processing real-time image and a deep learning production process monitoring model, the product processing real-time image is a shooting image of a target product processed by at least two cutters, and the deep learning production process monitoring model comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model;

the first generation unit is used for inputting the real-time image of product processing into the product defect detection convolution model to generate product defect probability;

The second acquisition unit is used for acquiring a cutter real-time image, a cutter initial image and a cutter defect image corresponding to the used cutter when the product defect probability indicates that the target product has defects;

The second generating unit is used for inputting the cutter defect image into the first defect feature extraction convolution model to generate a cutter defect feature set;

The third generation unit is used for inputting the cutter defect feature set serving as a defect enhancement tag and a cutter real-time image into the first generator to perform defect feature fusion, and generating a reconstructed image;

The fourth generation unit is used for inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability;

and a fifth generating unit for generating a tool state result according to the tool defect distribution probability.

after the first acquisition unit, before the first generation unit, the intelligent manufacturing production process monitoring method further includes:

a third obtaining unit, configured to obtain a product defect feature;

and the feature fusion unit is used for inputting the product defect feature serving as a defect enhancement tag and the product processing real-time image into the second generator for defect feature fusion.

A third acquisition unit including:

acquiring a raw material defect image of a target product;

Optionally, the feature fusion unit includes:

the intelligent manufacturing process monitoring system further comprises, after the second generating unit and before the fifth generating unit:

and the screening unit is used for screening the cutter defect characteristics in the cutter defect characteristic set according to the product defect distribution probability.

Optionally, the screening unit comprises:

Optionally, the third generating unit includes:

A third aspect of the present application provides an electronic device, comprising:

A processor, a memory, an input-output unit, and a bus;

The processor is connected with the memory, the input/output unit and the bus;

The memory holds a program that the processor invokes to perform any of the optional intelligent manufacturing process monitoring methods as in the first aspect and the first aspect.

A fourth aspect of the application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs the optional intelligent manufacturing process monitoring method of the first aspect as well as of the first aspect.

From the above technical solutions, the embodiment of the present application has the following advantages:

According to the application, the acquisition of a product processing real-time image and a deep learning production process monitoring model are realized, wherein the product processing real-time image is a shooting image of a target product processed by at least two cutters, and the deep learning production process monitoring model comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model. And inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability. The product defect detection convolution model is a deep learning image recognition model and is mainly used for judging whether product defects exist at corresponding positions on an image or not.

And when the product defect probability indicates that the target product has defects, acquiring a real-time image, an initial image and a defect image of the cutter corresponding to the used cutter. Then inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set, wherein the defect feature is extracted mainly for obtaining a reference defect, and the detection effect of a subsequent deep learning identification model is enhanced. And inputting the cutter defect characteristic set serving as a defect enhancement tag and a cutter real-time image into a first generator for defect characteristic fusion to generate a reconstructed image. The method aims to enhance the defects of the image, when the corresponding defects exist in the real-time image of the cutter, the detection probability of the depth information identification model can be greatly increased, when the corresponding defects do not exist in the real-time image of the cutter, the detection probability can be only slightly increased, and the step aims to conduct bipolar differentiation on the defect image, so that the identification effect can be improved, and the image without the corresponding features can be better screened out. And inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability. And finally, generating a cutter state result according to the cutter defect distribution probability.

And judging the defects of the product through a deep learning image recognition model, so as to deduce whether the numerical control cutter has defects or not. And then extracting the defect characteristics of the reference image, merging the defect characteristics into a real-time image of the cutter, and analyzing and judging through another depth information image recognition model to obtain a defect cutter and the types of defects, so that the defect positions and the types of the cutter do not need to be checked manually, the overhaul speed is increased, and the efficiency in the product manufacturing and production process is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of one embodiment of an artificial intelligence based intelligent manufacturing process monitoring method of the present application;

FIG. 2 is a schematic diagram of one embodiment of a first stage of the intelligent manufacturing process monitoring method based on artificial intelligence of the present application;

FIG. 3 is a schematic diagram of one embodiment of a second stage of the intelligent manufacturing process monitoring method based on artificial intelligence of the present application;

FIG. 4 is a schematic diagram of one embodiment of a third stage of the intelligent manufacturing process monitoring method based on artificial intelligence of the present application;

FIG. 5 is a schematic diagram of another embodiment of a fourth stage of the intelligent manufacturing process monitoring method based on artificial intelligence of the present application;

FIG. 6 is a schematic diagram of one embodiment of an artificial intelligence based intelligent manufacturing process monitoring system of the present application;

FIG. 7 is a schematic diagram of another embodiment of an artificial intelligence based intelligent manufacturing process monitoring method system of the present application;

FIG. 8 is a schematic diagram of an embodiment of an electronic device of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the prior art, the Micro-LED appearance defects refer to some bad appearance characteristics or defects possibly occurring in the manufacturing process of the Micro-LED display screen, such as lamp bead missing, offset, angle rotation, crystal breakage, crystal standing, scratches, damage, dirt and the like, and the defects may seriously affect the display quality. In the prior art, to detect and correct these appearance defects, automated inspection systems are often used in combination with machine vision, deep learning and other techniques to detect defects in time during the production process and reject bad products.

At present, with the continuous improvement of the automation degree of the numerical control machine tool, the monitoring of the cutter and the raw material processing link becomes necessary. If the monitoring of the cutter and the processing link of the raw materials is not designed, the condition that the cutter is damaged and not perceived in time is likely to occur, and if the processing is continued, a large number of unqualified products can be produced, and the cutter is further damaged when serious, so that other parts affecting the numerical control machine tool are damaged. The existing method for monitoring the numerical control machine tool in the manufacturing production process has the advantages that manual real-time inspection is carried out mainly on finished products or semi-finished products, whether defects exist on the cutter or not is judged by observing the states of the finished products and the semi-finished products, the method is effective for producing a small amount of products, fatigue is easily caused once a large amount of products are required to be produced, labor cost is increased, and meanwhile, the defects generated on the precise products cannot be accurately judged by manual observation on the fine damage defects of the cutter. In order to ensure the accuracy of the monitoring, it is necessary for the inspector to increase the inspection time.

Based on the above, the application discloses an intelligent manufacturing production process monitoring method and system based on artificial intelligence, which are used for improving the efficiency of the manufacturing production process of products.

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The method of the present application may be applied to a server, a device, a terminal, or other devices having logic processing capabilities, and the present application is not limited thereto. For convenience of description, the following description will take an execution body as an example of a terminal.

Referring to fig. 1, the present application provides an embodiment of an artificial intelligence based intelligent manufacturing process monitoring method, comprising:

101. Acquiring a product processing real-time image and a deep learning production process monitoring model, wherein the product processing real-time image is a shooting image of a target product processed by at least two cutters, and the deep learning production process monitoring model comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model;

In this embodiment, the product processing real-time image is obtained, where the product processing real-time image refers to an image taken after a raw material is cut by at least two numerical control cutters on a numerical control machine tool, where the two numerical control cutters cut the raw material into a target product, and the target product is not necessarily a finished product, but may also be a semi-finished product.

The cutting area of the numerical control tool can be an overlapping area or a non-overlapping area.

The deep learning production process monitoring model is composed of a plurality of deep learning modules and specifically comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model.

The product defect detection convolutional model is a convolutional neural network for analyzing images, the image recognition neural network can be used for training the shot images of the surfaces generated after cutting raw materials, and the normal images and the images with defects are trained simultaneously, so that the corresponding product defect detection convolutional model can be obtained, and the image recognition technology is the prior art. It should be noted that the product defect detection convolution model is a distributed neural network model using the output product defect type.

The first defect feature extraction convolution model is a structure comprising a plurality of convolution kernels, and is used for performing convolution feature extraction on a defect image or a defect region corresponding to the defect image.

The first generator is a deep learning module that fuses the image with the defect features.

The tool defect detection convolution model belongs to an image recognition module like the product defect detection convolution model, but the tool defect detection convolution model mainly uses two images to carry out similarity comparison and determine the difference degree.

102. Inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability;

In this embodiment, the terminal inputs the product processing real-time image into the product defect detection convolution model to generate product defect probability, and specifically only needs to determine the probability of defect features in the product processing real-time image, if the probability is greater than a preset reference value, the defect is determined to exist, if the probability is not greater than the preset reference value, the defect is determined to exist, the process is directly ended, and the target product is enabled to enter the next production and manufacturing link.

103. When the product defect probability indicates that the target product has defects, acquiring a real-time image of a cutter, an initial image of the cutter and a defect image of the cutter, which correspond to the used cutter;

When the product defect probability indicates that the target product has defects, the terminal acquires a real-time image of the cutter, an initial image of the cutter and a defect image of the cutter, namely, the cutter subjected to cutting is comprehensively checked, and particularly, the real-time image of the cutter is shot to obtain the real-time image of the cutter, wherein the damage of the numerical control cutter comprises natural abrasion, gaps, cracks and the like, and if the defect degree is overlarge, the detection is not needed by the method, and the judgment can be carried out only by collecting operation data such as pressure data of the cutter. For example, if there is a large gap in the nc tool, the raw material may not be touched at a predetermined position, or only a part of the raw material in the original planned depth may be touched, and a large difference may occur in the operation data generated by the nc tool.

In this embodiment, the fine defects in the nc tool are aimed at, and the requirements of the nc tool used for the precise mechanical component are also higher, so that the encrypting operation may cause the fine defects to occur after the nc tool is used for a long time, so that the fine product defects are generated on the surface of the mechanical component. Such fine tool defects are difficult to analyze on the operational data, but their impact on the product is enormous.

It should be noted that, if there is a processing sequence relationship between the numerical control tools, the processing area of one numerical control tool will be completely covered by another numerical control tool, that is, the corresponding processing area is trimmed, which will make some numerical control tools unable to collect the corresponding processing area, that is, miss inspection. In this embodiment, when a real-time image of product processing is acquired, unnecessary measures, such as controlling a machine tool to stop, are required to be taken, and the detection of the numerical control tool is performed before the processing area is covered. Or the adjustment of the processing flow is performed in advance, so that one cutter can be incorporated into the detection link.

104. Inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set;

the terminal inputs the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set, and operations such as convolution processing are carried out through a convolution kernel to produce corresponding defects. In this embodiment, the terminal needs to divide the area of the defect image of the tool, determine the action part of the tool, and extract the defect feature, so that the calculated amount is reduced. The tool defect image is a photographed image of a used tool when there are different defects, and such an image has been collected in advance.

105. Inputting the cutter defect feature set serving as a defect enhancement tag and a cutter real-time image into a first generator for defect feature fusion to generate a reconstructed image;

the terminal inputs the cutter defect feature set as a defect enhancement tag and a cutter real-time image into a first generator for defect feature fusion to generate a reconstructed image, wherein the reconstructed image is used for enhancing possible defects in the cutter real-time image, and specifically, the terminal can perform feature fusion on the cutter real-time image and different defect features to form a plurality of reconstructed images.

106. Inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability;

the terminal performs contrast analysis on the reconstructed images and an initial image of the cutter, wherein the initial image of the cutter is an image photographed before the cutter is used, and the operation is to ensure the homology of contrast. After the probabilities of a plurality of comparisons are obtained, the probabilities are converted into the probability of the distribution of the defects of the cutter.

107. And generating a cutter state result according to the cutter defect distribution probability.

And finally, the terminal generates a cutter state result according to the cutter defect distribution probability, namely, in the cutter defect distribution probability, determining whether cutter defects higher than a preset reference value exist, if so, indicating that the corresponding cutter has defects, and specifically, further analyzing what defects can be performed.

In this embodiment, obtaining a product processing real-time image and a deep learning production process monitoring model is achieved, where the product processing real-time image is a captured image of a target product processed by at least two tools, and the deep learning production process monitoring model includes a product defect detection convolution model, a first defect feature extraction convolution model, a first generator, and a tool defect detection convolution model. And inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability. The product defect detection convolution model is a deep learning image recognition model and is mainly used for judging whether product defects exist at corresponding positions on an image or not.

Referring to fig. 2,3,4 and 5, the present application provides an embodiment of an artificial intelligence based intelligent manufacturing process monitoring method, comprising:

201. Acquiring a product processing real-time image and a deep learning production process monitoring model, wherein the product processing real-time image is a shooting image of a target product processed by at least two cutters, and the deep learning production process monitoring model comprises a product defect detection convolution model, a first defect feature extraction convolution model, a first generator and a cutter defect detection convolution model;

Step 201 in this embodiment is similar to step 101 in the previous embodiment, and will not be repeated here.

202. Acquiring a product defect image corresponding to a target product when a used cutter has defects;

203. Acquiring a raw material defect image of a target product;

204. Inputting the product defect image into a second defect feature extraction convolution model to generate a first product defect feature, wherein the product defect feature corresponds to the cutter defect;

205. inputting the raw material defect image into a second defect feature extraction convolution model to generate a second product defect feature;

206. superposing the first product defect characteristic and the second product defect characteristic in a characteristic channel to generate a product defect characteristic;

In this embodiment, in order to distinguish whether the defect is a raw material defect problem or a cutter defect problem, not only a product defect image corresponding to a target product when a used cutter has a defect is acquired by a terminal, but also a raw material defect image of the target product is acquired, and the reason for the defect may be a raw material defect or a raw material defect plus a cutter defect. This possibility is easily present, and when other metal impurities exist on the raw material or the welding precision of the pretreatment is insufficient, the defects of the raw material are caused, and when the cutter is cut, the cutter touches the impurities which are too hard, and cracks or gaps are most likely to occur. When a defective tool is used to treat a defective welding region or impurity region, the defect of the raw material of the defective tool coincides with the defect of the tool. This can cause errors in the detection of defects.

The terminal inputs the product defect image into a second defect feature extraction convolution model to generate a first product defect feature, the product defect feature corresponds to the cutter defect, then the terminal inputs the raw material defect image into the second defect feature extraction convolution model to generate a second product defect feature, and finally the first product defect feature and the second product defect feature are subjected to feature channel superposition to generate a product defect feature, wherein the product defect feature can detect the cutter defect and the raw material defect and can detect fusion of the cutter defect and the raw material defect. In this embodiment, multiple product defect signatures may be generated based on different raw material defects and different tool defect machines.

207. Dividing the real-time image for product processing into areas, and generating processing areas corresponding to all cutters;

208. Inputting the defect characteristics of the product serving as a defect enhancement tag and a product processing real-time image into a second generator, and carrying out defect characteristic fusion according to a processing area;

The terminal divides the area of the product processing real-time image to generate processing areas corresponding to all the cutters, takes the product defect characteristics as defect enhancement labels and the product processing real-time image to be input into a second generator, and performs defect characteristic fusion according to the processing areas. After the processing area division is used, the defect characteristics can be fused on the same product processing real-time image, in the subsequent detection process, only the price area fused with the defects is detected, and compared with the original image, the calculation area is fewer, and the image needing to be calculated is fewer.

209. Inputting the product processing real-time image into a product defect detection convolution model to generate product defect probability;

210. When the product defect probability indicates that the target product has defects, acquiring a real-time image of a cutter, an initial image of the cutter and a defect image of the cutter, which correspond to the used cutter;

211. inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set;

steps 209 to 211 in this embodiment are similar to steps 102 to 104 in the previous embodiment, and are not repeated here.

212. Determining the product defect type larger than a preset threshold according to the product defect distribution probability;

213. Reversely deducing the defect distribution probability of the cutter according to the defect types of the product larger than a preset threshold value;

214. screening the cutter defect characteristics in the cutter defect characteristic set according to the cutter defect distribution probability;

In this embodiment, the product defect probability is a product defect distribution probability, which enables determination of the type of product defect, and since defects generated by different tools for raw materials are different, the distribution probability of tool defects can be reversely deduced according to the product defect distribution probability. For example: according to the defect of the product processing area A (cutter A) is a dent, table lookup or calculation is carried out according to the dent and the defect reason of the raw material, so that the cutter defect distribution probability is reversely deduced, the probability of abrasion of the cutter A is determined to be 30 percent, the probability of notch is determined to be 40 percent, the probability of crack is determined to be 20 percent, and the probability of other defects is determined to be 10 percent. At this time, these three defects can be performed. And finally, the terminal screens out the cutter defect characteristics in the cutter defect characteristic set according to the cutter defect distribution probability.

215. Carrying out 1*1 convolution processing on the first cutter defect characteristic to generate a defect convolution characteristic, and carrying out channel superposition processing on the cutter defect characteristic and the defect convolution characteristic;

216. Performing 1*1 convolution operation on the real-time image of the cutter to generate real-time convolution characteristics of the cutter;

217. Performing regional pixel attention generation processing and channel multiplication processing on the defect convolution characteristic to generate a first processing characteristic;

218. performing 1*1 convolution processing on the first processing characteristic to generate convolution data;

219. performing channel superposition on the convolution data and the first processing feature to generate a second processing feature;

220. Vector calculation is carried out on the characteristic channels of the second processing characteristic, a channel vector set is generated, and one-dimensional channel vectors are output according to the channel vector set;

221. correspondingly multiplying the second processing features according to the one-dimensional channel vector and generating third processing features;

222. residual extraction and residual fusion processing are carried out on the third processing characteristics, and fusion residual is generated;

223. carrying out channel superposition on the fusion residual error, the real-time convolution characteristic of the cutter and the defect convolution characteristic to generate a fourth processing characteristic;

224. performing edge reconstruction on the fourth processing feature to generate a fifth processing feature;

225. Distributing attention to each neuron corresponding to the fifth processing feature, and screening out neurons with the attention smaller than a first preset threshold value to generate a sixth processing feature;

226. Performing edge reconstruction on the sixth processing feature to generate an enhancement parameter;

227. Restoring and outputting the enhancement parameters to generate a reconstructed image;

In this embodiment, the terminal first performs 1*1 convolution processing on the first tool defect feature, generates a defect convolution feature, and performs channel superposition processing on the tool defect feature and the defect convolution feature. Performing 1*1 convolution operation on the real-time image of the cutter to generate real-time convolution characteristics of the cutter.

The terminal performs a region pixel attention generation process and a channel multiplication process on the defect convolution feature to generate a first processing feature, specifically, the terminal may perform a region pixel attention generation process and a channel multiplication process on the defect convolution feature by using a region pixel attention module RPA, where the region pixel attention module RPA in this step includes a BatchNorm-DefConv-ReLU, a BatchNorm-DefConv, a SigMoid function module, and a bilinear interpolation module. BatchNorm-DefConv-ReLU, batchNorm-DefConv, sigMoid function modules and bilinear interpolation modules are sequentially connected in series. The BatchNorm-DefConv-ReLU layer and the BatchNorm-DefConv layer belong to common characteristic processing layers in convolutional neural networks, the SigMoid function is a known function, and the bilinear interpolation operation method is a known algorithm. The regional pixel attention module RPA serves as a first attention mechanism, and because each block of regional pixels of the first sampling feature is assigned a weight, the neural network is more focused on regions with obvious first sampling features.

And the terminal carries out 1*1 convolution processing on the first processing characteristic to generate convolution data. And performing channel superposition on the convolution data and the first processing feature to generate a second processing feature.

In this embodiment, the terminal may perform vector calculation on the feature channel of the second processing feature through the channel Attention module Attention, generate a channel vector set, and output a one-dimensional channel vector according to the channel vector set. And correspondingly multiplying the second processing features according to the one-dimensional channel vector to generate third processing features. Specifically, the channel Attention module Attention includes a global average pooling layer, a1 x 1Conv-ReLU and a Conv-Sigmoid, and the operation principle of the channel Attention module is described in detail below. The method comprises the steps of generating a channel vector set through a global average pooling layer (Global Pooling) of a first channel Attention module Attention, carrying out channel compression through a 1X 1 convolution kernel and a ReLU activation function, outputting a one-dimensional channel vector with the dimension equal to the number of input characteristic channels through the 1X 1 convolution kernel and the Sigmoid activation function, namely the Attention weight of each characteristic channel, and multiplying each channel of the input characteristic.

And then, the terminal correspondingly multiplies the second processing characteristic by the channel according to the one-dimensional channel vector to generate a third processing characteristic.

The terminal performs residual extraction and residual fusion processing on the third processing feature to generate a fusion residual, specifically, the terminal performs residual extraction on the third processing feature to generate a first residual, then performs residual extraction on the first residual to generate a second residual, then performs residual extraction on the second residual to generate a third residual, and finally fuses the three residues according to a preset superposition coefficient to generate a final fusion residual, so that the change of an original image can be reduced.

And the terminal performs channel superposition on the fusion residual error, the cutter real-time convolution characteristic and the defect convolution characteristic to generate a fourth processing characteristic. And carrying out edge reconstruction on the fourth processing feature to generate a fifth processing feature. And distributing attention to each neuron corresponding to the fifth processing characteristic, and screening out neurons with the attention smaller than a first preset threshold value to generate the sixth processing characteristic. And carrying out edge reconstruction on the sixth processing characteristic to generate enhancement parameters. And restoring and outputting the enhanced parameters (specifically restoring and outputting the enhanced parameters through an output module Conv_out) to generate a reconstructed image.

228. Inputting the reconstructed image and the cutter initial image into a cutter defect detection convolution model to generate cutter defect distribution probability;

229. and generating a cutter state result according to the cutter defect distribution probability.

Steps 228 to 229 in this embodiment are similar to steps 106 to 107 in the previous embodiment, and will not be repeated here.

Referring to FIG. 6, the present application provides an embodiment of an artificial intelligence based intelligent manufacturing process monitoring system, comprising:

A first obtaining unit 601, configured to obtain a product processing real-time image and a deep learning production process monitoring model, where the product processing real-time image is a captured image of a target product processed by at least two tools, and the deep learning production process monitoring model includes a product defect detection convolution model, a first defect feature extraction convolution model, a first generator, and a tool defect detection convolution model;

The first generating unit 602 is configured to input a real-time image of product processing into a product defect detection convolution model, and generate a product defect probability;

a second obtaining unit 603, configured to obtain a real-time image of a tool, an initial image of the tool, and a defect image of the tool corresponding to the used tool when the product defect probability indicates that the target product has a defect;

a second generating unit 604, configured to input the tool defect image into the first defect feature extraction convolution model, and generate a tool defect feature set;

A third generating unit 605, configured to input the tool defect feature set as a defect enhancement tag and a tool real-time image into the first generator for performing defect feature fusion, and generate a reconstructed image;

A fourth generating unit 606, configured to input the reconstructed image and the tool initial image into a tool defect detection convolution model, and generate a tool defect distribution probability;

A fifth generating unit 607 for generating a tool state result according to the tool defect distribution probability.

Referring to FIG. 7, the present application provides an embodiment of an artificial intelligence based intelligent manufacturing process monitoring system, comprising:

A first obtaining unit 701, configured to obtain a product processing real-time image and a deep learning production process monitoring model, where the product processing real-time image is a captured image of a target product processed by at least two tools, and the deep learning production process monitoring model includes a product defect detection convolution model, a first defect feature extraction convolution model, a first generator, and a tool defect detection convolution model;

A third acquiring unit 702, configured to acquire a product defect feature;

The third acquisition unit 702 includes:

acquiring a raw material defect image of a target product;

A feature fusion unit 703, configured to input the product defect feature as a defect enhancement tag and a product processing real-time image into the second generator for performing defect feature fusion;

Optionally, the feature fusion unit 703 includes:

A first generating unit 704, configured to input a product processing real-time image into a product defect detection convolution model, and generate a product defect probability;

A second acquiring unit 705 configured to acquire a real-time image of a tool, an initial image of the tool, and a defect image of the tool corresponding to the used tool when the product defect probability indicates that the target product has a defect;

A second generating unit 706, configured to input the tool defect image into the first defect feature extraction convolution model, and generate a tool defect feature set;

A screening unit 707, configured to screen out the tool defect feature in the tool defect feature set according to the product defect distribution probability;

Optionally, the screening unit 707 includes:

A third generating unit 708, configured to input the tool defect feature set as a defect enhancement tag and a tool real-time image into the first generator for performing defect feature fusion, and generate a reconstructed image;

Optionally, the third generating unit 708 includes:

A fourth generating unit 709 for inputting the reconstructed image and the tool initial image into a tool defect detection convolution model to generate a tool defect distribution probability;

And a fifth generating unit 710, configured to generate a tool state result according to the tool defect distribution probability.

Referring to fig. 8, the present application provides an electronic device, including:

a processor 801, a memory 803, an input output unit 802, and a bus 804.

The processor 801 is connected to a memory 803, an input/output unit 802, and a bus 804.

The memory 803 holds a program that the processor 801 invokes to execute the intelligent manufacturing process monitoring method as in fig. 1,2 and 3, 4 and 5.

The present application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs the intelligent manufacturing process monitoring method as in fig. 1,2 and 3, and 4 and 5.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM, random access memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims

1. An intelligent manufacturing production process monitoring method based on artificial intelligence is characterized by comprising the following steps:

inputting the product processing real-time image into the product defect detection convolution model to generate product defect probability;

inputting the cutter defect feature set serving as a defect enhancement tag and the cutter real-time image into the first generator for defect feature fusion to generate a reconstructed image;

Inputting the reconstructed image and the cutter initial image into the cutter defect detection convolution model to generate cutter defect distribution probability;

2. The intelligent manufacturing process monitoring method of claim 1, wherein the deep learning process monitoring model further comprises a second generator;

After acquiring the product processing real-time image and the deep learning production process monitoring model, inputting the product processing real-time image into the product defect detection convolution model, and before generating the product defect probability, the intelligent manufacturing production process monitoring method further comprises the following steps:

obtaining defect characteristics of a product;

And inputting the product defect characteristics serving as defect enhancement tags and the product processing real-time images into the second generator for defect characteristic fusion.

3. The intelligent manufacturing process monitoring method of claim 2, wherein the deep learning process monitoring model further comprises a second defect feature extraction convolution model;

The obtaining product defect characteristics includes:

acquiring a product defect image corresponding to the target product when the used cutter has defects;

Acquiring a raw material defect image of the target product;

Inputting the product defect image into a second defect feature extraction convolution model to generate a first product defect feature, wherein the product defect feature corresponds to a cutter defect;

And superposing the first product defect characteristic and the second product defect characteristic by a characteristic channel to generate the product defect characteristic.

4. The intelligent manufacturing process monitoring method according to claim 2, wherein inputting the product defect feature as a defect enhancement tag and the product processing real-time image into the second generator for defect feature fusion comprises:

dividing the real-time image for processing the product into areas, and generating processing areas corresponding to all cutters;

And inputting the product defect characteristics serving as defect enhancement tags and the product processing real-time images into the second generator, and carrying out defect characteristic fusion according to the processing area.

5. The intelligent manufacturing process monitoring method according to any one of claims 1 to 4, wherein the product defect probability is a product defect distribution probability including probabilities of at least two defects;

After inputting the cutter defect image into a first defect feature extraction convolution model to generate a cutter defect feature set, inputting the cutter defect feature set as a defect enhancement tag and the cutter real-time image into the first generator to perform defect feature fusion, and before generating a reconstructed image, the intelligent manufacturing production process monitoring method further comprises the following steps:

6. The intelligent manufacturing process monitoring method according to claim 5, wherein the screening the tool defect feature in the tool defect feature set according to the probability of product defect distribution comprises:

7. The intelligent manufacturing process monitoring method according to any one of claims 1 to 4, wherein inputting the tool defect feature set as a defect enhancement tag and the tool real-time image into the first generator for defect feature fusion, generating a reconstructed image, comprises:

performing 1*1 convolution operation on the cutter real-time image to generate cutter real-time convolution characteristics;

vector calculation is carried out on the characteristic channels of the second processing characteristics, a channel vector set is generated, and one-dimensional channel vectors are output according to the channel vector set;

Carrying out channel superposition on the fusion residual error, the cutter real-time convolution characteristic and the defect convolution characteristic to generate a fourth processing characteristic;

8. An intelligent manufacturing production process monitoring system based on artificial intelligence, comprising:

the first generation unit is used for inputting the product processing real-time image into the product defect detection convolution model to generate product defect probability;

the third generation unit is used for inputting the cutter defect feature set serving as a defect enhancement tag and the cutter real-time image into the first generator to perform defect feature fusion, and generating a reconstructed image;

a fourth generating unit, configured to input the reconstructed image and the tool initial image into the tool defect detection convolution model, and generate a tool defect distribution probability;

And a fifth generating unit, configured to generate a tool state result according to the tool defect distribution probability.

9. The intelligent manufacturing process monitoring system of claim 8, wherein the deep learning process monitoring model further comprises a second generator;

after the first obtaining unit, before the first generating unit, the intelligent manufacturing production process monitoring method further includes:

a third obtaining unit, configured to obtain a product defect feature;

10. The intelligent manufacturing process monitoring system of claim 9, wherein the deep learning process monitoring model further comprises a second defect feature extraction convolution model;

The third acquisition unit includes:

Acquiring a raw material defect image of the target product;