CN114022409A - Coated medicine surface defect detection algorithm based on deep learning - Google Patents
Coated medicine surface defect detection algorithm based on deep learning Download PDFInfo
- Publication number
- CN114022409A CN114022409A CN202111169031.6A CN202111169031A CN114022409A CN 114022409 A CN114022409 A CN 114022409A CN 202111169031 A CN202111169031 A CN 202111169031A CN 114022409 A CN114022409 A CN 114022409A
- Authority
- CN
- China
- Prior art keywords
- image
- medicine
- coating medicine
- coating
- residual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000003814 drug Substances 0.000 title claims abstract description 113
- 230000007547 defect Effects 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 238000013135 deep learning Methods 0.000 title claims abstract description 11
- 239000011248 coating agent Substances 0.000 claims abstract description 86
- 238000000576 coating method Methods 0.000 claims abstract description 86
- 238000000034 method Methods 0.000 claims abstract description 26
- 229940079593 drug Drugs 0.000 claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims abstract description 13
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 238000003708 edge detection Methods 0.000 claims abstract description 7
- 230000005764 inhibitory process Effects 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 9
- 239000002699 waste material Substances 0.000 claims description 9
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 4
- 230000009471 action Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 15
- 238000000605 extraction Methods 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 6
- 239000000126 substance Substances 0.000 description 6
- CSCPPACGZOOCGX-UHFFFAOYSA-N Acetone Chemical compound CC(C)=O CSCPPACGZOOCGX-UHFFFAOYSA-N 0.000 description 2
- RTZKZFJDLAIYFH-UHFFFAOYSA-N Diethyl ether Chemical compound CCOCC RTZKZFJDLAIYFH-UHFFFAOYSA-N 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 108010025037 T140 peptide Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000012824 chemical production Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 239000002085 irritant Substances 0.000 description 1
- 231100000021 irritant Toxicity 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000003380 propellant Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a coating medicine surface defect detection algorithm based on deep learning, and belongs to the field of intelligent manufacturing. The method comprises the steps of reconstructing a coating medicine image with no surface defect by using a depth convolution-based self-editor, and obtaining a residual error image by subtracting an input image and a reconstructed image; performing edge extraction on an input image by using an edge detection operator, and obtaining a segmentation mask of the coating medicine by means of OpenCV; multiplying the defect map and the segmentation mask pixel by pixel to obtain a residual map with background interference removed; accurately positioning the surface defect position of the coated medicine on an original image by a sliding window method and a non-maximum value inhibition method and utilizing a space coordinate conversion rule; the central controller controls the corresponding pneumatic device according to the detection information to realize intelligent clustering of the surface defect coating medicine. The method can accurately detect the surface defects and the positions of the coating medicines, and solves the problems of low production capacity and the like of a coating medicine selection link caused by manual selection.
Description
Technical Field
The invention belongs to the field of intelligent manufacturing, and particularly relates to a coating medicine surface defect detection algorithm based on deep learning.
Background
The chemical plant still adopts the production line of the seventh and eighties of the last century in the chemical selection section, and has low automation level, long production period and high labor cost. Meanwhile, the production environment of the drug selection process is full of irritant gases such as ether, acetone, alcohol and the like, so that the physical health of workers is seriously threatened. At present, chemical plants still adopt a manual selection mode in a chemical selection process link, namely: 2 workers are specially configured in a factory for detecting and classifying the surface defects of the coating medicines, so that the labor intensity of the workers is increased, and the error rate of the classification of the coating medicines can be increased after the workers work for a long time. Although the production capacity of part of production workshops of a chemical plant is improved by using the traditional machine vision technology, the traditional machine vision technology has no adaptability and migratability in a real production environment due to various surface defect forms of the coating medicine, and still needs manual intervention.
In recent years, with the continuous integration of technologies such as artificial intelligence, computer vision and the like and the field of chemical equipment manufacturing, the production links of chemical production are promoted to turn towards the directions of intellectualization, integration and flexibility. Therefore, the computer vision technology is applied to accurately identify and position the surface defects of the coating medicine, so that the production capacity of a factory can be improved, and the labor intensity of workers can be reduced.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a coating medicine surface defect detection algorithm based on deep learning, aiming at the problems of low production efficiency, high labor intensity, severe working environment, potential safety hazard and the like of military product propellant in a medicine selection ring section. The method constructs a coating medicine surface defect detection algorithm based on deep learning, aims to detect the surface defects and the positions of the coating medicines rapidly, efficiently and in real time through a computer vision technology, has robustness and mobility, solves the 'neck clamping' link of further improving the production performance of a factory, and can accurately detect the defects and the positions of the surface of the coating medicines by providing a small amount of surface defect-free data sets through the provided neural network.
The technical scheme is as follows: a solution scheme for accurately detecting the surface defects of the coating medicine based on a deep convolution self-encoder and carrying out intelligent decision-making in real time is provided by a coating medicine surface defect detection algorithm based on deep learning, and specifically comprises the following steps:
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters;
step 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2, reconstructing to obtain a coating medicine image with a defect-free surface, and smoothing the reconstructed image by using a Gaussian filter GaussianBlur function, wherein the smoothed image is O(x，y)Wherein, (x, y) is the image coordinate;
step 4, inputting the image I in the step 3(x，y)And the coating medicine image O reconstructed in the step 3(x，y)Performing difference operation pixel by pixel to generate a coating drug residual error image R(x，y)：
R(x，y)＝I(x，y)-O(x，y)，x∈[0，h1)，y∈[0，w1) (1)
Wherein h is1For the height of the input image, w1Is the width of the input image;
step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and borrowing the edge characteristicsHelp OpenCV library to obtain coating medicine segmentation mask M(x，y)Wherein the value of the segmentation mask in the drug-coated region is 1, and the value of the background region is 0;
step 6, the residual error image R obtained in the step 4 is processed(x，y)And the coating medicine segmentation mask M obtained in the step 5(x，y)Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference(x，y)：
R*M(x，y)＝R(x，y)×M(x，y)，x∈[0，h1)，y∈[0，w1) (2)
Step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum value inhibition method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box.
The invention comprises the following steps of 7:
step 7-1, the height h of the residual image of the coating medicine obtained in the step 61And width w1Scaling to the maximum integer h with c as a divisor2And w2And generating m with c as the moving stepc×ncEmpty matrix G (x)c，yc)：
Wherein m isc×ncRepresenting the generation of a residual image of the coating medicine with c as a basic unit and m as line numberscThe number of columns is ncEmpty matrix G (x)c，yc)，(xc，yc) Represents the in-space matrix G (x)c，yc) InThe row is the x-thcColumn is the y-thcA spatial position bwDifference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, bhThe difference value of the residual error image obtained in the step 6 and the zoomed image height in the step 7-1 is obtained;
step 7-2, dividing the coating medicine residual error map generated in the step 6 into areas by using a sliding window method taking c as a moving step length, performing residual error calculation, and correspondingly placing the calculation result in the G obtained in the step 7-1(x，y)Among the matrices;
wherein e isiIs the maximum value of residual values in the ith region, SiThe ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step 7-3, using non-maximum suppression method to G of step 7-2(x，y)And (3) processing: if the ratio S of the ith area to the maximum residual median of each areaiIf the value is larger than the set threshold value T, setting the value set value in the area to be 255, otherwise, setting the value set value to be 0;
step 7-4, the G obtained in the step 7-3 is converted by the image space coordinate conversion principle(x，y)Correspondingly restoring the area with the middle pixel value of 255 into the original input image, and positioning;
(x1，y1) As coordinates of the original image, (x)2，y2) Scaled (x) for step 7-11，y1) Corresponding coordinates;
and 7-5, intelligently judging according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
Step 8 of the present invention comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste box and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes a neural network based on a deep convolution self-encoder to detect the defects of the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device;
and 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
Drawings
The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
FIG. 1 is a general flowchart of a deep learning-based coating drug surface defect detection algorithm according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the training and application of a deep convolutional auto-encoder network according to an embodiment of the present invention;
FIG. 3 is an original input image according to an embodiment of the present invention;
FIG. 4 is an input image with Gaussian noise added according to an embodiment of the invention;
FIG. 5 is a reconstructed image after Gaussian filtering according to an embodiment of the present invention;
FIG. 6 is a residual image of an embodiment of the present invention;
FIG. 7 is an image of the edge detection by the Canny operator according to an embodiment of the present invention;
FIG. 8 is a segmentation mask according to an embodiment of the present invention;
FIG. 9 is a residual image with background interference removed according to an embodiment of the present invention;
FIG. 10 is a reconstructed image after sliding window processing and non-maximum suppression in accordance with an embodiment of the present invention;
FIG. 11 is a defect localization image of an implementation of the present invention;
fig. 12 illustrates an intelligent sorting mechanism implemented in accordance with the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
The overall working flow of the device for coating the medicine surface defects, which is constructed by the method of the invention, is shown in figure 1 and is roughly divided into four steps: firstly, acquiring a residual error image of a coating medicine by utilizing a depth convolution self-encoder network; secondly, extracting a segmentation mask of the coating medicine by using a Canny edge detection operator and an OpenCV library to obtain a coating medicine residual image for inhibiting background interference; thirdly, intelligent decision is made by means of a window sliding method and a non-maximum value inhibition method, and defects are located on an original image; and fourthly, the central controller controls the corresponding pneumatic device according to the decision information to realize intelligent clustering of the defect coating medicine. The specific construction steps of the coating medicine surface defect edge detection algorithm based on deep learning in the embodiment of the invention are as follows:
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters, as shown in fig. 2;
step 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2, reconstructing to obtain a coating medicine image with a defect-free surface, and smoothing the reconstructed image by using a Gaussian filter GaussianBlur function, wherein the smoothed image is O(x，y)Wherein (x, y) is the image coordinates, as shown in fig. 5;
step 4, the output of the step 3Input image I(x，y)And the coating medicine image O reconstructed in the step 3(x，y)Performing difference operation pixel by pixel to generate a coating drug residual error image R(x，y)As shown in fig. 6:
R(x，y)＝I(x，y)-O(x，y)，x∈[0，h1)，y∈[0，w1) (1)
wherein h is1For the height of the input image, w1For the width of the input image, here h1＝224，w1＝224；
Step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and acquiring a coating medicine segmentation mask M by means of an OpenCV (open circuit vehicle library)(x，y)Wherein the value of the segmentation mask in the covered drug region is 1, and the value of the background region is 0, as shown in fig. 8;
step 6, the residual error image R obtained in the step 4 is processed(x，y)And the coating medicine segmentation mask M obtained in the step 5(x，y)Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference(x，y)As shown in fig. 9:
R*M(x，y)＝R(x，y)×M(x，y)，x∈[0，h1)，y∈[0，w1) (2)
step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum value inhibition method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule; the intelligent detection algorithm for the surface defects of the coated medicine based on the deep convolution self-encoder completes the following indexes: the surface defect detection accuracy rate is 97.35%;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box. Wherein, the selection technology link realizes unmanned operation, compares traditional manual selection mode, realizes that the reduction rate is 70%, and single set selection equipment production efficiency is more than or equal to 300 kg/class.
The invention comprises the following steps of 7:
step 7-1, the height h of the residual image of the coating medicine obtained in the step 61And width w1Scaling to the maximum integer h with c-4 as a divisor2And w2And generating m with c as the moving stepc×ncMatrix G (x, y):
wherein m isc×ncRepresenting the generation of residual image with c-4 as basic unit and m lines according to the size of the coated medicine residual imagec56 columns nc56 empty matrix G (x)c，yc)，(xc，yc) Represents the in-space matrix G (x)c，yc) The middle row is the x-th rowcColumn is the y-thcA spatial position bwDifference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, bhIs the difference between the residual map obtained in step 6 and the scaled image height obtained in step 7-1, where h1＝h2＝w1＝w2＝224，bw＝bh＝0；
And 7-2, dividing the coating medicine residual map generated in the step 6 into regions by using a sliding window method with the moving step length of c-4, calculating residual errors, and correspondingly placing the calculation results in G (x) obtained in the step 7-1c，yc) Among the matrices;
wherein e isiIs the maximum value of residual values in the ith region, SiThe ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step (ii) of7-3, treating G (x) of step 7-2 with a non-maxima suppression methodc，yc) And (3) processing: if the ratio S of the ith area to the maximum residual median of each areaiIf the value is greater than the set threshold T140, the value in the region is set to 255, otherwise, the value is set to 0, as shown in fig. 10;
step 7-4, G (x) obtained in the step 7-3 is converted by the image space coordinate conversion principlec，yc) The area with the middle pixel value of 255 is correspondingly restored to the original input image and is positioned, as shown in fig. 11;
(x1，y1) As coordinates of the original image, (x)2，y2) Scaled (x) for step 7-11，y1) Corresponding coordinates;
and 7-5, intelligently judging according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
Step 8 of the present invention comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste box and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes the neural network based on the deep convolution self-encoder to detect the defects of the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device, as shown in figure 12;
and 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
The invention provides a method for detecting surface defects of a coated medicine based on deep learning, the open source method, the image processing tool, the parameter values and the like are all referred to for helping readers to understand the principle of the invention, and the protection scope of the invention is not limited to the specific statement and the embodiment. Those skilled in the art can make such changes and modifications without departing from the principles of the present invention, and such changes and modifications are intended to be within the scope of the present invention.
Claims (3)
1. The covered medicine surface defect detection algorithm based on deep learning is characterized by comprising the following steps:
step 1, collecting a coating medicine image with a defect-free surface as a training data set, and randomly increasing Gaussian noise; the collected surface defect image is used as a test data set;
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters;
and 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2 to obtain a coating medicine image with a defect-free surface, smoothing the reconstructed image by using a Gaussian filter Gaussian Blur function, wherein the smoothed image is O(x，y)Wherein, (x, y) is the image coordinate;
step 4, inputting the image I in the step 3(x，y)And the coating medicine image O reconstructed in the step 3(x，y)Performing difference operation pixel by pixel to generate a coating drug residual error image R(x，y)：
R(x，y)＝I(x，y)-O(x，y)，x∈[0，h1)，y∈[0，w1) (1)
Wherein h is1For the height of the input image, w1Is the width of the input image;
step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and acquiring a coating medicine segmentation mask M by means of an OpenCV (open circuit vehicle library)(x，y)Wherein the value of the segmentation mask in the drug-coated region is 1, and the value of the background region is 0;
step 6, the residual error image R obtained in the step 4 is processed(x，y)And the coating medicine segmentation mask M obtained in the step 5(x，y)Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference(x，y)：
R*M(x，y)＝R(x，y)×M(x，y)，x∈[0，h1)，y∈[0，w1) (2)
Step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum value inhibition method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box.
2. The method according to claim 1, wherein step 7 comprises:
step 7-1, the height h of the residual image of the coating medicine obtained in the step 61And width w1Scaling to the maximum integer h with c as a divisor2And w2And generating m with c as the moving stepc×ncEmpty matrix G (x)c，yc)：
Wherein m isc×ncRepresenting the generation of a residual image of the coating medicine with c as a basic unit and m as line numberscThe number of columns is ncEmpty matrix G (x)c，yc)，(xc，yc) Represents the in-space matrix G (x)c，yc) The middle row is the x-th rowcColumn is the y-thcSpatial position of (b)wDifference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, bhThe difference value of the residual error image obtained in the step 6 and the zoomed image height in the step 7-1 is obtained;
step 7-2, dividing the coating medicine residual error map generated in the step 6 into areas by using a sliding window method taking c as a moving step length, performing residual error calculation, and correspondingly placing the calculation result in the G (x) obtained in the step 7-1c，yc) Among the matrices;
wherein e isiIs the maximum value of residual values in the ith region, SiThe ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step 7-3, using non-maximum suppression method to G (x) of step 7-2c，yc) And (3) processing: if the ratio S of the ith area to the maximum residual median of each areaiIf the value is larger than the set threshold value T, setting the value set value in the area to be 255, otherwise, setting the value set value to be 0;
step 7-4, G (x) obtained in the step 7-3 is converted by the image space coordinate conversion principlec，yc) Correspondingly restoring the area with the middle pixel value of 255 into the original input image, and positioning;
(x1，y1) As coordinates of the original image, (x)2，y2) Scaled (x) for step 7-11，y1) Corresponding coordinates;
and 7-5, intelligently judging according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
3. The method according to claim 1, wherein step 8 comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste box and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes a neural network based on a deep convolution self-encoder to detect the defects of the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device;
and 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169031.6A CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169031.6A CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114022409A true CN114022409A (en) | 2022-02-08 |
| CN114022409B CN114022409B (en) | 2023-04-18 |
Family
ID=80055375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111169031.6A Active CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114022409B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862847A (en) * | 2022-07-05 | 2022-08-05 | 聊城高新生物技术有限公司 | Method and system for detecting shape defects of coated medicine based on pattern recognition |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1675376A (en) * | 2002-06-13 | 2005-09-28 | 惠氏控股公司 | Inhibitors of inflammatory gene activity and cholesterol biosynthesis |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN109872313A (en) * | 2019-02-15 | 2019-06-11 | 苏州晓创光电科技有限公司 | A kind of method for detecting surface defects of products based on depth convolution self-encoding encoder |
| CN110605799A (en) * | 2019-09-23 | 2019-12-24 | 湖北航天化学技术研究所 | Side surface coating die and coating method for solid propellant grain |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| CN111815601A (en) * | 2020-07-03 | 2020-10-23 | 浙江大学 | Texture image surface defect detection method based on depth convolution self-encoder |
| CN112668725A (en) * | 2020-05-21 | 2021-04-16 | 广东顺德募优网络科技有限公司 | Metal hand basin defect target training method based on improved features |
| CN112692578A (en) * | 2020-12-21 | 2021-04-23 | 上海航天化工应用研究所 | Safe shaping and detection intelligent processing system for coated grains |
-
2021
- 2021-09-30 CN CN202111169031.6A patent/CN114022409B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1675376A (en) * | 2002-06-13 | 2005-09-28 | 惠氏控股公司 | Inhibitors of inflammatory gene activity and cholesterol biosynthesis |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN109872313A (en) * | 2019-02-15 | 2019-06-11 | 苏州晓创光电科技有限公司 | A kind of method for detecting surface defects of products based on depth convolution self-encoding encoder |
| CN110605799A (en) * | 2019-09-23 | 2019-12-24 | 湖北航天化学技术研究所 | Side surface coating die and coating method for solid propellant grain |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| CN112668725A (en) * | 2020-05-21 | 2021-04-16 | 广东顺德募优网络科技有限公司 | Metal hand basin defect target training method based on improved features |
| CN111815601A (en) * | 2020-07-03 | 2020-10-23 | 浙江大学 | Texture image surface defect detection method based on depth convolution self-encoder |
| CN112692578A (en) * | 2020-12-21 | 2021-04-23 | 上海航天化工应用研究所 | Safe shaping and detection intelligent processing system for coated grains |
Non-Patent Citations (1)
| Title |
|---|
| 蔡虹: "包覆药柱界面粘结质量的无损检测" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862847A (en) * | 2022-07-05 | 2022-08-05 | 聊城高新生物技术有限公司 | Method and system for detecting shape defects of coated medicine based on pattern recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114022409B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322510B (en) | 6D pose estimation method using contour information | |
| CN109658413B (en) | Method for detecting grabbing position of robot target object | |
| CN108280856B (en) | Unknown object grabbing pose estimation method based on mixed information input network model | |
| CN107194559B (en) | Workflow identification method based on three-dimensional convolutional neural network | |
| CN108196453B (en) | Intelligent calculation method for mechanical arm motion planning group | |
| CN111652853A (en) | Magnetic powder flaw detection method based on deep convolutional neural network | |
| CN111242202B (en) | Method for monitoring wear state of turning tool based on metric learning | |
| CN106600639A (en) | Genetic algorithm and adaptive threshold constraint-combined ICP (iterative closest point) pose positioning technology | |
| CN110210497B (en) | Robust real-time weld feature detection method | |
| CN111368637B (en) | Transfer robot target identification method based on multi-mask convolutional neural network | |
| CN114022409B (en) | Coated medicine surface defect detection algorithm based on deep learning | |
| CN106067031B (en) | Based on artificial mechanism for correcting errors and deep learning network cooperation machine vision recognition system | |
| CN114241522A (en) | Method, system, equipment and storage medium for field operation safety wearing identification | |
| CN113487538A (en) | Multi-target segmentation defect detection method and device and computer storage medium thereof | |
| CN109034540B (en) | Machine tool sequence arrangement dynamic prediction method based on work-in-process flow | |
| Wang et al. | Automated defect and contaminant inspection of HVAC duct | |
| CN115007474A (en) | Coal dressing robot and coal dressing method based on image recognition | |
| CN109409224B (en) | Method for detecting flame in natural scene | |
| CN114029941A (en) | Robot grabbing method and device, electronic equipment and computer medium | |
| CN116188467B (en) | Abnormality detection method for long surface of blade lithium battery shell | |
| CN110111358B (en) | Target tracking method based on multilayer time sequence filtering | |
| CN112818836B (en) | Method and system for detecting personnel target of transformer substation scene | |
| CN115424075A (en) | Method and system for monitoring pipeline state | |
| CN114841930A (en) | Product defect detection method based on parallel multi-scale fusion network | |
| Koker et al. | Development of a vision based object classification system for an industrial robotic manipulator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |