CN114022409B - Coated medicine surface defect detection algorithm based on deep learning - Google Patents
Coated medicine surface defect detection algorithm based on deep learning Download PDFInfo
- Publication number
- CN114022409B CN114022409B CN202111169031.6A CN202111169031A CN114022409B CN 114022409 B CN114022409 B CN 114022409B CN 202111169031 A CN202111169031 A CN 202111169031A CN 114022409 B CN114022409 B CN 114022409B
- Authority
- CN
- China
- Prior art keywords
- image
- medicine
- coating medicine
- coating
- residual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000003814 drug Substances 0.000 title claims abstract description 113
- 230000007547 defect Effects 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 238000013135 deep learning Methods 0.000 title claims abstract description 11
- 239000011248 coating agent Substances 0.000 claims abstract description 84
- 238000000576 coating method Methods 0.000 claims abstract description 84
- 238000000034 method Methods 0.000 claims abstract description 26
- 229940079593 drug Drugs 0.000 claims abstract description 17
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 238000003708 edge detection Methods 0.000 claims abstract description 7
- 230000005764 inhibitory process Effects 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 9
- 239000002699 waste material Substances 0.000 claims description 9
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 15
- 238000000605 extraction Methods 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 6
- 239000000126 substance Substances 0.000 description 6
- CSCPPACGZOOCGX-UHFFFAOYSA-N Acetone Chemical compound CC(C)=O CSCPPACGZOOCGX-UHFFFAOYSA-N 0.000 description 2
- RTZKZFJDLAIYFH-UHFFFAOYSA-N Diethyl ether Chemical compound CCOCC RTZKZFJDLAIYFH-UHFFFAOYSA-N 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000012824 chemical production Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 239000002085 irritant Substances 0.000 description 1
- 231100000021 irritant Toxicity 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000003380 propellant Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a coating medicine surface defect detection algorithm based on deep learning, and belongs to the field of intelligent manufacturing. The method comprises the steps of reconstructing a coating medicine image without surface defects by using a self-encoder based on depth convolution, and obtaining a residual error image by subtracting an input image and the reconstructed image; performing edge extraction on an input image by using an edge detection operator, and obtaining a segmentation mask of the coating medicine by means of OpenCV; multiplying the defect map and the segmentation mask pixel by pixel to obtain a residual map with background interference removed; accurately positioning the surface defect position of the coated medicine on an original image by a sliding window method and a non-maximum value inhibition method and utilizing a space coordinate conversion rule; the central controller controls the corresponding pneumatic device according to the detection information to realize intelligent outlier of the surface defect coating medicine. The method can accurately detect the surface defects and the positions of the coating medicines, and solves the problems of low production capacity and the like of a coating medicine selection link caused by manual selection.
Description
Technical Field
The invention belongs to the field of intelligent manufacturing, and particularly relates to a coating medicine surface defect detection algorithm based on deep learning.
Background
The chemical plant still adopts the production lines of the seventh and eighties of the last century in the chemical selection section, and has low automation level, long production period and high labor cost. Meanwhile, the production environment of the medicine selection process is full of irritant gases such as ethyl ether, acetone and alcohol, and therefore the physical health of workers is seriously threatened. At present, chemical plants still adopt a manual selection mode in a chemical selection process link, namely: 2 workers are specially configured in a factory for detecting and classifying the surface defects of the coating medicines, so that the labor intensity of the workers is increased, and the error rate of the classification of the coating medicines can be increased after the workers work for a long time. Although the production capacity of part of production workshops of a chemical plant is improved by using the traditional machine vision technology, the traditional machine vision technology has no adaptability and migratability in a real production environment due to various surface defect forms of the coating medicine, and still needs manual intervention.
In recent years, with the continuous integration of technologies such as artificial intelligence, computer vision and the like and the field of chemical equipment manufacturing, the production links of chemical production are promoted to turn towards the directions of intellectualization, integration and flexibility. Therefore, the computer vision technology is applied to accurately identify and position the surface defects of the coating medicine, so that the production capacity of a factory can be improved, and the labor intensity of workers can be reduced.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a coating medicine surface defect detection algorithm based on deep learning, aiming at the problems of low production efficiency, high labor intensity, severe working environment, potential safety hazard and the like of military product propellant in a medicine selection ring section. The method constructs a coating medicine surface defect detection algorithm based on deep learning, aims to detect the surface defects and the positions of the coating medicines rapidly, efficiently and in real time through a computer vision technology, has robustness and mobility, solves the 'neck clamping' link of further improving the production performance of a factory, and can accurately detect the defects and the positions of the surface of the coating medicines by providing a small amount of surface defect-free data sets through the provided neural network.
The technical scheme is as follows: a solution scheme for accurately detecting the surface defects of the coating medicine based on a deep convolution self-encoder and carrying out intelligent decision-making in real time is provided by a coating medicine surface defect detection algorithm based on deep learning, and specifically comprises the following steps:
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters;
step 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2, reconstructing to obtain a coating medicine image with a defect-free surface, and smoothing the reconstructed image by using a Gaussian filter GaussianBlur function, wherein the smoothed image is O (x，y) Wherein, (x, y) is the image coordinate;
step 4, inputting the image I in the step 3 (x，y) And the coating medicine image O reconstructed in the step 3 (x，y) Performing difference operation pixel by pixel to generate a coating drug residual error image R (x，y) ：
R (x，y) ＝I (x，y) -O (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (1)
Wherein h is 1 For the height of the input image, w 1 Is the width of the input image;
step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and acquiring a coating medicine segmentation mask M by means of an OpenCV (open circuit vehicle library) (x，y) Wherein the value of the segmentation mask in the drug-coated region is 1, and the value of the background region is 0;
step 6, the residual error image R obtained in the step 4 is processed (x，y) And the coating medicine segmentation mask M obtained in the step 5 (x，y) Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference (x，y) ：
R*M (x，y) ＝R (x，y) ×M (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (2)
Step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum suppression method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box.
The invention comprises the following steps of 7:
step 7-1, the height h of the coated medicine residual error image obtained in the step 6 1 And width w 1 Scaling to the maximum integer h with c as a divisor 2 And w 2 And generating m with c as the moving step c ×n c Empty matrix G (x) c ，y c )：
Wherein m is c ×n c Representing the generation of a residual image of the coating medicine with c as a basic unit and m as line numbers c The number of columns is n c Empty matrix G (x) c ，y c )，(x c ，y c ) Represents the in-space matrix G (x) c ，y c ) The middle row is the x-th row c Column is the y-th c A spatial position b w Difference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, b h The difference value of the residual error image obtained in the step 6 and the zoomed image height in the step 7-1 is obtained;
step 7-2, dividing the coating medicine residual error map generated in the step 6 into areas by using a sliding window method taking c as a moving step length, performing residual error calculation, and correspondingly placing the calculation result in the G obtained in the step 7-1 (x，y) Among the matrices;
wherein e is i Is the ith areaMaximum value of internal residual value, S i The ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step 7-3, using non-maximum suppression method to G of step 7-2 (x，y) And (3) processing: if the ratio S of the ith area to the maximum residual median of each area i If the value is larger than the set threshold value T, setting the value set value in the area to be 255, otherwise, setting the value set value to be 0;
step 7-4, the G obtained in the step 7-3 is converted by the image space coordinate conversion principle (x，y) Correspondingly restoring the area with the middle pixel value of 255 into the original input image, and positioning;
(x 1 ，y 1 ) As coordinates of the original image, (x) 2 ，y 2 ) Scaled for step 7-1 (x) 1 ，y 1 ) Corresponding coordinates;
step 7-5, performing intelligent judgment according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
Step 8 of the present invention comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste bin and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes a neural network based on a deep convolution self-encoder to detect the defects of the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device;
step 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
Drawings
The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
FIG. 1 is a general flowchart of a deep learning-based coating drug surface defect detection algorithm according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the training and application of a deep convolutional auto-encoder network according to an embodiment of the present invention;
FIG. 3 is an original input image according to an embodiment of the present invention;
FIG. 4 is an input image with Gaussian noise added according to an embodiment of the invention;
FIG. 5 is a reconstructed image after Gaussian filtering according to an embodiment of the present invention;
FIG. 6 is a residual image of an embodiment of the present invention;
FIG. 7 is an image of the edge detection by the Canny operator according to an embodiment of the present invention;
FIG. 8 is a segmentation mask according to an embodiment of the present invention;
FIG. 9 is a residual image with background interference removed according to an embodiment of the present invention;
FIG. 10 is a reconstructed image after sliding window processing and non-maximum suppression in accordance with an embodiment of the present invention;
FIG. 11 is a defect localization image of an implementation of the present invention;
fig. 12 is an intelligent sorting mechanism implemented in the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
The overall working flow of the device for coating the medicine surface defects, which is constructed by the method of the invention, is shown in figure 1 and is roughly divided into four steps: firstly, acquiring a coating medicine residual error image by utilizing a deep convolution self-encoder network; secondly, extracting a segmentation mask of the coating medicine by using a Canny edge detection operator and an OpenCV library to obtain a coating medicine residual image for inhibiting background interference; thirdly, intelligent decision is made by means of a window sliding method and a non-maximum value inhibition method, and defects are located on an original image; and fourthly, the central controller controls the corresponding pneumatic device according to the decision information to realize intelligent clustering of the defect coating medicine. The specific construction steps of the coating medicine surface defect edge detection algorithm based on deep learning in the embodiment of the invention are as follows:
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters, as shown in fig. 2;
step 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2, reconstructing to obtain a coating medicine image with a defect-free surface, and smoothing the reconstructed image by using a Gaussian filter GaussianBlur function, wherein the smoothed image is O (x，y) Wherein (x, y) is the image coordinates, as shown in fig. 5;
step 4, inputting the image I in the step 3 (x，y) And the coating medicine image O reconstructed in the step 3 (x，y) Performing difference operation pixel by pixel to generate a coating drug residual error image R (x，y) As shown in fig. 6:
R (x，y) ＝I (x，y) -O (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (1)
wherein h is 1 For the height of the input image, w 1 Is the width of the input image, where h 1 ＝224，w 1 ＝224；
Step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and acquiring a coating medicine segmentation mask M by means of an OpenCV (open circuit vehicle library) (x，y) Wherein the division mask covers a value of 1 in the drug regionThe value of the scene area is 0, as shown in fig. 8;
step 6, the residual error image R obtained in the step 4 is processed (x，y) And the coating medicine segmentation mask M obtained in the step 5 (x，y) Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference (x，y) As shown in fig. 9:
R*M (x，y) ＝R (x，y) ×M (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (2)
step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum value inhibition method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule; the intelligent detection algorithm for the surface defects of the coated medicine based on the deep convolution self-encoder completes the following indexes: the surface defect detection accuracy rate is 97.35%;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box. Wherein, the selection technology link realizes unmanned operation, compares traditional manual selection mode, realizes that the reduction rate is 70%, and single set selection equipment production efficiency is more than or equal to 300 kg/class.
Step 7 of the present invention comprises:
step 7-1, the height h of the residual image of the coating medicine obtained in the step 6 1 And width w 1 Scaling to the largest integer h with c =4 submultiples 2 And w 2 And generating m with c as the moving step c ×n c Matrix G (x, y):
wherein m is c ×n c Representing size generation from coated drug residual imagesHas c =4 as basic unit and m as line number c Number of columns n =56 c Empty matrix G (x) of =56 c ，y c )，(x c ，y c ) Represents the in-space matrix G (x) c ，y c ) The middle row is the x-th row c Column is the y-th c A spatial position b w Difference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, b h Is the difference between the residual map obtained in step 6 and the scaled image height obtained in step 7-1, where h 1 ＝h 2 ＝w 1 ＝w 2 ＝224，b w ＝b h ＝0；
Step 7-2, dividing the coated drug residual map generated in step 6 into regions by using a sliding window method with the moving step length of c =4, performing residual calculation, and correspondingly placing the calculation result in the G (x) obtained in step 7-1 c ，y c ) Among the matrices;
wherein e is i Is the maximum value of residual values in the ith region, S i The ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step 7-3, using non-maximum suppression method to G (x) of step 7-2 c ，y c ) And (3) processing: if the ratio S of the ith area to the maximum residual median of each area i If the value is greater than the set threshold value T =140, the value in the area is set to 255, otherwise, the value is set to 0, as shown in fig. 10;
step 7-4, G (x) obtained in the step 7-3 is converted by the image space coordinate conversion principle c ，y c ) The region with the middle pixel value of 255 is correspondingly restored in the original input image, and is positioned, as shown in fig. 11;
(x 1 ，y 1 ) As coordinates of the original image, (x) 2 ，y 2 ) Scaled (x) for step 7-1 1 ，y 1 ) Corresponding coordinates;
and 7-5, intelligently judging according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
Step 8 of the present invention comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste bin and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes the neural network based on the deep convolution self-encoder to detect the defects on the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device, as shown in figure 12;
and 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
The invention provides a method for detecting surface defects of a coated medicine based on deep learning, the open source method, the image processing tool, the parameter values and the like are all referred to for helping readers to understand the principle of the invention, and the protection scope of the invention is not limited to the specific statement and the embodiment. Those skilled in the art can make such changes and modifications without departing from the principles of the present invention, and such changes and modifications are intended to be within the scope of the present invention.
Claims (3)
1. The covered medicine surface defect detection algorithm based on deep learning is characterized by comprising the following steps:
step 1, collecting a coating medicine image with a defect-free surface as a training data set, and randomly increasing Gaussian noise; the collected surface defect image is used as a test data set;
step 2, building a neural network based on a deep convolution self-encoder, training the neural network by using the training data set generated in the step 1, and generating trained network parameters;
and 3, taking the test data set obtained in the step 1 as input, reconstructing a coating medicine image in the neural network trained in the step 2 to obtain a coating medicine image with a defect-free surface, smoothing the reconstructed image by using a Gaussian filter Gaussian Blur function, wherein the smoothed image is O (x，y) Wherein, (x, y) is the image coordinate;
step 4, inputting the image I in the step 3 (x，y) And the coating medicine image O reconstructed in the step 3 (x，y) Performing difference operation pixel by pixel to generate a coating drug residual error image R (x，y) ：
R (x，y) ＝I (x，y) -O (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (1)
Wherein h is 1 For the height of the input image, w 1 Is the width of the input image;
step 5, extracting the edge characteristics of the coating medicine from the input image in the step 3 by using a Canny edge detection operator, and acquiring a segmentation mask M of the coating medicine by means of an OpenCV (open computer vision library) (x，y) Wherein the value of the segmentation mask in the drug-coated region is 1, and the value of the background region is 0;
step 6, the residual error image R obtained in the step 4 is processed (x，y) And the coating medicine segmentation mask M obtained in the step 5 (x，y) Multiplying pixel by pixel to obtain a coating medicine residual error graph R M without background interference (x，y) ：
R*M (x，y) ＝R (x，y) ×M (x，y) ，x∈[0，h 1 )，y∈[0，w 1 ) (2)
Step 7, intelligently detecting the coating medicine residual error image obtained in the step 6 by using a sliding window method and a non-maximum value inhibition method, and positioning a defect area on the original image input in the step 3 by using an image space coordinate conversion rule;
and 8, the central controller takes the detection result obtained in the step 7 as an output signal to control the pneumatic device to act: if the signal is a coating medicine abnormal signal, the central controller controls the pneumatic device (2) to act, so that the coating medicine is sent into the waste bin; if the signal is a coating medicine normal signal, the central controller controls the pneumatic device (1) to act, so that the coating medicine is sent into the qualified box.
2. The method according to claim 1, wherein step 7 comprises:
step 7-1, the height h of the residual image of the coating medicine obtained in the step 6 1 And width w 1 Scaling to the maximum integer h with c as a divisor 2 And w 2 And generating m with c as the moving step c ×n c Empty matrix G (x) c ，y c )：
Wherein m is c ×n c Representing the generation of a residual image of the coating medicine with c as a basic unit and m as line numbers c The number of columns is n c Empty matrix G (x) c ，y c )，(x c ，y c ) Represents the in-space matrix G (x) c ，y c ) The middle row is the x-th row c Column is the y-th c Spatial position of (b) w Difference between the residual image of the coated drug obtained in step 6 and the zoomed image width obtained in step 7-1, b h The difference value of the residual error image obtained in the step 6 and the zoomed image height in the step 7-1 is obtained;
step 7-2, using a sliding window method with the step c as the moving step length to carry out the residual error treatment on the coating medicine generated in the step 6Dividing the graph into regions, performing residual calculation, and correspondingly placing the calculation result in G (x) obtained in step 7-1 c ，y c ) Among the matrices;
wherein e is i Is the maximum value of residual values in the ith region, S i The ratio of the maximum value of the residual values in the ith area to the median of the maximum values of the residual values in all the areas is obtained;
step 7-3, using non-maximum suppression method to G (x) of step 7-2 c ，y c ) And (3) processing: if the ratio S of the ith area to the maximum residual median of each area i If the value is larger than the set threshold value T, setting the value set value in the area to be 255, otherwise, setting the value set value to be 0;
step 7-4, G (x) obtained in the step 7-3 is converted by the image space coordinate conversion principle c ，y c ) Correspondingly restoring the area with the medium pixel value of 255 into the original input image, and positioning;
(x 1 ，y 1 ) As coordinates of the original image, (x) 2 ，y 2 ) Scaled (x) for step 7-1 1 ，y 1 ) Corresponding coordinates;
step 7-5, performing intelligent judgment according to the detection result of the step 7-3: if the area is 255, judging that the coating medicine has surface defects; otherwise, the coated medicine is judged to be qualified.
3. The method according to claim 1, wherein step 8 comprises:
step 8-1, the intelligent classification device comprises an explosion-proof camera, a central processing unit, a pneumatic device (1), a pneumatic device (2), a qualified box, a waste bin and a feeding device; the explosion-proof camera collects the coating medicine on the feeding device in real time and transmits the image to the central processing unit; the central processing unit utilizes a neural network based on a deep convolution self-encoder to detect the defects of the surface of the coated medicine, and the detection result is used as an output signal to control the action of the pneumatic device;
and 8-2, controlling the pneumatic device to act according to the control signal obtained in the step 8-1: if the detection result is that the medicine is normally coated, the central controller controls the pneumatic device (1) to send the coated medicine into a qualified box; if the detection result is abnormal coating medicine, the central controller controls the pneumatic device (2) to send the coating medicine into the waste bin.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169031.6A CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169031.6A CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114022409A CN114022409A (en) | 2022-02-08 |
| CN114022409B true CN114022409B (en) | 2023-04-18 |
Family
ID=80055375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111169031.6A Active CN114022409B (en) | 2021-09-30 | 2021-09-30 | Coated medicine surface defect detection algorithm based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114022409B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862847B (en) * | 2022-07-05 | 2022-09-16 | 聊城高新生物技术有限公司 | Method and system for detecting shape defects of coated medicine based on pattern recognition |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1675376A (en) * | 2002-06-13 | 2005-09-28 | 惠氏控股公司 | Inhibitors of inflammatory gene activity and cholesterol biosynthesis |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN109872313A (en) * | 2019-02-15 | 2019-06-11 | 苏州晓创光电科技有限公司 | A kind of method for detecting surface defects of products based on depth convolution self-encoding encoder |
| CN110605799A (en) * | 2019-09-23 | 2019-12-24 | 湖北航天化学技术研究所 | Side surface coating die and coating method for solid propellant grain |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| CN111815601A (en) * | 2020-07-03 | 2020-10-23 | 浙江大学 | Texture image surface defect detection method based on depth convolution self-encoder |
| CN112668725A (en) * | 2020-05-21 | 2021-04-16 | 广东顺德募优网络科技有限公司 | Metal hand basin defect target training method based on improved features |
| CN112692578A (en) * | 2020-12-21 | 2021-04-23 | 上海航天化工应用研究所 | Safe shaping and detection intelligent processing system for coated grains |
-
2021
- 2021-09-30 CN CN202111169031.6A patent/CN114022409B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1675376A (en) * | 2002-06-13 | 2005-09-28 | 惠氏控股公司 | Inhibitors of inflammatory gene activity and cholesterol biosynthesis |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN109872313A (en) * | 2019-02-15 | 2019-06-11 | 苏州晓创光电科技有限公司 | A kind of method for detecting surface defects of products based on depth convolution self-encoding encoder |
| CN110605799A (en) * | 2019-09-23 | 2019-12-24 | 湖北航天化学技术研究所 | Side surface coating die and coating method for solid propellant grain |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| CN112668725A (en) * | 2020-05-21 | 2021-04-16 | 广东顺德募优网络科技有限公司 | Metal hand basin defect target training method based on improved features |
| CN111815601A (en) * | 2020-07-03 | 2020-10-23 | 浙江大学 | Texture image surface defect detection method based on depth convolution self-encoder |
| CN112692578A (en) * | 2020-12-21 | 2021-04-23 | 上海航天化工应用研究所 | Safe shaping and detection intelligent processing system for coated grains |
Non-Patent Citations (1)
| Title |
|---|
| 蔡虹.包覆药柱界面粘结质量的无损检测.上海航天.2004,(第03期),54-57. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114022409A (en) | 2022-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108280856B (en) | Unknown object grabbing pose estimation method based on mixed information input network model | |
| CN109658413B (en) | Method for detecting grabbing position of robot target object | |
| CN107194559B (en) | Workflow identification method based on three-dimensional convolutional neural network | |
| CN107818571B (en) | Ship automatic tracking method and system based on deep learning network and average drifting | |
| RU2592650C2 (en) | Method of filtering images of target object in robotics system | |
| CN108196453B (en) | Intelligent calculation method for mechanical arm motion planning group | |
| CN114022409B (en) | Coated medicine surface defect detection algorithm based on deep learning | |
| CN111652853A (en) | Magnetic powder flaw detection method based on deep convolutional neural network | |
| CN107138432A (en) | Non-rigid object method for sorting and device | |
| CN112861880B (en) | Weak supervision RGBD image saliency detection method and system based on image classification | |
| CN111062278A (en) | Abnormal behavior identification method based on improved residual error network | |
| CN111368637B (en) | Transfer robot target identification method based on multi-mask convolutional neural network | |
| CN114511786A (en) | Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution | |
| CN108161938A (en) | Visual tracking method and system in a kind of robot | |
| CN112598684A (en) | Open-pit area ground feature segmentation method based on semantic segmentation technology | |
| CN109493370B (en) | Target tracking method based on space offset learning | |
| CN113312966A (en) | Action identification method and device based on first-person visual angle | |
| CN110111358B (en) | Target tracking method based on multilayer time sequence filtering | |
| CN109800793B (en) | Target detection method and system based on deep learning | |
| CN109409224B (en) | Method for detecting flame in natural scene | |
| CN113780136B (en) | VOCs gas leakage detection method, system and equipment based on space-time texture recognition | |
| CN111460941B (en) | Visual navigation feature point extraction and matching method in wearable navigation equipment | |
| CN112818836B (en) | Method and system for detecting personnel target of transformer substation scene | |
| CN109934155B (en) | Depth vision-based collaborative robot gesture recognition method and device | |
| CN113487538A (en) | Multi-target segmentation defect detection method and device and computer storage medium thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |