CN115508366B - Product defect intelligent detection system and method based on multispectral imaging - Google Patents
Product defect intelligent detection system and method based on multispectral imaging Download PDFInfo
- Publication number
- CN115508366B CN115508366B CN202211289620.2A CN202211289620A CN115508366B CN 115508366 B CN115508366 B CN 115508366B CN 202211289620 A CN202211289620 A CN 202211289620A CN 115508366 B CN115508366 B CN 115508366B
- Authority
- CN
- China
- Prior art keywords
- defect
- data
- product
- detection
- production
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 199
- 230000007547 defect Effects 0.000 title claims abstract description 131
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000000701 chemical imaging Methods 0.000 title claims abstract description 14
- 238000004519 manufacturing process Methods 0.000 claims abstract description 118
- 238000004364 calculation method Methods 0.000 claims abstract description 59
- 238000004458 analytical method Methods 0.000 claims abstract description 43
- 238000013135 deep learning Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims description 33
- 230000011218 segmentation Effects 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 26
- 238000005286 illumination Methods 0.000 claims description 15
- 230000035945 sensitivity Effects 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 8
- 238000003709 image segmentation Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 238000007689 inspection Methods 0.000 claims description 4
- 239000000654 additive Substances 0.000 claims description 3
- 230000000996 additive effect Effects 0.000 claims description 3
- 238000013434 data augmentation Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000000513 principal component analysis Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 2
- 230000000875 corresponding effect Effects 0.000 description 22
- 238000005516 engineering process Methods 0.000 description 7
- 230000001276 controlling effect Effects 0.000 description 6
- 238000011161 development Methods 0.000 description 6
- 230000018109 developmental process Effects 0.000 description 6
- 235000013305 food Nutrition 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000007306 turnover Effects 0.000 description 4
- 235000013311 vegetables Nutrition 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 235000016709 nutrition Nutrition 0.000 description 2
- 230000035764 nutrition Effects 0.000 description 2
- 238000005481 NMR spectroscopy Methods 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 239000002253 acid Substances 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229920002678 cellulose Polymers 0.000 description 1
- 239000001913 cellulose Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 230000000050 nutritive effect Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 229940088594 vitamin Drugs 0.000 description 1
- 229930003231 vitamin Natural products 0.000 description 1
- 235000013343 vitamin Nutrition 0.000 description 1
- 239000011782 vitamin Substances 0.000 description 1
- 150000003722 vitamin derivatives Chemical class 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
- G06F17/153—Multidimensional correlation or convolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2201/00—Features of devices classified in G01N21/00
- G01N2201/12—Circuits of general importance; Signal processing
- G01N2201/129—Using chemometrical methods
- G01N2201/1296—Using chemometrical methods using neural networks
Abstract
The invention relates to the technical field of defect detection, and discloses a product defect intelligent detection system and a method thereof based on multispectral imaging, wherein the system comprises a production detection end for intelligent detection and edge calculation of the product defect and a cloud processing platform; the production detection end comprises a production detection line, a multispectral camera, a robot and a product defect detection deep learning system, and the cloud processing platform comprises a defect problem analysis module and a defect problem association module.
Description
Technical Field
The invention relates to the technical field of defect detection, in particular to a product defect intelligent detection system and method based on multispectral imaging.
Background
With the development of computer technology and artificial intelligence, particularly deep learning, in order to ensure the quality of products in food processing, quality detection is generally required, and currently common quality detection methods mainly comprise machine vision, thermal imaging technology, nuclear magnetic resonance technology, multispectral image detection method and the like.
The multispectral image technology is used as a novel analysis and detection technology and has certain application in a plurality of research fields such as biomedicine, food detection and the like. Currently, multispectral image technology breaks through the geological remote sensing field and is gradually applied to evaluation and inspection of food quality. The development trend is mainly divided into: (1) The vegetable food is not suitable for storage due to short shelf life, and the nutrition substances change too fast, so that the quick nondestructive detection of the nutrition quality of the vegetables is urgently needed. The content of soluble solids is a comprehensive index comprising components such as soluble sugar, acid, vitamin, mineral, cellulose and the like, is an important parameter for evaluating the nutritive value of vegetables, and is long in time consumption and needs to destroy samples by using a refractometer method at present, so that rapid nondestructive detection of the vegetables by using a multispectral image system is an important point of attention of researchers; (2) The multi-spectral image technology is combined with unmanned aerial vehicle aerial photography in the field of grain and oil crop planting, so that large-area, rapid and accurate pest control can be realized; (3) The near infrared diffuse reflection spectrum has the advantages of high analysis speed, low cost, no damage, simple pretreatment and the like in the field of quality control of traditional Chinese medicinal materials, and can accurately classify and identify the easily-mixed medicinal materials.
The prior art can adopt a machine vision system and multispectral imaging to identify and classify products and detect defects of the products, but a detection system in the prior art lacks an edge calculation function, and has low intelligent degree of detection at a production end, large limitation and no autonomous learning function, so that the intelligent degree of the existing multispectral imaging product defect intelligent detection system is still to be developed, rich information under different defect depths is difficult to obtain, and the system and the method are difficult to be suitable for the development of multispectral imaging product defect detection.
Disclosure of Invention
The present invention has been made in view of the above-described problems in quality inspection of production and manufacture of the conventional products.
Therefore, the invention provides a product defect intelligent detection system and a method based on multispectral imaging, which can automatically identify the defects of a processed product to be detected by multispectral screening, machine learning and edge calculation, and simultaneously perform optimized special detection through a cloud after the edge calculation, so that data of autonomous learning and training are further optimized and fed back, the accuracy of product defect detection and the comprehensiveness of other related data application are improved, and the intelligent development of product defects is improved.
In order to solve the technical problems, the invention provides the following technical scheme: the production detection end for intelligent detection and edge calculation of the product defects and the cloud processing platform are included; the production detection end comprises a production detection line for enabling a product to be detected to displace, a multispectral camera for shooting the product to be detected, a robot for controlling the spatial position of the multispectral camera, and a product defect detection deep learning system for forming multispectral image data according to the multispectral camera after shooting; the cloud processing platform comprises a defect problem analysis module and a defect problem association module, and is used for calling edge calculation of the production detection end to generate edge calculation data, then carrying out defect problem analysis, and simultaneously carrying out association autonomous learning to generate feedback data and transmitting the feedback data to the production detection end, and the cloud processing platform is used for controlling corresponding intelligent product defect detection equipment and carrying out analysis optimization of a product defect detection deep learning system by the production detection end;
the product defect detection deep learning system comprises a data acquisition module, a data processing module, a model training module and an image recognition module; the data acquisition module is used for screening characteristic wave bands based on multispectral image data; the data processing module is used for preprocessing the multispectral image data to obtain training data, and the preprocessing comprises sample labeling, principal component analysis and data augmentation processing; the model training module is used for constructing a semantic segmentation model, inputting the training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; the image recognition module is used for completing image recognition after image segmentation is carried out on a preset area of a product to be detected based on the trained hierarchical segmentation model, and outputting a defect detection result of the product to be detected.
As a preferred embodiment of the present invention, wherein: the defect problem analysis module calls the edge calculation generation data of the production detection end to perform defect problem analysis, and the defect problem analysis module comprises the following steps:
an error calculation unit for analyzing and calculating a square error cost function;
a convolution layer calculation unit for analyzing and calculating a convolution layer output;
a sensitivity calculation unit for analyzing and calculating sensitivity;
the partial guide calculation unit is used for analyzing and calculating partial guides of the square error cost function to the offset and calculating partial guides of the square error cost function to the convolution layer;
and the weight updating unit is used for updating the weight of the sampling layer in the convolution network.
As a preferred embodiment of the present invention, wherein: the defect problem association module comprises a classification and sorting unit, a judging unit, an association optimizing unit, a retrieval matching unit and an output feedback unit;
the classification and sorting unit divides the edge calculation data into words to obtain keywords or keywords, and the corresponding sentences are divided into words by a field controller and then sorted according to weight scores;
the judging unit judges the number of the production detection ends, judges a single production detection end and a plurality of production detection ends, and obtains a corresponding defect judgment production detection end system through judgment, wherein the defect judgment production detection end system corresponds to a defect detection problem library table;
the association optimizing unit calculates data keywords or key words according to edges, and optimizes, compares and searches a corresponding defect detection problem library table or a preset defect detection problem system library table;
the searching and matching unit outputs defect detection edge calculation data of the production detection end or/and defect detection edge calculation data corresponding to the production detection end which are related to each other according to the production detection end system, historical data in the production detection end system and defects of the site situation, and results data are formed;
and the output feedback unit outputs the result data after the correlation optimization to the corresponding production detection end.
As a preferred embodiment of the present invention, wherein: the production detection end further comprises an illumination light source arranged on the robot.
As a preferred embodiment of the present invention, wherein: the production detection end further comprises an upper computer connected with the production detection line, the multispectral camera, the robot and the illumination light source, and a control system of the production detection line, the multispectral camera, the robot and the illumination light source is arranged in the upper computer;
the product defect detection deep learning system is arranged in the upper computer and is connected with the production detection line, the multispectral camera, the robot and the control system of the illumination light source.
As a preferred embodiment of the present invention, wherein: the robot adopts a three-axis or five-axis robot; the production detection line adopts a belt transmission mechanism or a round roller transmission mechanism; the illumination light sources are arranged around the multispectral camera in a surrounding mode.
As a preferred embodiment of the present invention, wherein: the production detection line further comprises a turnover mechanism for controlling the product to be detected to turn over on the belt conveying mechanism or the round roller conveying mechanism.
As a preferred scheme of the invention, the method of the product defect intelligent detection system based on multispectral imaging comprises the following steps:
the method comprises the steps that a product to be detected is arranged on a production detection line, the production detection line is used for spatially moving the product to be detected to a detection station and correspondingly adjusting, a multispectral camera is controlled by a robot to shoot the product to be detected on the production detection line at multiple angles, multispectral image data are formed and then transmitted to a product defect detection deep learning system, and edge calculation is carried out, namely characteristic wave bands are screened based on the multispectral image data; preprocessing the multispectral image data to obtain training data; constructing a semantic segmentation model, inputting the training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; performing image segmentation on a preset area of a product to be detected based on the trained hierarchical segmentation model, and completing image recognition to output a defect detection result of the product to be detected;
meanwhile, the cloud processing platform comprises a defect problem analysis module and a defect problem association module, and is used for calling edge calculation data of the production detection end to perform defect problem analysis, and meanwhile, generating feedback data after associated independent learning to transmit the feedback data to the production detection end, so that the production detection end can perform corresponding control of the intelligent product defect detection equipment and analysis optimization of a product defect detection deep learning system.
The defect problem analysis module calls the edge calculation generated data of the production detection end to analyze the defect problem, and the defect problem analysis module specifically comprises the following steps:
step one, the square error cost function E is calculated by the analysis of the formula (1), as follows:
wherein N represents the number of samples; c represents the dimension of the label, i.e. the samples are classified into class c;represents the nth sample t n The kth dimension of the tag; />A kth dimension representing an nth sample network output;
step two, analyzing and calculating the output of the kth layer of the convolution layer by the formula (2)The following are provided:
wherein i represents the ith eigenvalue, f is the activation function, M j Representing the selected combination of input characteristic values,represents the last layer output of the kth layer, is->B is a convolution kernel for connection between the input ith eigenvalue and the output jth eigenvalue j The bias corresponding to the j-th characteristic value;
step three, calculating sensitivity by analysis of formula (3)The following are provided:
wherein ,uk Updating coefficients for the partial derivative weights;
step four, calculating the bias b of the square error cost function E by the formula (4) j And summing the partial derivatives of the square error cost function E to the convolutional layer as follows:
wherein, (u, v) is the position of the element in the sensitivity matrix;
step five, updating the weight of a sampling layer in the convolution network through the step (5)The following are provided:
where down represents the downsampling layer,weights, af, representing the last sampling layer 1 To activate the function +.>Is an additive bias.
The invention has the beneficial effects that: based on the above, the invention combines the edge calculation and the deep learning of the production detection end, so that the deep learning can directly calculate at the edge equipment in the network-free environment, and brings convenience to calculation, analysis and storage of various data collected in real time at the edge equipment, wherein the embedded equipment calculation directly performed at the production detection end can avoid the time delay problem caused by network transmission, the product defect detection result can be packaged and sent to the corresponding production detection line of the cloud processing platform in real time, the data after being combined are combined with each other in a single production detection line data tree form and a plurality of system production detection line data tree forms, the self-learning training is performed on the data after being combined with each other in a hierarchical manner, the possible problems of the detected product in the current production detection line are qualitatively detected, the rapid positioning and standardized analysis result model of the product defect between the single production detection line and the plurality of production detection lines are realized, the defect detection output is performed more rapidly, rapidly and accurately at the follow-up, and simultaneously, the feedback is sent to the corresponding production detection end, the special detection after the optimization is performed by combining with the production detection line, a multispectral camera, a robot and other equipment on the production detection end, and the like, so that the self-learning, the training and the quality reminding measures are further carried out by corresponding to alert personnel.
In summary, the defects of the processed product to be detected can be automatically identified through multispectral screening, machine learning and edge calculation, and meanwhile, optimized special detection is carried out through a cloud after the edge calculation, so that data of autonomous learning and training are further optimized and fed back, the accuracy of product defect detection and the comprehensiveness of other related data application are improved, and the intelligent development of the product defects is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a schematic diagram of a modular structure of an intelligent detection system for product defects based on multispectral imaging in an embodiment of the invention;
FIG. 2 is a schematic diagram of a defect analysis module according to an embodiment of the present invention;
fig. 3 is a schematic block diagram of a defect problem association module according to an embodiment of the present invention.
10, producing a detection end; 20. the cloud processing platform; 101. a multispectral camera; 102. a robot; 103. a production detection line; 104. a product defect detection deep learning system; 201. an error calculation unit; 202. a convolution layer calculation unit; 203. a sensitivity calculation unit; 204. a partial conductance calculation unit; 205. a weight updating unit; 210. a sorting unit; 220. a determination unit; 230. an association optimizing unit; 240. a search matching unit; 250. and outputting a feedback unit.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which are obtained by a person skilled in the art based on the described embodiments of the invention, fall within the scope of protection of the invention.
Referring to fig. 1, 2 and 3, an embodiment of the present invention provides a product defect intelligent detection system based on multispectral imaging. The production detection end 10 for intelligent detection of product defects and edge calculation is included, and the cloud processing platform 20 is included;
the production detection end 10 comprises a production detection line 103 for enabling a product to be detected to displace, a multispectral camera 101 for shooting the product to be detected, a robot 102 for controlling the space position of the multispectral camera 101, and a product defect detection deep learning system 104 for forming multispectral image data according to the shooting of the multispectral camera 101;
the cloud processing platform 20 comprises a defect problem analysis module and a defect problem association module, and is used for calling edge calculation of the production detection end 10 to generate edge calculation data, then performing defect problem analysis, and simultaneously performing association autonomous learning to generate feedback data, and transmitting the feedback data to the production detection end 10, wherein the feedback data are used for controlling corresponding intelligent product defect detection equipment and performing analysis optimization of the product defect detection deep learning system 104 by the production detection end 10;
the product defect detection deep learning system 104 comprises a data acquisition module, a data processing module, a model training module and an image recognition module; the data acquisition module is used for screening characteristic wave bands based on multispectral image data; the data processing module is used for preprocessing the multispectral image data to obtain training data, wherein the preprocessing comprises sample labeling, principal component analysis and data augmentation processing; the model training module is used for constructing a semantic segmentation model, inputting training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; the image recognition module is used for completing image recognition after image segmentation is carried out on a preset area of the product to be detected based on the trained hierarchical segmentation model, and outputting a defect detection result of the product to be detected.
Based on the above, the invention combines the edge calculation and the deep learning of the production detection end 10, so that the deep learning can directly calculate at the edge equipment in the network-free environment, and brings convenience to the calculation, analysis and storage of various data collected in real time at the edge equipment, wherein the embedded equipment calculation directly performed at the production detection end 10 can avoid the time delay problem caused by network transmission, package and send the product defect detection result to the corresponding production detection line 103 of the cloud processing platform 20 in real time, combine the data tree form of the single production detection line 103 and the data tree form of the multiple system production detection lines 103, perform self-learning training by combining the data after being correlated with each other, hierarchically characterize the product defect, qualify the possible problem of the detected product in the current production detection line 103, realize the rapid positioning, standardized analysis result model of the product defect between the single production detection line 103 and the multiple production detection lines 103, and further perform more rapid, rapider and more accurate follow-up defect detection output, simultaneously feed back to the corresponding production detection end 10, and combine the production detection lines 103, multiple cameras 101, robots 102 and other equipment to perform optimization and special detection after optimizing, and further perform special training, and further prompt the quality and quality measurement of the corresponding training.
Further illustratively, the defect problem analysis module calls the edge calculation generation data of the production detection end 10 to perform defect problem analysis, and includes: an error calculation unit 201 for analytically calculating a square error cost function; a convolution layer calculation unit 202 for analyzing and calculating a convolution layer output; a sensitivity calculation unit 203 for analyzing the calculated sensitivity; a bias guide calculation unit 204 for analyzing and calculating bias guide of the square error cost function to bias, and calculating bias guide of the square error cost function to convolution layer; a weight updating unit 205 for updating the weights of the sampling layers in the convolutional network.
The defect problem association module of the present embodiment includes a sorting unit 210, a determination unit 220, an association optimizing unit 230, a search matching unit 240, and an output feedback unit 250; the classification and sorting unit 210 divides the edge calculation data into words to obtain keywords or keywords, and the corresponding sentences are classified into words by the field controller and then sorted according to the weight scores; the judging unit 220 judges the number of the production detection ends 10, judges the single production detection end 10 and the plurality of production detection ends 10, and obtains a corresponding defect judgment production detection end system through judgment, wherein the defect judgment production detection end system corresponds to the defect detection problem library table; the association optimizing unit 230 calculates data keywords or key words according to edges, and optimizes and compares the corresponding defect detection problem library table or a preset defect detection problem system library table; the retrieval matching unit 240 outputs defect detection edge calculation data of the production detection end 10 or/and defect detection edge calculation data corresponding to the production detection end 10 which are associated with each other according to the production detection end system, history data in the production detection end system and defects of the site situation, and forms result data; the output feedback unit 250 outputs the correlation-optimized result data to the corresponding production inspection terminal 10.
The production test end 10 also preferably includes an illumination source mounted on the robot 102. In addition, the production detection end 10 further comprises an upper computer connected with the production detection line 103, the multispectral camera 101, the robot 102 and the illumination light source, and a control system of the production detection line 103, the multispectral camera 101, the robot 102 and the illumination light source is arranged in the upper computer; the product defect detection deep learning system 104 is arranged in the upper computer and is connected with the production detection line 103, the multispectral camera 101, the robot 102 and the control system of the illumination light source.
The robot 102 in the present embodiment adopts a three-axis or five-axis robot; the production detection line 103 adopts a belt transmission mechanism or a round roller transmission mechanism; at least two illumination sources are circumferentially arranged around the multispectral camera 101. The production line 103 further includes a turnover mechanism for controlling the turnover of the product to be inspected on the belt conveyor or the round roller conveyor.
Based on the above, the present embodiment further provides a method for a product defect intelligent detection system based on multispectral imaging, which includes the following steps:
the product to be detected is arranged on a production detection line 103, the production detection line 103 is used for spatially moving the product to be detected to a detection station and correspondingly adjusting, a multispectral camera 101 is controlled by a robot 102 to shoot the product to be detected on the production detection line 103 at multiple angles, multispectral image data are formed and then transmitted to a product defect detection deep learning system 104, and edge calculation is carried out, namely characteristic wave bands are screened based on the multispectral image data; preprocessing multispectral image data to obtain training data; constructing a semantic segmentation model, inputting training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; performing image segmentation on a preset area of a product to be detected based on the trained hierarchical segmentation model, completing image recognition, and outputting a defect detection result of the product to be detected; meanwhile, the cloud processing platform 20 includes a defect problem analysis module and a defect problem association module, which are used for calling edge calculation data of the production detection end 10 to perform defect problem analysis, and simultaneously performing association autonomous learning to generate feedback data to be transmitted to the production detection end 10, so that the production detection end 10 performs control of corresponding intelligent product defect detection equipment and analysis optimization of the product defect detection deep learning system 104.
The defect problem analysis module calls the edge calculation generation data of the production detection end 10 to analyze the defect problem, and the defect problem analysis module specifically comprises the following steps:
step one, the square error cost function E is calculated by the analysis of the formula (1), as follows:
wherein N represents the number of samples; c represents the dimension of the label, i.e. the samples are classified into class c;represents the nth sample t n The kth dimension of the tag; />A kth dimension representing an nth sample network output;
step two, analyzing and calculating the output of the kth layer of the convolution layer by the formula (2)The following are provided:
wherein i represents the ith eigenvalue, f is the activation function, M j Representing the selected combination of input characteristic values,represents the last layer output of the kth layer, is->B is a convolution kernel for connection between the input ith eigenvalue and the output jth eigenvalue j The bias corresponding to the j-th characteristic value;
step three, calculating sensitivity by analysis of formula (3)The following are provided:
wherein ,uk Updating coefficients for the partial derivative weights;
step four, calculating the bias b of the square error cost function E by the formula (4) j And summing the partial derivatives of the square error cost function E to the convolutional layer as follows:
wherein, (u, v) is the position of the element in the sensitivity matrix;
step five, updating the weight of a sampling layer in the convolution network through the step (5)The following are provided:
where down represents the downsampling layer,weights, af, representing the last sampling layer 1 To activate the function +.>Is an additive bias.
Based on the above, compared with the traditional machine learning model, the multi-spectral detection technical scheme of the invention adopts a convolutional neural network model which uses a multi-layer architecture, namely multi-dimensional autonomous capturing features from different angles to form multi-layer abstractions, then inputting the obtained new variables into a convolutional layer and a sampling layer for training, and obtaining the effect of simulating and predicting the defects of the product according to the discrimination values output by a full-connection layer.
In summary, defects of a processed product to be detected can be automatically identified through multispectral screening, machine learning and edge calculation, and meanwhile, optimized special detection is carried out through a cloud after the edge calculation, so that data of autonomous learning and training are further optimized and fed back, accuracy of product defect detection and comprehensiveness of other related data application are improved, and intelligent development of product defects is improved.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (6)
1. The method of the product defect intelligent detection system based on multispectral imaging is characterized in that the product defect intelligent detection system based on multispectral imaging comprises a production detection end (10) for product defect intelligent detection and edge calculation, and a cloud processing platform (20);
the production detection end (10) comprises a production detection line (103) for enabling a product to be detected to displace, a multispectral camera (101) for shooting the product to be detected, a robot (102) for controlling the spatial position of the multispectral camera (101), and a product defect detection deep learning system (104) for forming multispectral image data according to the multispectral image data shot by the multispectral camera (101);
the cloud processing platform (20) comprises a defect problem analysis module and a defect problem association module, and is used for calling edge calculation of the production detection end (10) to generate edge calculation data, then carrying out defect problem analysis, and simultaneously carrying out association autonomous learning to generate feedback data and transmitting the feedback data to the production detection end (10), and is used for controlling corresponding intelligent product defect detection equipment and carrying out analysis optimization of a product defect detection deep learning system (104) by the production detection end (10); the defect problem association module comprises a classification ordering unit (210), a judging unit (220), an association optimizing unit (230), a retrieval matching unit (240) and an output feedback unit (250);
the classification and sorting unit (210) divides the edge calculation data into words to obtain keywords or keywords, and the corresponding sentences are divided into words by a field controller and then sorted according to weight scores;
the judging unit (220) judges the number of the production detection ends (10), judges the single production detection end (10) and the plurality of production detection ends (10), and obtains a corresponding defect judgment production detection end system through judgment, wherein the defect judgment production detection end system is correspondingly matched with a defect detection problem library table;
the association optimizing unit (230) calculates data keywords or key words according to edges, and optimizes and compares the corresponding defect detection problem library table or a preset defect detection problem system library table;
the retrieval matching unit (240) judges historical data and site situation defects generated by the production detection end system and the production detection end (10) according to the defects, and outputs defect detection edge calculation data of the production detection end (10) or/and defect detection edge calculation data corresponding to the production detection end (10) which are associated with each other to form result data;
the output feedback unit (250) outputs the correlation optimized result data to the corresponding production detection end (10);
the product defect detection deep learning system (104) comprises a data acquisition module, a data processing module, a model training module and an image recognition module; the data acquisition module is used for screening characteristic wave bands based on multispectral image data; the data processing module is used for preprocessing the multispectral image data to obtain training data, and the preprocessing comprises sample labeling, principal component analysis and data augmentation processing; the model training module is used for constructing a semantic segmentation model, inputting the training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; the image recognition module is used for completing image recognition after image segmentation is carried out on a preset area of a product to be detected based on the trained hierarchical segmentation model, and outputting a defect detection result of the product to be detected;
the method comprises the following steps:
the method comprises the steps that a product to be detected is arranged on a production detection line (103), the production detection line (103) is used for spatially moving the product to be detected to a detection station and correspondingly adjusting, a multispectral camera (101) is controlled by a robot (102) to shoot the product to be detected on the production detection line (103) at multiple angles, multispectral image data are formed and then transmitted to a product defect detection deep learning system (104), and edge calculation is carried out, namely characteristic wave bands are screened based on the multispectral image data; preprocessing the multispectral image data to obtain training data; constructing a semantic segmentation model, inputting the training data into the semantic segmentation model for training, and adjusting segmentation parameters of the semantic segmentation model; performing image segmentation on a preset area of a product to be detected based on the trained hierarchical segmentation model, completing image recognition, and outputting a defect detection result of the product to be detected;
the defect problem analysis module calls the edge calculation generated data of the production detection end (10) to analyze the defect problem, and the defect problem analysis module specifically comprises the following steps:
step one, calculating a square error cost function by analysis of the formula (1)EThe following are provided: wherein ,Nrepresenting the number of samples;crepresenting the dimensions of the tag, i.e. the samples are divided intocClass; />Represent the firstnSample tn tag ofkDimension; />Represent the firstnThe first sample network outputkDimension;
step two, dividing by the formula (2)Analysis of output of k-th layer of convolution layerThe following are provided: /> wherein ,irepresent the firstiThe seed characteristic value is used for generating a seed characteristic value,fto activate the function +.>Representing selected combinations of input features, +.>Represents the last layer output of the kth layer, is->Is the first to inputiSeed characteristic value and output of the firstjThe convolution kernel between the seed eigenvalues for the connection,is the firstjBias corresponding to the seed characteristic value;
step three, calculating sensitivity by analysis of formula (3)The following are provided: /> wherein ,/>Updating coefficients for the partial derivative weights;
step four, calculating a square error cost function through a formula (4)ETo biasIs used for calculating square error cost function by summationEThe bias for the convolutional layer is as follows: /> wherein ,/>Is the position of the element in the sensitivity matrix;
step five, updating the weight of a sampling layer in the convolution network through the step (5)The following are provided: /> wherein ,downrepresenting the downsampling layer, ">Weight representing last sample layer, +.>To activate the function +.>Is an additive bias.
2. The method according to claim 1, wherein the defect problem analysis module calls the edge calculation generation data of the production inspection end (10) to perform defect problem analysis, and includes:
an error calculation unit (201) for analytically calculating a square error cost function;
a convolution layer calculation unit (202) for analyzing and calculating a convolution layer output;
a sensitivity calculation unit (203) for analyzing and calculating sensitivity;
a bias guide calculation unit (204) for analyzing and calculating the bias guide of the square error cost function to the bias, and calculating the bias guide of the square error cost function to the convolution layer;
a weight updating unit (205) for updating weights of sampling layers in the convolutional network.
3. The method of claim 1, wherein the production test end (10) further comprises an illumination source mounted on the robot (102).
4. A method according to claim 3, wherein the production detection end (10) further comprises a host computer connected with the production detection line (103), the multispectral camera (101), the robot (102) and the illumination light source, and a control system of the production detection line (103), the multispectral camera (101), the robot (102) and the illumination light source is arranged in the host computer;
the product defect detection deep learning system (104) is arranged in the upper computer and is connected with the production detection line (103), the multispectral camera (101), the robot (102) and the control system of the illumination light source.
5. The method according to claim 4, wherein the robot (102) is a three-axis or five-axis robot; the production detection line (103) adopts a belt transmission mechanism or a round roller transmission mechanism; at least two illumination light sources are arranged around the multispectral camera (101) in a surrounding mode.
6. The method according to claim 5, wherein the production line (103) further comprises a turning mechanism for controlling the turning of the product to be tested on the belt conveyor or on the round roller conveyor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211289620.2A CN115508366B (en) | 2022-10-20 | 2022-10-20 | Product defect intelligent detection system and method based on multispectral imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211289620.2A CN115508366B (en) | 2022-10-20 | 2022-10-20 | Product defect intelligent detection system and method based on multispectral imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115508366A CN115508366A (en) | 2022-12-23 |
| CN115508366B true CN115508366B (en) | 2023-09-22 |
Family
ID=84511153
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211289620.2A Active CN115508366B (en) | 2022-10-20 | 2022-10-20 | Product defect intelligent detection system and method based on multispectral imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115508366B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115950666B (en) * | 2023-03-15 | 2023-05-30 | 石家庄市惠源淀粉有限公司 | Intelligent fault detection method for starch glucose production equipment |
| CN117058541B (en) * | 2023-08-03 | 2024-02-13 | 国网吉林省电力有限公司通化供电公司 | Insulator hyperspectral data acquisition system and method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006352173A (en) * | 2001-04-10 | 2006-12-28 | Hitachi Ltd | Method of analyzing defect data and device thereof |
| CN110441312A (en) * | 2019-07-30 | 2019-11-12 | 上海深视信息科技有限公司 | A kind of surface defects of products detection system based on multispectral imaging |
| CN111431986A (en) * | 2020-03-18 | 2020-07-17 | 宁波智诚祥科技发展有限公司 | Industrial intelligent quality inspection system based on 5G and AI cloud edge cooperation |
| CN112069069A (en) * | 2020-09-03 | 2020-12-11 | 平安信托有限责任公司 | Defect automatic positioning analysis method, device and readable storage medium |
| CN114677671A (en) * | 2022-02-18 | 2022-06-28 | 深圳大学 | Automatic identifying method for old ribs of preserved szechuan pickle based on multispectral image and deep learning |
-
2022
- 2022-10-20 CN CN202211289620.2A patent/CN115508366B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006352173A (en) * | 2001-04-10 | 2006-12-28 | Hitachi Ltd | Method of analyzing defect data and device thereof |
| CN110441312A (en) * | 2019-07-30 | 2019-11-12 | 上海深视信息科技有限公司 | A kind of surface defects of products detection system based on multispectral imaging |
| CN111431986A (en) * | 2020-03-18 | 2020-07-17 | 宁波智诚祥科技发展有限公司 | Industrial intelligent quality inspection system based on 5G and AI cloud edge cooperation |
| CN112069069A (en) * | 2020-09-03 | 2020-12-11 | 平安信托有限责任公司 | Defect automatic positioning analysis method, device and readable storage medium |
| CN114677671A (en) * | 2022-02-18 | 2022-06-28 | 深圳大学 | Automatic identifying method for old ribs of preserved szechuan pickle based on multispectral image and deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 李梦园.深度学习算法在表面缺陷识别中的应用研究.《中国优秀硕士学位论文全文数据库 信息科技辑》.2016,(第04期),第5-8、19-32页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115508366A (en) | 2022-12-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115508366B (en) | Product defect intelligent detection system and method based on multispectral imaging | |
| Li et al. | Fast detection and location of longan fruits using UAV images | |
| CN106383095B (en) | Device and method for detecting total bacteria on surface of cooled mutton | |
| CA2903041C (en) | Network of intelligent machines | |
| Lee et al. | Potato detection and segmentation based on mask R-CNN | |
| Fahad et al. | Fruits and Vegetables Freshness Categorization Using Deep Learning. | |
| Sarkar et al. | Comparative analysis of statistical and supervised learning models for freshness assessment of oyster mushrooms | |
| Arango et al. | Quality control of apples by means of convolutional neural networks-Comparison of bruise detection by color images and near-infrared images | |
| Yin et al. | Non-destructive detection of foreign contaminants in toast bread with near infrared spectroscopy and computer vision techniques | |
| Gnanavel et al. | Quality detection of fresh fruits and vegetables to improve horticulture and agro-industries | |
| Guo et al. | Bruise detection and classification of strawberries based on thermal images | |
| Mohi‐Alden et al. | Design and evaluation of an intelligent sorting system for bell pepper using deep convolutional neural networks | |
| CN110991469B (en) | Fruit soluble solid online detection method and system | |
| Aljaafreh et al. | A Real-Time Olive Fruit Detection for Harvesting Robot Based on YOLO Algorithms | |
| Dewantara et al. | Recognition of Food Material and Measurement of Quality using YOLO and WLD-SVM | |
| Lopez-Juarez et al. | A hybrid non-invasive method for internal/external quality assessment of potatoes | |
| Bastidas-Alva et al. | Recognition and classification system for trinitario cocoa fruits according to their ripening stage based on the Yolo v5 algorithm | |
| Zupan | Can an instrument learn from experiments done by itself? | |
| Sehgal et al. | Auto-annotation of tomato images based on ripeness and firmness classification for multimodal retrieval | |
| Ramesh et al. | Identification of Timber Defects Using Convolution Neural Network | |
| CN110077622A (en) | A kind of digital intelligent nondestructive detection system and its detection method | |
| Sean et al. | Development of Computer Vision System for Fruits | |
| Mendoza et al. | Application of Machine Learning for Insect Monitoring in Grain Facilities. AI 2023, 4, 348–360 | |
| Buyukarikan et al. | Classification of physiological disorders in apples using deep convolutional neural network under different lighting conditions | |
| Custodio et al. | Performance analysis of deep learning architectures in classifying calamansi (citrofortunella microcarpa) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |