CN112082999A - Industrial product defect detection method and industrial intelligent camera - Google Patents
Industrial product defect detection method and industrial intelligent camera Download PDFInfo
- Publication number
- CN112082999A CN112082999A CN202010730238.5A CN202010730238A CN112082999A CN 112082999 A CN112082999 A CN 112082999A CN 202010730238 A CN202010730238 A CN 202010730238A CN 112082999 A CN112082999 A CN 112082999A
- Authority
- CN
- China
- Prior art keywords
- defect detection
- industrial
- deep learning
- image
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 180
- 238000001514 detection method Methods 0.000 title claims abstract description 132
- 238000013135 deep learning Methods 0.000 claims abstract description 67
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000003384 imaging method Methods 0.000 claims abstract description 14
- 238000005286 illumination Methods 0.000 claims abstract description 9
- 238000013528 artificial neural network Methods 0.000 claims description 26
- 238000012549 training Methods 0.000 claims description 25
- 238000004891 communication Methods 0.000 claims description 19
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000013210 evaluation model Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000013526 transfer learning Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000010972 statistical evaluation Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Abstract
The invention relates to an industrial product defect detection method and an industrial intelligent camera. A method for detecting industrial product defects comprises the following steps: s1, opening the industrial intelligent camera, and adjusting the illumination parameters of the controllable light source by the on-board camera chip according to imaging requirements; s2, shooting industrial products by using the low-distortion lens, and transmitting shot product images to the embedded processor through the on-board camera chip; and S3, the embedded processor performs defect identification on the product image by using the deep learning defect detection model and outputs an inference result to the outside. The industrial intelligent camera carries a deep learning defect detection model, can efficiently identify defects with complex structure and complex production environment, can be loaded and updated through the server to adapt to deep learning defect detection models of different workpieces and environments, and is used for carrying out image acquisition on industrial products, so that the real-time detection of product defects can be realized.
Description
Technical Field
The invention relates to industrial product detection, in particular to an industrial product defect detection method and an industrial intelligent camera.
Background
At present, a machine vision system in the field of industrial defect detection is mainly divided into two parts: the machine vision detection system comprises an image acquisition unit consisting of a traditional camera, a lens, a light source, a camera fixing and moving mechanism, and an image processing unit consisting of a PC host, an image acquisition card and the like. This machine vision has high extensibility and plasticity. The customized hardware and the customized software development can be performed according to different product defects. However, the traditional detection system has the problems of larger hardware structure volume, more complex deployment and loading and unloading, more professional technicians required by system deployment and the like.
The industrial intelligent camera system applies the latest DSP, FPGA and a microminiature machine vision system with high integration of a large-capacity storage technology, integrates an image acquisition unit, an image processing unit, image processing software and a communication function into one camera, and provides a machine vision solution which is easy to deploy and realize. However, the existing processing software of the industrial smart camera generally only can provide functions of simple edge extraction, Blob, gray histogram, simple positioning and searching and the like. These functions cannot be adapted to the defect location and judgment of the defect surface of a complex workpiece, and often different products need to be developed in a customized manner. And the traditional visual detection algorithm cannot effectively judge defects with complex shapes and unobvious edge features.
The disadvantages of the prior art include:
(1) the image processing software uses already cured defect detection methods such as: the method comprises the following steps of size measurement, edge extraction, simple positioning search and the like, wherein the methods can not accurately and effectively position the defect detection with complex defect structure and complex production environment;
(2) different products and defects cannot be specially processed;
(3) the detected workpiece cannot be memorized and learned, and once the detection model is fixed, the precision is relatively fixed.
Patent No. ZL 201410524785.2 discloses an industrial intelligent camera, which includes an imaging module, a main control module, a transmission module, a power supply module and a storage module, wherein the imaging module includes a CMOS image sensor and a peripheral circuit, and the main control module is an FPGA chip and includes an image acquisition control module, an image preprocessing module, an image advanced processing module, a transmission control module and a storage control module. The industrial intelligent camera provided by the invention adopts the FPGA chip to realize all image processing functions, avoids the complex interactive design of FPGA and DSP, fully exerts the advantages of high processing speed, stable performance and the like of the FPGA for realizing the algorithm by a hardware circuit, has high integration level, small integral size, low weight and low power consumption, and is particularly suitable for application occasions with higher requirements on the size and the image processing speed of the camera.
Patent No. ZL 201711025035.0 discloses a defect detection system for an industrial smart camera, which includes a camera, a lens connected to the camera, and a plurality of light sources, wherein the light sources face the front of a product at different angles, a main control unit of the camera controls the lighting sequence of the light sources, and an image acquisition unit of the camera is started to take a picture to generate a plurality of pictures; the camera comprises an image processing unit, wherein the image processing unit comprises a plurality of fixed modules packaged by an algorithm and used for processing images and synthesizing and detecting a plurality of images. The method has the advantages that the images shot at a plurality of angles are synthesized, and then the product defects are detected on the synthesized images, so that the problem that the product defects cannot be completely detected by the traditional single-angle shot images is solved.
However, the above problems still cannot be effectively solved, and thus the existing industrial intelligent camera detection field is insufficient and needs to be improved and enhanced.
Disclosure of Invention
In view of the defects of the prior art, the invention aims to provide an industrial product defect detection method and an industrial intelligent camera, which can respectively detect different industrial products and different defects and quickly identify a defect result.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for detecting industrial product defects comprises the following steps:
s1, opening the industrial intelligent camera, and adjusting the illumination parameters of the controllable light source by the on-board camera chip according to imaging requirements;
s2, shooting industrial products by using the low-distortion lens, and transmitting shot product images to the embedded processor through the on-board camera chip;
and S3, the embedded processor performs defect identification on the product image by using the deep learning defect detection model, outputs an inference result to the outside, and uploads the inference result to a server.
Preferably, before step S1, the method for detecting defects of industrial products further includes the steps of:
s0, training the deep learning defect detection model, and loading the trained deep learning defect detection model into the embedded processor.
Preferably, in the industrial product defect detection method, the training process of the deep learning defect detection model is as follows:
s01, constructing a deep learning defect detection model by using a deep neural network;
s02, acquiring a plurality of defect images, calibrating defect positions of part of the defect images to be used as training images, and using the other part of the defect images as test images;
s03, training the deep neural network by using all the training images, testing the trained deep neural network by using the test image, judging whether the requirements of an evaluation model are met, and if so, judging that the deep neural network is well trained; if not, step S02 is executed.
Preferably, the industrial product defect detecting method, in step S0, further includes:
and after the camera working system judges that the deep learning defect detection model is detected for a preset number of times in an accumulated mode, the camera working system requests a server to update the deep learning defect detection model.
Preferably, in the method for detecting defects of industrial products, in step S1, the imaging requirement is determined by an image peak signal-to-noise ratio algorithm; the image peak signal-to-noise ratio algorithm is as follows:
given a clean image I and a noisy image K of size m × n, the mean square error MSE is calculated as:
further, the calculation formula of the peak signal-to-noise ratio PSNR is:
wherein, MAXIFor the maximum pixel value possible for the picture, 255 if each pixel is represented by an 8-bit binary; in general, MAX is the value of a pixel if it is represented by a B-bit binaryI＝2B-1。
Preferably, the defect detection method for industrial products is implemented by determining the imaging requirements of the color image through three determination methods, which are respectively:
the first judgment method comprises the following steps: respectively calculating peak signal-to-noise ratios (PSNR) of the RGB three channels of the color image, and then averaging;
and a second judgment method: respectively calculating Mean Square Error (MSE) of RGB three channels of the color image, and then taking an average value;
the third judgment method comprises the following steps: the color image is converted to YCbCr format and then only the peak signal-to-noise ratio PSNR of the Y component is calculated.
An industrial intelligent camera using the industrial product defect detection method comprises an image acquisition system and an image defect identification system;
the image acquisition system comprises an on-board camera chip, a low-distortion lens and a controllable light source; the low distortion lens and the controllable light source are respectively connected with the on-board camera chip;
the image defect identification system comprises an embedded processor and a storage chip; the on-board camera chip is connected with the embedded processor; the embedded processor is internally provided with a deep learning defect detection model and a camera working system; the embedded processor is connected with the server.
Preferably, in the industrial intelligent camera, the deep learning defect detection model is a deep neural network, a RestNet network is added to the deep neural network, and an identity mapping algorithm is added among networks of each layer.
Preferably, in the industrial smart camera, the low distortion lens is a replaceable lens.
Preferably, in the industrial smart camera, a connection mode between the embedded processor and the server is a communication mode, and the communication mode includes: 4G/3G communication and Internet communication.
Compared with the prior art, the industrial product defect detection method and the industrial intelligent camera provided by the invention have the advantages that the industrial intelligent camera is provided with the deep learning defect detection model, so that defects with complex structures and complex production environments can be efficiently identified, meanwhile, the deep learning defect detection model which can adapt to different workpieces and environments can be loaded and updated through the server, a user only needs to set the industrial intelligent camera according to the defect detection method provided by the invention, and the industrial intelligent camera is used for carrying out image acquisition on an industrial product, so that the real-time detection of the product defects can be realized; meanwhile, the self-learning of the identification of the defects in the detection process can be improved by self-learning.
Drawings
FIG. 1 is a block diagram of the construction of an industrial smart camera of the present invention;
FIG. 2 is a flow chart of a defect detection method provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1 and 2 together, the present invention provides an industrial intelligent camera for defect detection, which includes an image acquisition system and an image defect recognition system;
the image acquisition system comprises an on-board camera chip 11, a low-distortion lens 12 and a controllable light source 13; the low-distortion lens 12 and the controllable light source 13 are respectively connected with the on-board camera chip 11;
the image defect identification system comprises an embedded processor 21 and a memory chip 22; the on-board camera chip 11 is connected with the embedded processor 21; the embedded processor 21 is internally provided with a deep learning defect detection model and a camera working system; the embedded processor 21 is connected to a server. The embedded processor 21 is preferably a high-computation processor, preferably a CPU, and the specific model is not limited, and the CPU of an ordinary computer can be implemented.
Specifically, the intelligent industrial intelligent camera for defect detection is an embedded intelligent camera based on an embedded machine vision technology and a deep learning technology. The industrial intelligent camera can work independently under a general condition, namely, the image acquisition system and the image defect identification system in the camera are used, so that the defect detection function of the industrial intelligent camera can be realized. The industrial intelligent camera consists of an image acquisition system and an image defect identification system, wherein the depth learning defect detection model in the image defect identification system is loaded by the server, before the depth learning defect detection model is loaded to the industrial intelligent camera, the deep learning defect detection model is trained, of course, the training is performed according to a predetermined defect type, different defect types have great correlation with industrial products, and product defects of different industrial products are different in general, so that, the industrial intelligent camera needs to carry out customized training on the deep learning defect detection model in the industrial intelligent camera before working, then the industrial intelligent camera is loaded into the industrial intelligent camera, and the industrial intelligent camera can automatically start working; meanwhile, in the working process, if new product defects appear in the industrial products to be detected, the deep learning defect detection model can be updated. It should be noted that the deep learning defect detection model is mainly used for detecting whether the industrial product to be detected has defects or not and analyzing the result; the camera working system is mainly used for realizing other functions of defect detection except for cameras, for example, the image acquisition system and the image defect recognition system in the industrial intelligent camera coordinate, for example, pictures transmitted by an on-board camera chip 11 are transmitted to the deep learning defect detection module, analysis results output by the deep learning defect detection module are received, double-system safe working is realized, the industrial intelligent camera is ensured not to easily generate faults, and the camera working system is also mainly used for communication connection work between the industrial intelligent camera and the server; preferably, the memory chip 22 is used for storing local system data, analyzing detection data, and the like.
Preferably, in this embodiment, the deep learning defect detection model in the industrial smart camera is preset with a defect detection model that can detect that multiple products have been subjected to multiple defect detections for each product, and before the industrial smart camera is used, different functional models need to be selected for different industrial products, where the selection may be manual, or the industrial smart camera may automatically identify which industrial product is, and then automatically select which defect detection functional model is used; meanwhile, the defect learning detection model also performs training of detection precision in the product detection process, so that the industrial intelligent camera is more frequently used and the detection precision is higher.
The on-board camera chip 11 in the image acquisition system is a high-resolution on-board camera chip 11, and is not particularly limited as long as a high-resolution digital image can be acquired according to the low-distortion lens 12; the low distortion lens 12 is a portable replaceable lens, and can replace lenses of different models according to requirements; the controllable light source 13 is a programmable control light source, parameters of the control light source are generally brightness and angle of the light source, the on-board camera chip 11 sends the control parameters to the controllable light source 13 according to the definition of the acquired image, and in addition, the controllable light source 13 is an LED. The image processing embedded hardware platform consists of an embedded neural network processor supporting depth calculation model reasoning and a communication module. The deep learning system for defect detection is provided with independently researched and developed defect detection operation software and a deep learning algorithm model for carrying out complex defect type reasoning, and can adapt to different products to be detected, different defect types and different measurement precision requirements. The multi-form data signal input and output device supports the defect detection result of the intelligent camera to be output in various forms, and comprises a gigabit network port which is not limited to outputting an image result, a USB3.0 interface, an RS232 interface which outputs a digital model, an IO port which outputs a level signal, a WIFI module which performs wireless communication and other various interfaces. In summary, the intelligent camera has strong adaptability to the defect identification and detection of different workpiece surfaces, and has the characteristics of strong flexibility, high detection precision and the like.
Therefore, preferably, when embodied, the following steps may be performed:
a1, opening the industrial intelligent camera, and adjusting the illumination parameters of the controllable light source by the on-board camera chip according to the imaging requirement;
a2, using the low-distortion lens to take a picture of an industrial product, and transmitting the taken image of the product to the embedded processor through the on-board camera chip;
and A3, setting an industrial intelligent camera detection model, and selecting a preset model similar to the environmental characteristics of the workpiece to be detected from a preset deep learning defect detection model library for loading.
And A4, the embedded processor performs defect identification on the product image by using the preset deep learning defect detection model, outputs an inference result to the outside, and uploads the inference result to a server.
A5, presetting a deep learning defect detection model detection effect in the data statistical analysis and evaluation step S4, and if the detection effect cannot meet the production requirement, carrying out deep learning detection model training optimization.
As a preferred scheme, in this embodiment, the deep learning defect detection model is a deep neural network, a RestNet network is added to the deep neural network, and an identity mapping algorithm is added between each layer of networks.
Specifically, the deep neural network is composed of a plurality of connecting layers, convolutional layers and pooling layers, and the model progress is not always improved along with the increase of the network depth due to the self attributes of the deep neural network, because not only the test error becomes high after the network is deepened, but also the training error becomes high, the deeper network layer is accompanied with the problem of gradient disappearance/explosion, so that the convergence of the network is hindered, and the performance of the deep neural network is reduced due to the deepening of the network at this time, which is the degradation problem of the deep neural network. In order to solve the problem of network degradation, a ResNet network structure is added in the deep learning defect detection model in the industrial intelligent camera provided by the invention, and the problem that the gradient of a deep network disappears is solved by using a residual network in the ResNet network structure. It should be noted that the addition of the ResNet network used in the present invention to the deep network is not limited in hierarchical position, and may be performed according to an addition scheme well known to those skilled in the art; in addition, the ResNet network structure also uses the conventional purpose thereof, and is not specially set and described. The residual error network is also a common technical means in the field, and is not limited. The residual error network is added with identity mapping, the current output is directly transmitted to the next layer of network (all 1:1 transmission without adding extra parameters), namely a shortcut is taken, the operation of the current layer is skipped, the direct connection is named as 'skip connection', and meanwhile, in the backward propagation process, the gradient of the next layer of network is directly transmitted to the previous layer of network, so that the problem of gradient disappearance of the deep layer of network is solved.
Preferably, in the present embodiment, the low distortion lens 12 is an interchangeable lens.
Preferably, in this embodiment, the connection between the embedded processor 21 and the server is a communication mode, where the communication mode includes: 4G/3G communication and Internet communication. Preferably, the industrial intelligent camera also adopts a plurality of data exchange modes to the outside; the data exchange mode is that the multi-form data signal input and output device supports the defect detection result of the industrial intelligent camera to be output in various forms, and includes but is not limited to a gigabit network port for outputting an image result, a USB3.0 interface, an RS232 interface for outputting a digital model, an IO port for outputting a level signal and other various interfaces, a WIFI module for performing wireless communication and the like. Meanwhile, the communication mode may also use the above-mentioned multiple interfaces for communication connection.
Correspondingly, the invention also provides a defect detection method of the industrial intelligent camera suitable for the defect detection, which comprises the following steps:
s1, opening the industrial intelligent camera, and adjusting the illumination parameters of the controllable light source 13 by the on-board camera chip 11 according to the imaging requirements;
s2, shooting industrial products by using the low-distortion lens 12, and transmitting the shot product images to the embedded processor 21 through the on-board camera chip 11;
s3, the embedded processor 21 uses the deep learning defect detection model to identify the defects of the product image, and outputs an inference result to the outside, and the inference result is uploaded to a server.
Specifically, the invention provides an industrial intelligent camera for defect detection, and relates to an embedded intelligent camera based on an embedded machine vision technology and a deep learning technology. The industrial intelligent camera is opened in a manual mode or a remote control mode, after the industrial intelligent camera is opened, the on-board camera chip 11 performs a primary imaging test on an industrial product through the low-distortion lens 12, and monitors illumination parameters of the surrounding environment, if the parameters such as illumination intensity and angle are suitable, the step S2 is directly executed, if the surrounding light intensity and angle have flaws, the pipeline of the surrounding environment of the low-distortion lens 12 is adjusted by adjusting the brightness and angle of the controllable light source 13, namely, the illumination of the surrounding environment is controlled by adjusting the illumination parameters of the controllable light source 13; it should be noted here that the control manner for controlling the controllable light source 13 is a common technical means in the art; when the defects of the industrial products are identified, the industrial products to be detected are photographed through the low distortion lens 12, the acquired images are transmitted to the image defect identification system, the deep learning defect detection model in the embedded processor 21 is used for defect analysis, and then the detection results are output. The deep learning defect detection model for defect detection can adapt to different products to be detected, different defect types and different measurement precision requirements. In summary, the intelligent camera has strong adaptability to the defect identification and detection of different workpiece surfaces, and has the characteristics of strong flexibility, high detection precision and the like.
Preferably, in this embodiment, before step S1, the method further includes the steps of:
s0, training the deep learning defect detection model, and loading the trained deep learning defect detection model into the embedded processor 21.
As a preferred solution, in this embodiment, a training process of the deep learning defect detection model is as follows:
s01, constructing a deep learning defect detection model by using a deep neural network; in general, the creation and use of a new deep-learning defect detection model may require
S02, acquiring a plurality of defect images, calibrating defect positions of part of the defect images to be used as training images, and using the other part of the defect images as test images; generally, the selection of the defect image is to select a defect on an industrial product, and if the defects of other industrial products or other defects of the same industrial product need to be detected, the learning needs to be performed in sequence, and the defect detection method provided by the invention is further improved compared with other defect detection methods, all learning processes are performed in a server, namely independent learning without using the industrial intelligent camera, so that the most complete deep learning defect detection model can be left in the server, when the industrial intelligent camera needs to update the deep learning defect detection model, only pictures of one type of defects are uploaded to the server, the realization can be realized, and all the industrial intelligent cameras connected with the server can be added with functions; meanwhile, if a plurality of industrial intelligent cameras are connected with the server, when the internal deep learning defect detection model of the industrial intelligent cameras is updated, the industrial intelligent cameras at different detection positions update the deep learning defect detection model with targeted defect detection, namely the deep neural network, so that the defect detection model in the industrial intelligent cameras is prevented from being too fat and swollen, and the possible downtime is avoided;
s03, training the deep neural network by using all the training images, testing the trained deep neural network by using the test image, judging whether the requirements of an evaluation model are met, and if so, judging that the deep neural network is well trained; if not, step S02 is executed. Preferably, the number of the training images is greater than 2000, and the number of the test images is not less than 1000.
Preferably, in this embodiment, the step S0 further includes:
and after the camera working system judges that the deep learning defect detection model is detected for a preset number of times in an accumulated mode, the camera working system requests a server to update the deep learning defect detection model. The predetermined number of times is preferably 10000-.
Preferably, when the industrial intelligent camera detects the defects of industrial products, the detected pictures are also transmitted to the server, and the deep learning defect detection model on the server also performs detection once and performs training once to optimize the deep learning defect detection model; the method is used for judging the detection result of the industrial intelligent camera again, and is also used for optimizing the server deep learning defect detection model once. Here, the monitoring of the count of the number of detections is performed by using the camera work system, and the monitoring includes requesting the server to update the deep learning defect detection model, and is also performed by the camera work system.
Preferably, in this embodiment, when the deep learning defect detection model in the industrial intelligent camera needs to be updated, in order to improve the auditing efficiency, a migration updating method may be used, where the migration learning method is to solve the problem of tedious repeated labeling work (or insufficient data source), and improve the data use efficiency. For example, the deep learning defect detection model is preset with a bicycle detection model, when a motorcycle needs to be detected in later use, the detection precision is not high, the model needs to be retrained, the bicycle and the motorcycle have different similarities, so that a large amount of motorcycle data is not expected or cannot be collected (the environment is not allowed) when the defect detection model of the motorcycle is trained, the model is trained from the beginning, a small amount of motorcycle data can be collected at the moment, and the defect detection model capable of detecting the motorcycle is constructed by performing transfer learning on the basis of the original bicycle defect detection model, so that the training speed and the training efficiency are high. The conventional learning method of deep learning neural network is used in the transfer learning method, which is not limited and described in detail.
Preferably, in this embodiment, when the deep learning defect detection model in the industrial intelligent camera needs to be updated, according to the data statistics evaluation result of the preselected model and the similarity between the defect of the workpiece to be detected and the defect of the preset model, the following two methods can be selected for performing model optimization updating: 1. when the score is high and the similarity between the workpiece defect to be detected and the workpiece defect of the preset model is high, a small amount of image data of the workpiece defect to be detected can be collected and uploaded to a server side, transfer learning is carried out on the basis of the preset model, and a conventional learning method of a deep learning neural network is used for the transfer learning, so that limitation and repeated description are not needed. 2. And when the score is low and the similarity between the defects of the workpiece to be detected and the workpiece of the preset model is low, at least 2000 training data are required to be collected, and a deep learning defect detection model is reconstructed by network training. And after the trained deep learning defect detection model is downloaded to a camera, one-time optimization and upgrade of the deep learning defect detection model is completed.
Preferably, in this embodiment, in step S1, the imaging requirement is determined by an image peak signal-to-noise ratio algorithm; the image peak signal-to-noise ratio algorithm is as follows:
given a clean image I and a noisy image K of size m × n, the mean square error MSE is calculated as:
further, the calculation formula of the peak signal-to-noise ratio PSNR is:
wherein, MAXIFor the maximum pixel value possible for the picture, 255 if each pixel is represented by an 8-bit binary; in general, MAX is the value of a pixel if it is represented by a B-bit binaryI＝2B-1。
Preferably, in this embodiment, for a color image, the imaging requirement is determined by three determination methods, which are respectively:
the first judgment method comprises the following steps: respectively calculating peak signal-to-noise ratios (PSNR) of the RGB three channels of the color image, and then averaging;
and a second judgment method: respectively calculating Mean Square Error (MSE) of RGB three channels of the color image, and then taking an average value;
the third judgment method comprises the following steps: the color image is converted to YCbCr format and then only the peak signal-to-noise ratio PSNR of the Y component is calculated.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.
Claims (10)
1. A method for detecting industrial product defects is characterized by comprising the following steps:
s1, opening the industrial intelligent camera, and adjusting the illumination parameters of the controllable light source by the on-board camera chip according to imaging requirements;
s2, shooting industrial products by using the low-distortion lens, and transmitting shot product images to the embedded processor through the on-board camera chip;
and S3, the embedded processor performs defect identification on the product image by using the deep learning defect detection model, outputs an inference result to the outside, and uploads the inference result to a server.
2. The industrial product defect detection method of claim 1, further comprising, before step S1, the steps of:
s0, training the deep learning defect detection model, and loading the trained deep learning defect detection model into the embedded processor.
3. The industrial product defect detection method of claim 2, wherein the deep learning defect detection model is trained as follows:
s01, constructing a deep learning defect detection model by using a deep neural network;
s02, acquiring a plurality of defect images, calibrating defect positions of part of the defect images to be used as training images, and using the other part of the defect images as test images;
s03, training the deep neural network by using all the training images, testing the trained deep neural network by using the test image, judging whether the requirements of an evaluation model are met, and if so, judging that the deep neural network is well trained; if not, step S02 is executed.
4. The industrial product defect detection method according to claim 2, wherein the step S0 further comprises:
and after the camera working system judges that the deep learning defect detection model is detected for a preset number of times in an accumulated mode, the camera working system requests a server to update the deep learning defect detection model.
5. The industrial product defect detection method according to claim 1, wherein in the step S1, the imaging requirement is determined by an image peak signal-to-noise ratio algorithm; the image peak signal-to-noise ratio algorithm is as follows:
given a clean image I and a noisy image K of size m × n, the mean square error MSE is calculated as:
further, the calculation formula of the peak signal-to-noise ratio PSNR is:
wherein, MAXIFor the maximum pixel value possible for a picture, if each pixel is derived from an 8-bit binaryThis indicates that this is 255; in general, MAX is the value of a pixel if it is represented by a B-bit binaryI＝2B-1。
6. The industrial product defect detection method according to claim 5, wherein the imaging requirement is determined by three determination methods for the color image, which are respectively:
the first judgment method comprises the following steps: respectively calculating peak signal-to-noise ratios (PSNR) of the RGB three channels of the color image, and then averaging;
and a second judgment method: respectively calculating Mean Square Error (MSE) of RGB three channels of the color image, and then taking an average value;
the third judgment method comprises the following steps: the color image is converted to YCbCr format and then only the peak signal-to-noise ratio PSNR of the Y component is calculated.
7. An industrial intelligent camera using the industrial product defect detection method of claims 1-6, characterized by comprising an image acquisition system, an image defect recognition system;
the image acquisition system comprises an on-board camera chip, a low-distortion lens and a controllable light source; the low distortion lens and the controllable light source are respectively connected with the on-board camera chip;
the image defect identification system comprises an embedded processor and a storage chip; the on-board camera chip is connected with the embedded processor; the embedded processor is internally provided with a deep learning defect detection model and a camera working system; the embedded processor is connected with the server.
8. The industrial smart camera of claim 7, wherein the deep learning defect detection model is a deep neural network, a RestNet network is added to the deep neural network, and an identity mapping algorithm is added between each layer of network.
9. The industrial smart camera of claim 7, wherein the low distortion lens is a replaceable lens.
10. The industrial smart camera of claim 7, wherein the connection between the embedded processor and the server is a communication mode, the communication mode comprising: 4G/3G communication and Internet communication.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010730238.5A CN112082999A (en) | 2020-07-27 | 2020-07-27 | Industrial product defect detection method and industrial intelligent camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010730238.5A CN112082999A (en) | 2020-07-27 | 2020-07-27 | Industrial product defect detection method and industrial intelligent camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112082999A true CN112082999A (en) | 2020-12-15 |
Family
ID=73736142
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010730238.5A Pending CN112082999A (en) | 2020-07-27 | 2020-07-27 | Industrial product defect detection method and industrial intelligent camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112082999A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112581462A (en) * | 2020-12-25 | 2021-03-30 | 北京邮电大学 | Method and device for detecting appearance defects of industrial products and storage medium |
| CN112711603A (en) * | 2020-12-30 | 2021-04-27 | 广东粤云工业互联网创新科技有限公司 | Workpiece detection method and system based on cloud and computer-readable storage medium |
| CN112712505A (en) * | 2020-12-30 | 2021-04-27 | 广东粤云工业互联网创新科技有限公司 | Workpiece detection method and system based on cloud and computer-readable storage medium |
| CN112881412A (en) * | 2021-02-01 | 2021-06-01 | 南京耘瞳科技有限公司 | Method for detecting non-metal foreign bodies in scrap steel products |
| CN113538420A (en) * | 2021-09-07 | 2021-10-22 | 深圳新视智科技术有限公司 | Defect detection method and system based on double cameras and multiple light sources |
| CN116343213A (en) * | 2023-05-31 | 2023-06-27 | 成都数之联科技股份有限公司 | Model training and chip character recognition method, device, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5936725A (en) * | 1997-10-28 | 1999-08-10 | Materials Technologies Corp. | Apparatus and method for viewing and inspecting a circumferential surface area of a test object |
| CN109658376A (en) * | 2018-10-24 | 2019-04-19 | 哈尔滨工业大学 | A kind of surface defect recognition method based on image recognition |
| US10346969B1 (en) * | 2018-01-02 | 2019-07-09 | Amazon Technologies, Inc. | Detecting surface flaws using computer vision |
| CN110349126A (en) * | 2019-06-20 | 2019-10-18 | 武汉科技大学 | A kind of Surface Defects in Steel Plate detection method based on convolutional neural networks tape label |
| CN110702699A (en) * | 2019-11-15 | 2020-01-17 | 湖南讯目科技有限公司 | Rolled glass defect detection device and method |
| CN111272763A (en) * | 2018-12-04 | 2020-06-12 | 通用电气公司 | System and method for workpiece inspection |
-
2020
- 2020-07-27 CN CN202010730238.5A patent/CN112082999A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5936725A (en) * | 1997-10-28 | 1999-08-10 | Materials Technologies Corp. | Apparatus and method for viewing and inspecting a circumferential surface area of a test object |
| US10346969B1 (en) * | 2018-01-02 | 2019-07-09 | Amazon Technologies, Inc. | Detecting surface flaws using computer vision |
| CN109658376A (en) * | 2018-10-24 | 2019-04-19 | 哈尔滨工业大学 | A kind of surface defect recognition method based on image recognition |
| CN111272763A (en) * | 2018-12-04 | 2020-06-12 | 通用电气公司 | System and method for workpiece inspection |
| CN110349126A (en) * | 2019-06-20 | 2019-10-18 | 武汉科技大学 | A kind of Surface Defects in Steel Plate detection method based on convolutional neural networks tape label |
| CN110702699A (en) * | 2019-11-15 | 2020-01-17 | 湖南讯目科技有限公司 | Rolled glass defect detection device and method |
Non-Patent Citations (2)
| Title |
|---|
| 双锴: "《计算机视觉》", 31 January 2020 * |
| 赵小川 等: "《MATLAB数字图像处理——从仿真到C/C++代码的自动生成》", 30 September 2015 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112581462A (en) * | 2020-12-25 | 2021-03-30 | 北京邮电大学 | Method and device for detecting appearance defects of industrial products and storage medium |
| CN112711603A (en) * | 2020-12-30 | 2021-04-27 | 广东粤云工业互联网创新科技有限公司 | Workpiece detection method and system based on cloud and computer-readable storage medium |
| CN112712505A (en) * | 2020-12-30 | 2021-04-27 | 广东粤云工业互联网创新科技有限公司 | Workpiece detection method and system based on cloud and computer-readable storage medium |
| CN112881412A (en) * | 2021-02-01 | 2021-06-01 | 南京耘瞳科技有限公司 | Method for detecting non-metal foreign bodies in scrap steel products |
| CN112881412B (en) * | 2021-02-01 | 2023-03-10 | 南京耘瞳科技有限公司 | Method for detecting non-metal foreign matters in scrap steel products |
| CN113538420A (en) * | 2021-09-07 | 2021-10-22 | 深圳新视智科技术有限公司 | Defect detection method and system based on double cameras and multiple light sources |
| CN116343213A (en) * | 2023-05-31 | 2023-06-27 | 成都数之联科技股份有限公司 | Model training and chip character recognition method, device, equipment and medium |
| CN116343213B (en) * | 2023-05-31 | 2023-08-25 | 成都数之联科技股份有限公司 | Model training and chip character recognition method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112082999A (en) | Industrial product defect detection method and industrial intelligent camera | |
| US20200349875A1 (en) | Display screen quality detection method, apparatus, electronic device and storage medium | |
| WO2021036824A1 (en) | Information collection device and method, inspection robot and storage medium | |
| JP7476145B2 (en) | Image processing method and imaging device | |
| US20210227126A1 (en) | Deep learning inference systems and methods for imaging systems | |
| WO2020007096A1 (en) | Method and device for detecting quality of display screen, electronic device, and storage medium | |
| CN110689539B (en) | Workpiece surface defect detection method based on deep learning | |
| GB2595558A (en) | Exposure defects classification of images using a neural network | |
| CN113096098A (en) | Casting appearance defect detection method based on deep learning | |
| CN112347887A (en) | Object detection method, object detection device and electronic equipment | |
| JP2022538242A (en) | Method, mobile terminal equipment and system for evaluating laser cut edges | |
| WO2018040105A1 (en) | System and method for food recognition, food model training method, refrigerator and server | |
| CN111027415A (en) | Vehicle detection method based on polarization image | |
| CN112040198A (en) | Intelligent water meter reading identification system and method based on image processing | |
| CN109614994A (en) | A kind of tile typology recognition methods and device | |
| CN114119489A (en) | Automatic detection method for excess of electric connector and needle retracting and needle reversing defects | |
| CN113012228A (en) | Station positioning system and workpiece positioning method based on deep learning | |
| CN116523853A (en) | Chip detection system and method based on deep learning | |
| CN115841520A (en) | Camera internal reference calibration method and device, electronic equipment and medium | |
| CN117011214A (en) | Object detection method, device, equipment and storage medium | |
| CN114663299A (en) | Training method and device suitable for image defogging model of underground coal mine | |
| CN110514662B (en) | Visual detection system with multi-light-source integration | |
| CN109862268B (en) | Image processing polishing method capable of sensing and automatically adjusting brightness | |
| CN117237939B (en) | Image data-based detection method and device for food maturity of young cooker | |
| CN117058150B (en) | Method and device for detecting defects of lamp beads |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201215 |
|
| RJ01 | Rejection of invention patent application after publication |