CN114663392A - Knowledge distillation-based industrial image defect detection method - Google Patents

Knowledge distillation-based industrial image defect detection method Download PDF

Info

Publication number
CN114663392A
CN114663392A CN202210285918.XA CN202210285918A CN114663392A CN 114663392 A CN114663392 A CN 114663392A CN 202210285918 A CN202210285918 A CN 202210285918A CN 114663392 A CN114663392 A CN 114663392A
Authority
CN
China
Prior art keywords
network
image
student
teacher
student network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210285918.XA
Other languages
Chinese (zh)
Inventor
雷可
杨俊杰
郑军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Matrixtime Robotics Shanghai Co ltd
Original Assignee
Matrixtime Robotics Shanghai Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Matrixtime Robotics Shanghai Co ltd filed Critical Matrixtime Robotics Shanghai Co ltd
Priority to CN202210285918.XA priority Critical patent/CN114663392A/en
Publication of CN114663392A publication Critical patent/CN114663392A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0006—Industrial image inspection using a design-rule based approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Abstract

The invention relates to the technical field of image processing, in particular to a knowledge distillation-based industrial image defect detection method, which trains a teacher network and a student network by using a normal image set, takes the difference between characteristic graphs generated by the teacher network and the student network as a loss function, only updates the parameters of the student network, stops training under the condition that the loss does not decrease any more, and obtains the trained student network; inputting an image to be detected to a teacher network and a student network, recording the difference between feature maps of different scales of the teacher network and the student network as an abnormal score standard, sampling the feature maps to the size of the input image, multiplying corresponding positions to obtain the final abnormal score of each pixel point on the image, setting a threshold value, and determining an abnormal area if the threshold value is larger than the threshold value. According to the invention, the mode of extracting OK image features from the student network to the learning teacher network is adopted, and when NG images are encountered, the difference between the student network and the teacher network is highlighted, so that the purpose of detecting the NG images is achieved.

Description

Knowledge distillation-based industrial image defect detection method
Technical Field
The invention relates to the technical field of image processing, in particular to an industrial image defect detection method based on knowledge distillation.
Background
With the rapid development of deep learning in recent years, more and more deep learning algorithms are applied to various industries. The knowledge distillation algorithm in the deep learning algorithm has important research and application values for application in industrial image defect detection.
The method comprises the following steps of 1, most of detection pictures are OK samples, the proportion of the defect pictures in the total detection pictures is small, the requirement that a supervision algorithm needs to label defects in advance is not very favorable, 2, the occurrence positions and the forms of the defects on the industrial images are random, the types of the defects are difficult to define, 3, the number difference among the types of the defects on the industrial images is large, 4, the defects on the industrial images are unknown, and a plurality of defects can appear along with the aging of equipment.
For the current situation and problems of the defect detection of the current industrial image summarized above, the traditional image algorithm and the supervised deep learning algorithm have certain limitations on the degree of solution of the scene, and the final detection effect is generally difficult to achieve an ideal effect. Therefore, an industrial image defect detection method based on knowledge distillation is provided.
Disclosure of Invention
Based on the technical problems in the background art, the invention provides the industrial image defect detection method based on knowledge distillation, which does not need to screen and label a data set in a supervision algorithm, reduces a large amount of manual workload, and solves the problem that the traditional image algorithm and the supervision deep learning algorithm have certain limitation on the solution degree of industrial image defect detection.
The invention provides the following technical scheme: a knowledge distillation-based industrial image defect detection method comprises the following steps:
s1, firstly, a normal image set I containing no defect is given1，I2，...，InPreprocessing each picture in the input image set; constructing teacher network and student network, inputting image I in training processkThe teacher network and the student network respectively output three feature maps with different scales, the difference between the teacher network feature map and the student network feature maps is used as a loss function, the SGD method is used in the training process, the parameters of the student network are updated, the parameters of the teacher network remain unchanged, the training is stopped under the condition that the loss does not decrease any more, and the trained student network is obtained;
s2, preprocessing the input image to be detected, simultaneously inputting the preprocessed image into a teacher network and a student network to respectively obtain three feature graphs of different scales, and recording the difference between the feature graphs of different scales of the teacher network and the feature graphs of different scales of the student network as an abnormal score standard:
Figure BDA0003558237150000021
the final anomaly score is calculated as follows:
Figure BDA0003558237150000022
the up-sampling mode adopts bilinear interpolation, each characteristic image is up-sampled to the size of an input image, the more abnormal part on the image to be detected is, the higher the score is, and the image area larger than the thresh part is regarded as the abnormal area on the detection image by setting a threshold thresh.
Preferably, Ik∈Rwxh xcW is the input image width, h is the input image height, and c is the input image channel number;
the teacher network outputs three characteristic graphs with different scales as
Figure BDA0003558237150000023
Figure BDA0003558237150000024
The student network outputs three characteristic graphs with different scales as
Figure BDA0003558237150000025
Figure BDA0003558237150000031
Preferably, the loss function is defined as follows:
Figure BDA0003558237150000032
i.e. the sum of the distances at the corresponding locations (i, j) for each feature scale.
Preferably, the preprocessing is histogram equalization processing for eliminating the influence of image gray level non-uniformity caused in the shooting process.
Preferably, the teacher network is a deep convolutional network pre-trained on imagenet, the student network and the teacher network have the same structure, and the parameters of the student network do not use the parameters in the pre-trained network, but are initialized randomly.
The invention provides an industrial image defect detection method based on knowledge distillation, which is used for detecting industrial defects by using a knowledge distillation method in deep learning, does not need to screen and label a data set in a supervision algorithm, and reduces a large amount of manual workload. By designing a knowledge distillation network and a related loss function, the student network is trained to learn normal image feature extraction in the teacher network. In the inference process, the difference between the two networks is compared to be used as the abnormal score of each pixel point on the current image, and the abnormal area can be determined by setting a threshold value.
Drawings
FIG. 1 is a flow chart of the training process of the present invention;
FIG. 2 is a flow chart of the inference process of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a technical scheme that: a knowledge distillation-based industrial image defect detection method is divided into two parts, namely a training part and an inference part.
As shown in FIG. 1, the training part is mainly divided into the following steps, firstly, a normal image set I without defects is given1，I2，...，InAnd preprocessing each picture in the input image set, mainly histogram equalization, so as to eliminate the influence of uneven image gray scale caused in the shooting process.
The second step is to build a teacher network, which is a deep convolutional network pre-trained on imagenet, typically resnet 50.
And thirdly, constructing a student network, wherein the student network and the teacher network have the same structure, but the parameters of the student network do not use the parameters in the pre-training network, but are randomly initialized.
The fourth step is to construct the whole training network and input images I in the training processkIn which Ik∈Rwxh xcW is the width of the input image, h is the height of the input image, c is the number of channels of the input image, and the teacher network outputs three feature maps with different scales
Figure BDA0003558237150000041
Figure BDA0003558237150000042
The student network outputs three feature maps with different scales
Figure BDA0003558237150000043
Figure BDA0003558237150000044
Using the difference between the teacher network and the student network profiles as a loss function, the loss function is defined as follows
Figure BDA0003558237150000045
I.e., the sum of the distances of l2 at the corresponding locations (i, j) for each feature scale.
And updating parameters of the student network by using an SGD method in the training process, keeping the parameters of the teacher network unchanged, and stopping training under the condition that the loss is not reduced any more to obtain the trained student network.
As shown in fig. 2, the inference part is the following steps,
and (4) preprocessing the input image to be detected, wherein the specific operation and the training process are consistent.
Inputting the preprocessed image I into a teacher network to obtain three characteristic graphs with different scales
Figure BDA0003558237150000051
Figure BDA0003558237150000052
Simultaneously inputting the images into a student network to obtain three characteristic graphs with different scales
Figure BDA0003558237150000053
Figure BDA0003558237150000054
The difference between the feature maps of different scales of the teacher network and the student network is recorded as an abnormal score standard:
Figure BDA0003558237150000055
the final anomaly score is calculated as follows:
Figure BDA0003558237150000056
wherein the upsampling mode adopts bilinear interpolation. The respective feature maps are up-sampled to the input image size, w, h. The more abnormal places on the image to be detected are, the higher the score is, and by setting a threshold thresh, the image area larger than the thresh is regarded as the abnormal area on the detected image.
The teacher network and the student network of the distillation network in the prior art are generally complex, the student network is simple, the teacher network and the student network in the application are of the same structure, the learning process in the prior art needs to label original data, the method in the application does not need to be implemented only by completely dropping OK images, the central idea of the algorithm in the application is a mode of hopeing the student network to learn the characteristics of the OK images in the teacher network, and when NG images are encountered, the difference between the student network and the teacher network is highlighted, so that the purpose of detecting the NG images is achieved.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.

Claims (5)

1. A knowledge distillation-based industrial image defect detection method is characterized by comprising the following steps: the method comprises the following steps:
s1, firstly, a training normal image set I containing no defects is given1，I2，...，InFor each picture in the input image setCarrying out pretreatment; constructing teacher network and student network, inputting image I in training processkThe teacher network and the student network respectively output three feature maps with different scales, the difference between the teacher network feature map and the student network feature maps is used as a loss function, the SGD method is used in the training process, the parameters of the student network are updated, the parameters of the teacher network remain unchanged, the training is stopped under the condition that the loss does not decrease any more, and the trained student network is obtained;
s2, preprocessing the input image to be detected, simultaneously inputting the preprocessed image into a teacher network and a student network to respectively obtain three feature graphs with different scales, and recording the difference between the feature graphs with different scales of the teacher network and the feature graphs with different scales of the student network as an abnormal score standard:
Figure FDA0003558237140000011
the final anomaly score is calculated as follows:
Figure FDA0003558237140000012
the up-sampling mode adopts bilinear interpolation, each characteristic image is up-sampled to the size of an input image, the more abnormal part on the image to be detected is, the higher the score is, and the image area larger than the thresh part is regarded as the abnormal area on the detection image by setting a threshold thresh.
2. The knowledge-based distillation industrial image defect detection method according to claim 1, wherein: i isk∈Rwxh xcW is the input image width, h is the input image height, and c is the input image channel number;
the teacher network outputs three characteristic graphs with different scales as
Figure FDA0003558237140000013
Figure FDA0003558237140000021
The student network outputs three characteristic graphs with different scales as
Figure FDA0003558237140000022
Figure FDA0003558237140000023
3. The knowledge-based distillation industrial image defect detection method according to claim 1, wherein: the loss function is defined as follows:
Figure FDA0003558237140000024
i.e. the sum of the distances at the corresponding locations (i, j) for each feature scale.
4. The knowledge-based distillation industrial image defect detection method according to claim 1, wherein: the preprocessing is histogram equalization processing and is used for eliminating the influence of uneven image gray scale caused in the shooting process.
5. The knowledge-based distillation industrial image defect detection method according to claim 1, wherein: the teacher network is a deep convolutional network pre-trained on imagenet, the student network and the teacher network have the same structure, and parameters of the student network do not use parameters in the pre-trained network, but are initialized randomly.
CN202210285918.XA 2022-03-22 2022-03-22 Knowledge distillation-based industrial image defect detection method Pending CN114663392A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210285918.XA CN114663392A (en) 2022-03-22 2022-03-22 Knowledge distillation-based industrial image defect detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210285918.XA CN114663392A (en) 2022-03-22 2022-03-22 Knowledge distillation-based industrial image defect detection method

Publications (1)

Publication Number Publication Date
CN114663392A true CN114663392A (en) 2022-06-24

Family

ID=82031243

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210285918.XA Pending CN114663392A (en) 2022-03-22 2022-03-22 Knowledge distillation-based industrial image defect detection method

Country Status (1)

Country Link
CN (1) CN114663392A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115471645A (en) * 2022-11-15 2022-12-13 南京信息工程大学 Knowledge distillation anomaly detection method based on U-shaped student network
CN116862885A (en) * 2023-07-14 2023-10-10 江苏济远医疗科技有限公司 Segmentation guide denoising knowledge distillation method and device for ultrasonic image lesion detection
WO2024000372A1 (en) * 2022-06-30 2024-01-04 宁德时代新能源科技股份有限公司 Defect detection method and apparatus

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024000372A1 (en) * 2022-06-30 2024-01-04 宁德时代新能源科技股份有限公司 Defect detection method and apparatus
CN115471645A (en) * 2022-11-15 2022-12-13 南京信息工程大学 Knowledge distillation anomaly detection method based on U-shaped student network
CN116862885A (en) * 2023-07-14 2023-10-10 江苏济远医疗科技有限公司 Segmentation guide denoising knowledge distillation method and device for ultrasonic image lesion detection

Similar Documents

Publication Publication Date Title
US11176408B2 (en) Tire image recognition method and tire image recognition device
CN114663392A (en) Knowledge distillation-based industrial image defect detection method
CN108229381B (en) Face image generation method and device, storage medium and computer equipment
CN108334936B (en) Fault prediction method based on migration convolutional neural network
CN109919032B (en) Video abnormal behavior detection method based on motion prediction
CN110717481B (en) Method for realizing face detection by using cascaded convolutional neural network
CN108805002B (en) Monitoring video abnormal event detection method based on deep learning and dynamic clustering
CN113240580A (en) Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation
CN111091109B (en) Method, system and equipment for predicting age and gender based on face image
CN109740721B (en) Wheat ear counting method and device
CN111079539B (en) Video abnormal behavior detection method based on abnormal tracking
CN109635763B (en) Crowd density estimation method
CN104091341A (en) Image blur testing method based on significance testing
CN104091340A (en) Blurred image rapid detection method
CN111489401A (en) Image color constancy processing method, system, equipment and storage medium
CN109740553B (en) Image semantic segmentation data screening method and system based on recognition
CN115147418B (en) Compression training method and device for defect detection model
CN111242026A (en) Remote sensing image target detection method based on spatial hierarchy perception module and metric learning
CN116760583B (en) Enhanced graph node behavior characterization and abnormal graph node detection method
CN112070720A (en) Transformer substation equipment defect identification method based on deep learning model
CN107203980A (en) The Underwater Target Detection image enchancing method of self-adapting multi-dimension dark channel prior
CN115240259A (en) Face detection method and face detection system based on YOLO deep network in classroom environment
CN116452810A (en) Multi-level semantic segmentation method and device, electronic equipment and storage medium
CN115578585A (en) Industrial image anomaly detection method, system, computer device and storage medium
CN111047618A (en) Multi-scale-based non-reference screen content image quality evaluation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination