CN114926682A - Local outlier factor-based industrial image anomaly detection and positioning method and system - Google Patents

Local outlier factor-based industrial image anomaly detection and positioning method and system Download PDF

Info

Publication number
CN114926682A
CN114926682A CN202210547394.7A CN202210547394A CN114926682A CN 114926682 A CN114926682 A CN 114926682A CN 202210547394 A CN202210547394 A CN 202210547394A CN 114926682 A CN114926682 A CN 114926682A
Authority
CN
China
Prior art keywords
image
positioning
abnormal
feature
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210547394.7A
Other languages
Chinese (zh)
Inventor
谢锐
陈华华
郭春生
叶学义
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202210547394.7A priority Critical patent/CN114926682A/en
Publication of CN114926682A publication Critical patent/CN114926682A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an industrial image anomaly detection and positioning method and system based on local outlier factors, and the method specifically comprises the following steps: (1) acquiring a training data set; (2) extracting characteristics; (3) and (4) carrying out anomaly detection and positioning by adopting a local outlier factor algorithm. The invention provides an unsupervised anomaly detection and positioning technical scheme by combining deep learning and a traditional method, and the unsupervised anomaly detection and positioning technical scheme has higher anomaly detection accuracy and more accurate anomaly positioning performance.

Description

Local outlier factor-based industrial image anomaly detection and positioning method and system
Technical Field
The invention belongs to the technical field of industrial image abnormity detection and positioning, and particularly relates to an industrial image abnormity detection and positioning method and system based on local outlier factors.
Background
Humans are very good at identifying whether an image is similar to what they observed before, or whether it is abnormal, a task known as anomaly detection and has a wide range of applications, including industrial image anomaly detection. However, anomalies are relatively difficult to detect on a production line, resulting in human detection being difficult. Therefore, the automation of abnormality detection contributes to reducing the work difficulty of the operator and achieving continuous quality control.
Disclosure of Invention
In order to solve the problem of the existing industrial image anomaly detection and positioning, the invention provides an industrial image anomaly detection and positioning method and system based on Local Outlier Factor (LOF).
In order to achieve the purpose, the invention adopts the following technical scheme:
an industrial image anomaly detection and positioning method based on local outlier factors comprises a training stage, wherein the training stage specifically comprises the following steps:
step (1), acquiring a training data set;
step (2), feature extraction;
and (3) adopting a local outlier factor algorithm to carry out anomaly detection and positioning.
Preferably, the step (1) acquires a training data set, specifically:
the training data set consists of normal pictures, the training set image is resized to 256 × 256 and centered and cropped to 224 × 224.
Preferably, the step (2) of feature extraction specifically includes:
the ResNet model obtained by pre-training is used as a feature extractor F, and the first three layers of the feature extractor are recorded as F 3 . For a given image x i Extracted feature f i ：f i ＝F 3 (x i ). And in the training stage, the characteristics of all training images are extracted and stored. TestingIn this case, only the features of the target image are extracted.
Preferably, in step (3), the local outlier factor algorithm is used for detecting and locating the abnormality, specifically:
the step is divided into two parts, one part is the third layer output characteristic f of the ResNet network i Training is performed as an input to the LOF algorithm. And the other part fuses and reduces the dimension of the first layer output feature, the second layer output feature and the third layer output feature of the ResNet network, the size of the fused feature map is 1792 multiplied by 56, the dimension of each super pixel on the feature map after dimension reduction is 550 multiplied by 1, and finally each super pixel is used as the input of the LOF algorithm for training.
Preferably, the abnormality detection process is specifically as follows:
outputting the features F by the third layer of the feature extractor F in the step (2) i And is trained as an input to the LOF algorithm. Each f is i Consider a data point, calculate its kth distance, and then find the kth distance neighborhood for the data point. And finally, calculating the local reachable density of each data point, and calculating a local outlier factor to serve as an abnormal score. The specific calculation method is as follows:
calculating the data points f i Is the kth distance of f i Point k to f i Has a Euclidean distance of d k (f i )。
Find f i K-th distance neighborhood N k (f i ) I.e. f i All points within the kth distance of (c), including the kth distance. | N k (f i ) And | is more than or equal to k, | represents the number of elements in the set, k is more than 0 and less than or equal to L, and L is the total number of samples.
Calculate the local reachable density for each data point:
Figure BDA0003649958870000021
wherein, reach _ dist k (f i T) is f i And N k (f i ) The reachable distance of a certain point t is calculated according to the formula: reach _ dist k (f i ,t)＝max{d k (f i ),d(f i T) }; d (·,. cndot.) represents the Euclidean distance, t is N k (f i ) Internal removal of f i At some point outside.
Calculating f i The local outlier factor takes this as the anomaly score:
Figure BDA0003649958870000022
LOF k (f i ) Representing point f i Neighborhood point N of k (f i ) Local reachable density and point f i Is averaged over the locally achievable density ratio. During testing, the local outlier factor is used as the abnormal score of the tested sample, the threshold is set as th (0 < th < 1), the abnormal sample is judged if the threshold is greater than th, and the normal sample is judged if the threshold is less than th.
Preferably, the anomaly locating process is as follows:
selecting the first, second and third layers of output features of the feature extractor F in the step (2) to be fused to obtain fused features
Figure BDA0003649958870000023
Where N is the number of input training images, C s The sum of the number of the first, second and third layers of output channels is shown, H and W show the resolution of the maximum output characteristic diagram. Firstly, amplifying each channel of a first, a second and a third layers of output feature maps of F into a feature map with the resolution of H multiplied by W, and then connecting the output feature maps from different layers front and back to obtain a fusion feature map containing information of different semantic levels and resolutions, thereby realizing coding of different granularities and global contexts.
The fusion characteristic diagram possibly carries redundant information, and in order to reduce the calculation complexity of the model and reduce the memory occupation, the invention adopts a random dimension reduction mode to reduce the dimension of the fusion characteristic diagram P from the channel dimension C of the P s (C s 1792) to randomly select 550 dimensions. With a super pixel p ij ∈P N×550×H×W ((i,j)∈[1,W]×[1,H]) Trained as input to the LOF algorithm, calculates the kth distance d for each superpixel k (p ij ) And then finds its k-th distance neighborhood. Finally, the local reachable density of each superpixel is calculated according to the formula (1), and the abnormal score is calculated according to the formula (2).
Preferably, the invention also comprises a step (4) of testing stage, specifically:
step (i) the test image is resized to 256 x 256 and centered to 224 x 224.
And (II) extracting the features of the test image by using a ResNet feature extractor F obtained by pre-training in the step (2) in the training stage. For the abnormal detection task, the abnormal score of the test image is calculated according to the formula (2), and the abnormal image is judged to be an abnormal image when the abnormal score is larger than the threshold th, and the normal image is judged to be a normal image when the abnormal score is smaller than or equal to the threshold th.
And (3) for the abnormal positioning task, extracting the features of the test image by using the feature extractor F in the step (2), fusing the output features of the first layer, the second layer and the third layer, reducing the dimension into 550 dimensions, and calculating the abnormal score of each super pixel after fusion to obtain a pixel-level abnormal score map M of each feature map. Because the resolution of M is less than that of the image to be measured, M is up-sampled to the size of the image to be measured by adopting bilinear interpolation, and the result is smoothed by using a Gaussian filter. Finally, calculating PR curves of the M and the test image true value image, and taking a balance point of the PR curves as an optimal threshold Th; judging the value smaller than or equal to Th in M as 0, judging the value larger than Th as 1, and judging that 1 represents that the abnormal pixel 0 represents a normal pixel to obtain an abnormal positioning mask image M'; and setting the position of 0 in M' to be 0 in the test image to obtain an image abnormal pixel positioning result image.
The invention also discloses an industrial image anomaly detection and positioning system based on the local outlier factor, which comprises the following modules:
the acquisition module is used for acquiring a training data set;
an extraction module: for extracting features;
an anomaly detection and location module: and (4) carrying out anomaly detection and positioning by adopting a local outlier factor algorithm.
The invention focuses on the detection of image anomalies, particularly in the localization of industrial images. The invention discloses an efficient and practical industrial image anomaly detection and positioning method and system based on local outlier factors, which are used for extracting image characteristics by adopting a ResNet network obtained by pre-training. The method uses the third layer output characteristics of the characteristic extraction network as the input of the local outlier factor algorithm to calculate the abnormal score of the test image, thereby realizing the abnormal detection. The output features from different layers of ResNet are fused, and the abnormal score of each super pixel of the fused feature map is calculated by using a local outlier factor algorithm, so that abnormal positioning is realized. The invention provides an unsupervised anomaly detection and positioning technical scheme by combining deep learning and a traditional method, and has higher anomaly detection accuracy and more accurate anomaly positioning performance.
Drawings
FIG. 1 is a diagram of an industrial image anomaly detection model based on local anomaly factors;
FIG. 2 is a diagram of an industrial image anomaly location model based on local anomaly factors;
FIG. 3 is a feature map fusion example diagram;
FIG. 4 is a block diagram of an industrial image anomaly detection and localization system based on local outlier factors according to the present invention.
Detailed Description
The present invention is described in detail below with reference to examples so that those skilled in the art can better understand the present invention. It should be particularly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the main aspects of the present invention.
Example 1
As shown in fig. 1-3, the present embodiment is a local outlier factor-based industrial image anomaly detection and localization method, which includes a training phase and a testing phase.
The training phase is specifically as follows:
and (1) acquiring a training data set.
The training dataset consists of normal pictures in the MVTec AD (https:// www.mvtec.com/company/research/datasets/MVTec-AD /) dataset, the training set image is resized to 256 × 256, and centered cropped to 224 × 224.
And (2) extracting features.
In this embodiment, a ResNet model obtained by pre-training on an ImageNet dataset is used as a feature extractor F, and the first three layers of the feature extractor are denoted as F 3 . For a given image x i Extracted feature f i ：f i ＝F 3 (x i ). And in the training stage, the characteristics of all training images are extracted and stored. During testing, only the features of the target image are extracted.
And (3) carrying out anomaly detection and positioning by using a local outlier factor algorithm.
The step is divided into two parts, one part is the third layer output characteristic f of the ResNet network i Training is performed as an input to the LOF algorithm. And the other part fuses and reduces the dimension of the first layer output feature, the second layer output feature and the third layer output feature of the ResNet network, the size of the fused feature map is 1792 multiplied by 56, the dimension of each super pixel on the feature map after dimension reduction is 550 multiplied by 1, and finally each super pixel is used as the input of the LOF algorithm for training.
The anomaly detection process is as follows:
outputting the features F by the third layer of the feature extractor F in the step (2) i Training is performed as an input to the LOF algorithm. Each f is i Consider a data point, calculate its kth distance, and then find the kth distance neighborhood for that data point. And finally, calculating the local reachable density of each data point, and calculating a local outlier factor to serve as an abnormal score. The specific calculation method is as follows:
calculating the data point f i Is the kth distance of f i Point k to f i Has a Euclidean distance of d k (f i )。
Find f i K distance neighborhood N k (f i ) I.e. f i All points within the kth distance of (c), including the kth distance. | N k (f i ) And | is more than or equal to k, |, represents the number of elements in the set, and the value of k in the invention is 20.
Calculate the local reachable density for each data point:
Figure BDA0003649958870000051
wherein, reach _ dist k (f i T) is f i And N k (f i ) The reachable distance of a certain point t is calculated according to the formula: reach _ dist k (f i ,t)＝max{d k (f i ),d(f i T) }; d (,) represents the Euclidean distance, t is N k (f i ) Internal removing f i At some point outside.
Calculating f i The local outlier factor of (a) is taken as the outlier score:
Figure BDA0003649958870000052
LOF k (f i ) Representing point f i Neighborhood point N of k (f i ) Local achievable density and point f i Is averaged over the locally achievable density ratio. During testing, the local outlier factor is used as the abnormal score of the tested sample, the threshold th is 0.5, the sample which is larger than the threshold th is judged as an abnormal sample, and the sample which is smaller than the threshold th is judged as a normal sample.
The anomaly locating process is as follows:
selecting the first, second and third layers of output features of the feature extractor F in the step (2) to be fused to obtain a fused feature map
Figure BDA0003649958870000053
Where N is the number of input training images, C s The sum of the number of the first, second and third layers of output channels is shown, H and W show the resolution of the maximum output characteristic diagram. Firstly, amplifying each channel of a first, a second and a third layers of output feature maps of F into a feature map with the resolution of H multiplied by W, and then connecting the output feature maps from different layers front and back to obtain a fusion feature map containing information of different semantic levels and resolutions, thereby realizing coding of different granularities and global contexts.
The fused feature map may carry redundant information for purposes of reductionThe invention adopts a random dimension reduction mode to reduce the dimension of the fusion characteristic diagram P from the channel dimension C of the P s (C s 1792) to randomly select 550 dimensions. By super-pixel p ij ∈P N×550×H×W ((i,j)∈[1,W]×[1,H]) Training as input to the LOF algorithm, calculating the kth distance d of each superpixel k (p ij ) Then find its k-th distance neighborhood. Finally, the local reachable density of each superpixel is calculated according to the formula (3), and the abnormal score is calculated according to the formula (4).
The specific process of the test stage is as follows:
step (i) the test data set consists of test pictures in the MVTec AD data set, resized to 256 x 256, and cropped centered to 224 x 224.
And (II) extracting the features of the test image by using a ResNet feature extractor F obtained by pre-training in the step (2) in the training stage. For the abnormal detection task, the abnormal score of the test image is calculated according to the formula (2), and the abnormal image is judged to be an abnormal image when the abnormal score is larger than the threshold th, and the normal image is judged to be a normal image when the abnormal score is smaller than or equal to the threshold th.
And (3) for the abnormal positioning task, extracting the features of the test image by using the feature extractor F in the step (2), fusing the output features of the first layer, the second layer and the third layer, reducing the dimension to 550 dimensions, and calculating the abnormal score of each super pixel after fusion to obtain a pixel-level abnormal score map M of each feature map. And because the resolution of M is less than that of the image to be detected, the M is up-sampled to the size of the image to be detected by adopting bilinear interpolation, and the result is smoothed by using a Gaussian filter. Finally, calculating PR curves of the M and the test image true value image, and taking the balance point of the PR curves as an optimal threshold Th; judging the value smaller than or equal to Th in M as 0, judging the value larger than Th as 1, and judging that 1 represents that the abnormal pixel 0 represents a normal pixel to obtain an abnormal positioning mask image M'; and setting the position of 0 in M' as 0 in the test image to obtain an image abnormal pixel positioning result image.
Example 2
As shown in fig. 4, the system for detecting and positioning an abnormality of an industrial image based on local outlier includes an obtaining module, an extracting module, an abnormality detecting and positioning module, and a testing module, where each module is specifically introduced as follows:
the acquisition module is used for acquiring a training data set.
The training data set consists of normal pictures in the MVTec AD (https:// www.mvtec.com/company/research/datasets/MVTec-AD /) data set, the training set image is resized to 256 × 256, and centered and cropped to 224 × 224.
The extraction module is used for feature extraction.
In this embodiment, a ResNet model obtained by pre-training on an ImageNet dataset is used as a feature extractor F, and the first three layers of the feature extractor are denoted as F 3 . For a given image x i Extracted feature f i ：f i ＝F 3 (x i ). And during training, extracting and storing the characteristics of all training images. During testing, only the features of the target image are extracted.
And the anomaly detection and positioning module adopts a local outlier factor algorithm to carry out anomaly detection and positioning.
The module is divided into two parts, wherein one part is the third-layer output characteristic f of the ResNet network i And is trained as an input to the LOF algorithm. And the other part fuses and reduces dimensions of the first layer output feature, the second layer output feature and the third layer output feature of the ResNet network, the size of the fused feature map is 1792 multiplied by 56, the dimension of each super pixel on the feature map after dimension reduction is 550 multiplied by 1, and finally each super pixel is used as the input of the LOF algorithm for training.
The abnormality detection process is as follows:
outputting the features F with the third layer of the feature extractor F i And is trained as an input to the LOF algorithm. Each f is i Consider a data point, calculate its kth distance, and then find the kth distance neighborhood for that data point. And finally, calculating the local reachable density of each data point, and calculating a local outlier factor to serve as an abnormal score. The specific calculation method is as follows:
calculating the data point f i Is the kth distance of f i Point k to f i Has a Euclidean distance of d k (f i )。
Find f i K distance neighborhood N k (f i ) I.e. f i All points within the kth distance of (c), including the kth distance. | N k (f i ) And | is more than or equal to k, |, represents the number of elements in the set, and the value of k in the invention is 20.
Calculate the local achievable density for each data point:
Figure BDA0003649958870000071
wherein, reach _ dist k (f i T) is f i And N k (f i ) The reachable distance of a certain point t is calculated by the following formula: reach _ dist k (f i ,t)＝max{d k (f i ),d(f i T) }; d (·,. cndot.) represents the Euclidean distance, t is N k (f i ) Internal removal of f i At some point outside.
Calculating f i The local outlier factor of (a) is taken as the outlier score:
Figure BDA0003649958870000072
LOF k (f i ) Representing point f i Neighborhood point N of k (f i ) Local reachable density and point f i Is averaged over the locally achievable density ratio. During testing, the local outlier factor is used as the abnormal score of the tested sample, the threshold th is 0.5, the sample which is larger than the threshold th is judged as an abnormal sample, and the sample which is smaller than the threshold th is judged as a normal sample.
The anomaly locating process is as follows:
selecting the first, second and third layers of output features of the feature extractor F to be fused to obtain a fused feature map
Figure BDA0003649958870000073
Where N is the number of input training images, C s Represents the sum of the first, second and third output channels, H, W represents the maximumResolution of large output signatures. Firstly, amplifying each channel of a first, a second and a third layers of output feature maps of F into a feature map with the resolution of H multiplied by W, and then connecting the output feature maps from different layers front and back to obtain a fusion feature map containing information of different semantic levels and resolutions, thereby realizing coding of different granularities and global contexts.
The fusion characteristic diagram possibly carries redundant information, and in order to reduce the calculation complexity of the model and reduce the memory occupation, the invention adopts a random dimension reduction mode to reduce the dimension of the fusion characteristic diagram P from the channel dimension C of the P s (C s 1792) to randomly select 550 dimensions. By super-pixel p ij ∈P N×550×H×W ((i,j)∈[1,W]×[1,H]) Trained as input to the LOF algorithm, calculates the kth distance d for each superpixel k (p ij ) Then find its k-th distance neighborhood. Finally, the local reachable density of each super-pixel is calculated according to the formula (3), and the abnormal score is calculated according to the formula (4).
The test module specifically comprises the following components:
the test data set consists of test pictures in the MVTec AD data set, resized to 256 × 256, and cropped centered to 224 × 224.
The features of the test image are extracted using a pretrained ResNet feature extractor F. For the abnormal detection task, the abnormal score of the test image is calculated according to the formula (2), and the abnormal image is judged to be an abnormal image when the abnormal score is larger than the threshold th, and the normal image is judged to be a normal image when the abnormal score is smaller than or equal to the threshold th.
For the abnormal positioning task, extracting the features of the test image by using a feature extractor F, fusing the output features of the first layer, the second layer and the third layer, reducing the dimension into 550 dimensions, and calculating the abnormal score of each super pixel after fusion to obtain a pixel-level abnormal score map M of each feature map. Because the resolution of M is less than that of the image to be measured, M is up-sampled to the size of the image to be measured by adopting bilinear interpolation, and the result is smoothed by using a Gaussian filter. Finally, calculating PR curves of the M and the test image true value image, and taking a balance point of the PR curves as an optimal threshold Th; judging the value smaller than or equal to Th in M as 0, judging the value larger than Th as 1, and judging that 1 indicates that the abnormal pixel 0 represents a normal pixel to obtain an abnormal positioning mask image M'; and setting the position of 0 in M' as 0 in the test image to obtain an image abnormal pixel positioning result image.
According to the method, the third-layer output of the pre-trained residual error network is used as an image-level feature, the abnormal score of the detected image is calculated by using the local outlier, the detected image is judged to be abnormal if the abnormal score is larger than a threshold value, otherwise, the detected image is normal, and abnormal detection is realized. In order to accurately position abnormal pixels in an image, the output characteristics of the first three layers of the residual error network are fused to obtain a characteristic diagram, the abnormal score of each super pixel in the characteristic diagram is calculated to obtain an abnormal score diagram of a pixel level, the abnormal score diagram is sampled, an abnormal pixel mask diagram is output, and an abnormal pixel positioning result diagram of the image is obtained through the mask diagram.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (8)

1. An industrial image anomaly detection and positioning method based on local outlier factors is characterized by comprising the following steps:
step (1), acquiring a training data set;
step (2), feature extraction;
and (3) adopting a local outlier factor algorithm to carry out anomaly detection and positioning.
2. The method for detecting and positioning the industrial image anomaly based on the local outlier factor as claimed in claim 1, wherein the step (1) is as follows:
the training data set consists of normal pictures, the training set image is resized to 256 × 256 and centered and cropped to 224 × 224.
3. The method for detecting and positioning the industrial image anomaly based on the local outlier factor as claimed in claim 2, wherein the step (2) is specifically as follows:
the ResNet model obtained by pre-training is used as a feature extractor F, and the first three layers of the feature extractor are recorded as F 3 (ii) a For a given image x i Extracted feature f i ，f i ＝F 3 (x i )。
4. The method for detecting and positioning the industrial image anomaly based on the local outlier factor as claimed in claim 3, wherein the step (3) is divided into two parts:
the first part is anomaly detection: third-layer output characteristic f of ResNet network i Training as input to the LOF algorithm;
and the second part is positioned abnormally: firstly, the first layer output feature, the second layer output feature and the third layer output feature of the ResNet network are fused and dimension reduction is carried out, the size of a fused feature graph is 1792 multiplied by 56, the dimension of each super pixel on the feature graph after dimension reduction is 550 multiplied by 1, and finally, each super pixel is used as the input of an LOF algorithm for training.
5. The method as claimed in claim 4, wherein the anomaly detection process comprises the following steps:
calculating the data points f i Is the kth distance of f i Point k to f i Has a Euclidean distance of d k (f i )；
Find f i K-th distance neighborhood N k (f i ) I.e. f i All points within the kth distance of (a), including the kth distance; | N k (f i ) L is more than or equal to k, L represents the number of elements in the set, k is more than 0 and less than or equal to L, and L is the total number of samples;
calculate the local reachable density for each data point:
Figure FDA0003649958860000011
wherein, reach _ dist k (f i T) is f i And N k (f i ) The reachable distance of a certain point t is calculated according to the formula:
reach_dist k (f i ,t)＝max{d k (f i ),d(f i t) }; d (·,. cndot.) represents the Euclidean distance, t is N k (f i ) Internal removal of f i A point outside;
calculating f i The local outlier factor of (a) is taken as the outlier score:
Figure FDA0003649958860000021
LOF k (f i ) Representing point f i Neighborhood point N of k (f i ) Local achievable density and point f i Average of the ratio of local achievable densities of (a); during testing, the local outlier factor is used as the abnormal score of the tested sample, the threshold is set as th (0 < th < 1), the sample which is larger than the threshold th is judged as an abnormal sample, and the sample which is smaller than the threshold th is judged as a normal sample.
6. The method for detecting and positioning the industrial image anomaly based on the local outlier factor as claimed in claim 5, wherein the anomaly positioning process comprises the following steps:
selecting the first, second and third layers of output features of the feature extractor F in the step (2) to be fused to obtain a fused feature map
Figure FDA0003649958860000022
Where N is the number of input training images, C s Representing the sum of the first, second and third layers of output channels, H, W representing the resolution of the maximum output characteristic diagram; firstly, amplifying each channel of a first, a second and a third layers of output feature maps of a feature extractor F into a feature map with the resolution of H multiplied by W, then connecting the output feature maps from different layers front and back to obtain a fusion feature map containing information of different semantic levels and resolutions, and realizing coding of different granularities and global contexts;
method for reducing fusion characteristics by adopting random dimension reductionDimension of the feature map P, channel dimension C from the fused feature map P s Selecting 550D, C s 1792; by super-pixel p ij ∈P N×550×H×W Training as input to the LOF algorithm, (i, j) e [1, W]×[1,H]Calculating the kth distance d of each super pixel k (p ij ) Then finding its k-th distance neighborhood; finally, the local reachable density of each superpixel is calculated according to the formula (1), and the abnormal score is calculated according to the formula (2).
7. The method for detecting and positioning the industrial image anomaly based on the local outlier factor as claimed in claim 6, further comprising the step (4) of testing the process specifically as follows:
step (i) resize test image to 256 × 256 and crop it centrally to 224 × 224;
step (II), extracting the features of the test image by using the ResNet feature extractor F obtained by pre-training in the step (2); for the abnormal detection task, calculating the abnormal score of the test image according to the formula (2), judging the abnormal image as an abnormal image if the abnormal score is larger than a threshold th, and judging the normal image as a normal image if the abnormal score is smaller than or equal to the threshold th;
for the abnormal positioning task, extracting the features of the test image by using the feature extractor F in the step (2), fusing and reducing the dimensions of the output features of the first layer, the second layer and the third layer into 550 dimensions, and calculating the abnormal score of each super pixel after fusion to obtain a pixel-level abnormal score map M of each feature map; because the resolution of M is less than that of the image to be detected, the M is up-sampled by adopting bilinear interpolation, and the result is smoothed by using a Gaussian filter; finally, calculating PR curves of the M and the test image true value image, and taking the balance point of the PR curves as an optimal threshold Th; judging the value smaller than or equal to Th in M as 0, judging the value larger than Th as 1, and judging that 1 represents that the abnormal pixel 0 represents a normal pixel to obtain an abnormal positioning mask image M'; and setting the position of 0 in M' as 0 in the test image to obtain an image abnormal pixel positioning result image.
8. An industrial image anomaly detection and positioning system based on local outlier factors is characterized by comprising the following modules:
the acquisition module is used for acquiring a training data set;
an extraction module: for extracting features;
an anomaly detection and location module: and (4) carrying out anomaly detection and positioning by adopting a local outlier factor algorithm.
CN202210547394.7A 2022-05-18 2022-05-18 Local outlier factor-based industrial image anomaly detection and positioning method and system Pending CN114926682A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210547394.7A CN114926682A (en) 2022-05-18 2022-05-18 Local outlier factor-based industrial image anomaly detection and positioning method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210547394.7A CN114926682A (en) 2022-05-18 2022-05-18 Local outlier factor-based industrial image anomaly detection and positioning method and system

Publications (1)

Publication Number Publication Date
CN114926682A true CN114926682A (en) 2022-08-19

Family

ID=82808603

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210547394.7A Pending CN114926682A (en) 2022-05-18 2022-05-18 Local outlier factor-based industrial image anomaly detection and positioning method and system

Country Status (1)

Country Link
CN (1) CN114926682A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116682043A (en) * 2023-06-13 2023-09-01 西安科技大学 SimCLR-based unsupervised depth contrast learning abnormal video cleaning method
CN117274148A (en) * 2022-12-05 2023-12-22 魅杰光电科技(上海)有限公司 Unsupervised wafer defect detection method based on deep learning

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117274148A (en) * 2022-12-05 2023-12-22 魅杰光电科技(上海)有限公司 Unsupervised wafer defect detection method based on deep learning
CN116682043A (en) * 2023-06-13 2023-09-01 西安科技大学 SimCLR-based unsupervised depth contrast learning abnormal video cleaning method
CN116682043B (en) * 2023-06-13 2024-01-26 西安科技大学 SimCLR-based unsupervised depth contrast learning abnormal video cleaning method

Similar Documents

Publication Publication Date Title
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN110189255B (en) Face detection method based on two-stage detection
CN114926682A (en) Local outlier factor-based industrial image anomaly detection and positioning method and system
CN111126453A (en) Fine-grained image classification method and system based on attention mechanism and cut filling
CN110059728B (en) RGB-D image visual saliency detection method based on attention model
CN111402237B (en) Video image anomaly detection method and system based on space-time cascade self-encoder
CN111768388A (en) Product surface defect detection method and system based on positive sample reference
CN112258537B (en) Method for monitoring dark vision image edge detection based on convolutional neural network
CN112465759A (en) Convolutional neural network-based aeroengine blade defect detection method
CN114742799B (en) Industrial scene unknown type defect segmentation method based on self-supervision heterogeneous network
CN110310305B (en) Target tracking method and device based on BSSD detection and Kalman filtering
CN112419317B (en) Visual loop detection method based on self-coding network
CN111539456B (en) Target identification method and device
CN115187921A (en) Power transmission channel smoke detection method based on improved YOLOv3
CN114299383A (en) Remote sensing image target detection method based on integration of density map and attention mechanism
CN115861210B (en) Transformer substation equipment abnormality detection method and system based on twin network
EP2423850B1 (en) Object recognition system and method
CN117975267A (en) Remote sensing image change detection method based on twin multi-scale cross attention
CN114820541A (en) Defect detection method based on reconstructed network
CN113536971B (en) Target detection method based on incremental learning
CN114463300A (en) Steel surface defect detection method, electronic device, and storage medium
CN112419243B (en) Power distribution room equipment fault identification method based on infrared image analysis
CN117372413A (en) Wafer defect detection method based on generation countermeasure network
CN117671299A (en) Loop detection method, device, equipment and storage medium
CN113034359A (en) Intelligent diagnosis method and device for generator set stator and rotor defects, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination