CN116612308A - Abnormal data detection method, device, equipment and storage medium - Google Patents
Abnormal data detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116612308A CN116612308A CN202310314985.4A CN202310314985A CN116612308A CN 116612308 A CN116612308 A CN 116612308A CN 202310314985 A CN202310314985 A CN 202310314985A CN 116612308 A CN116612308 A CN 116612308A
- Authority
- CN
- China
- Prior art keywords
- image
- detected
- clustering
- abnormal data
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 112
- 238000001514 detection method Methods 0.000 title claims abstract description 58
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 239000013598 vector Substances 0.000 claims abstract description 94
- 238000000034 method Methods 0.000 claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 238000012937 correction Methods 0.000 claims description 17
- 230000015654 memory Effects 0.000 claims description 14
- 238000009826 distribution Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 230000009467 reduction Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 238000010845 search algorithm Methods 0.000 claims description 5
- 238000012614 Monte-Carlo sampling Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 10
- 238000013135 deep learning Methods 0.000 abstract description 3
- 238000007781 pre-processing Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 238000000513 principal component analysis Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000011022 operating instruction Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000002779 inactivation Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration by non-spatial domain filtering
-
- G06T5/77—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Abstract
The application discloses a method, a device, equipment and a storage medium for detecting abnormal data, which are applied to the field of deep learning and comprise the following steps: extracting a feature vector of an image to be detected; clustering the feature vectors to obtain a clustering result; judging whether the image to be detected is an abnormal image or not according to the clustering result; if the image to be detected is a normal image, the normal image is secondarily detected by using a reconstruction probability method, and whether abnormal data exist in the normal image is determined. The application provides a two-stage abnormal data detection method based on the combination of clustering and reconstruction probability, which improves the accuracy of abnormal picture information detection. The clustering method is suitable for scenes with larger image feature differences, and has poor effect on images with smaller feature differences, so that after the clustering-based detection method, the method based on the reconstruction probability is used for secondary judgment, and the recognition accuracy of abnormal data is improved.
Description
Technical Field
The present application relates to the field of deep learning, and in particular, to a method, apparatus, device, and storage medium for detecting abnormal data.
Background
In the digital economic age, digital commerce is the core content and important carrier of outward digital economies and is becoming a new engine for international trade growth. The prosperity of digital commerce is the absence of secure and orderly circulation of cross-border data. On one hand, the development of digital trade is accelerated, and free traversing circulation of data is needed; on the other hand, cross-border circulation of data often involves problems such as data security and personal privacy protection.
The existing abnormal data detection method based on unstructured data is generally divided into a clustering method, a reconstruction loss method and a classification method, but all have certain defects: the clustering method mainly performs feature extraction by a dimension reduction method, clusters the extracted features, judges abnormal data based on clustering results, and has a common detection effect when feature differences are smaller; the reconstruction loss method is mainly based on the thought of a self-encoder, the encoder is used for extracting the characteristics of the data, then the decoder is used for restoring the extracted characteristics, and the error of the original data and the restored data is calculated to judge the abnormal data; the classification method mainly uses a network structure (such as CNN (convolutional neural network), LSTM (Long Short-Term Memory network) and the like) based on deep learning to perform feature extraction, and then uses a classification layer to detect abnormal data, wherein the detection effect of the method is general when the data category is extremely unbalanced. Therefore, the existing abnormal data detection method has poor detection effect aiming at the conditions of small feature difference and similar feature types.
Disclosure of Invention
Accordingly, the present application is directed to a method, apparatus, device and storage medium for detecting abnormal data, which solve the problem of poor abnormal data detection effect in the prior art.
In order to solve the above technical problems, the present application provides an abnormal data detection method, including:
extracting a feature vector of an image to be detected;
clustering the feature vectors to obtain a clustering result;
judging whether the image to be detected is an abnormal image or not according to the clustering result;
if the image to be detected is a normal image, performing secondary detection on the normal image by using a reconstruction probability method, and determining whether abnormal data exists in the normal image.
Optionally, after extracting the feature vector of the image to be detected, the method further includes:
performing dimension reduction processing on the feature vector by using a PCA algorithm to obtain a processed feature vector;
correspondingly, the clustering the feature vectors to obtain a clustering result includes:
and clustering the processed feature vectors to obtain the clustering result.
Optionally, the clustering the processed feature vectors to obtain the clustering result, and judging whether the image to be detected is an abnormal image according to the clustering result includes:
cutting the processed feature vector to obtain a sub-vector;
clustering the sub-vectors by using a K-means algorithm to obtain a codebook;
and calculating the distance between the data by using an ADC (analog to digital converter) similar search algorithm based on the codebook, and if the distance is larger than a preset distance, determining that the image to be detected is an abnormal image.
Optionally, the secondary judging of the normal image by using the method of reconstruction probability includes:
obtaining probability distribution by using the trained variation self-encoder;
calculating to obtain the mean value and variance of the normal image after the probability distribution is subjected to Monte Carlo sampling for a plurality of times;
obtaining the reconstruction probability of the normal image according to the mean value and the variance;
when the reconstruction probability is greater than the threshold value, abnormal data exists in the normal image;
and when the reconstruction probability is smaller than the threshold value, the normal image has no abnormal data.
Optionally, the extracting the feature vector of the image to be detected includes:
performing rotation correction on the image to be detected to obtain a preprocessed image;
and extracting the characteristics of the preprocessed image to obtain the characteristic vector.
Optionally, the performing rotation correction on the image to be detected includes:
performing Fourier transform on the image to be detected based on an OpenCV module to obtain bright lines perpendicular to the rows on a spectrogram;
detecting the bright line through hough, and calculating an included angle between the bright line and a horizontal line;
and rotating the image to be detected according to the included angle.
Optionally, the extracting features of the preprocessed image to obtain the feature vector includes:
extracting the feature vector of the preprocessed image by using a pre-trained ViT model.
The application also provides an abnormal data detection device, which comprises:
the feature extraction module is used for extracting feature vectors of the image to be detected;
the clustering module is used for clustering the feature vectors to obtain a clustering result;
the first judging module is used for judging whether the image to be detected is an abnormal image or not according to the clustering result;
and the second judging module is used for carrying out secondary detection on the normal image by using a reconstruction probability method if the image to be detected is the normal image, and determining whether illegal data exist in the normal image.
The present application also provides an abnormal data detection apparatus including:
a memory for storing a computer program;
and the processor is used for realizing the steps of the abnormal data detection method when executing the computer program.
The present application also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described abnormal data detection method.
The application also provides a storage medium for storing a computer program, wherein the computer program realizes the abnormal data detection method when being executed by a processor.
Therefore, the feature vector of the image to be detected is extracted; clustering the feature vectors to obtain a clustering result; judging whether the image to be detected is an abnormal image or not according to the clustering result; if the image to be detected is a normal image, the normal image is secondarily detected by using a reconstruction probability method, and whether abnormal data exist in the normal image is determined. According to the two-stage abnormal data detection method based on the combination of clustering and reconstruction probability, the accuracy of abnormal picture information detection is improved. The clustering method is suitable for scenes with larger image feature differences, and has poor effect on images with smaller feature differences, so that after the clustering-based detection method, the method based on the reconstruction probability is used for secondary judgment, and the recognition accuracy of abnormal data is improved.
In addition, the application also provides an abnormal data detection device, equipment and a storage medium, which have the same beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of detecting abnormal data according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an abnormal data detecting device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an abnormal data detecting apparatus according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a flowchart of detecting abnormal data according to an embodiment of the present application. The method may include:
s101: and extracting the feature vector of the image to be detected.
The execution body of the embodiment is a processor. The present embodiment does not limit whether to perform the preprocessing operation on the image to be detected. For example, the image to be detected may be subjected to a preprocessing operation, or may not be subjected to a preprocessing operation. The present embodiment is not limited to a specific pretreatment operation. For example, the preprocessing operation may be to make rotation correction on the image to be detected; or the preprocessing operation can also be cutting the image to be detected; or the preprocessing operation can also be the edge sharpening processing of the image to be processed; or the preprocessing operation may also be any combination of the above.
Further, in order to extract the accuracy and ensure the efficiency of the feature vector, the extracting the feature vector of the image to be detected may include the following steps:
step 51: performing rotation correction on the image to be detected to obtain a preprocessed image;
step 52: and extracting the characteristics of the preprocessed image to obtain a characteristic vector.
The present embodiment is not limited to the operation of the rotation correction, as long as the rotation correction of the image to be processed can be completed.
Further, in order to ensure the picture correction effect and the correction efficiency, the performing rotation correction on the image to be detected to obtain the preprocessed image may include the following steps:
step 61: performing Fourier transform on the image to be detected based on an OpenCV module to obtain bright lines perpendicular to the rows on the spectrogram;
step 62: detecting a bright line through hough, and calculating an included angle between the bright line and a horizontal line;
step 63: and rotating the image to be detected according to the included angle.
In this embodiment, the rotation correction is performed on the image to be detected using a fourier transform method, which is used to convert the time domain signal into the frequency domain in signal processing. The image can also be regarded as a two-dimensional signal in planar space, so that the image can be fourier transformed, i.e. it is transformed from the spatial domain into the frequency domain. Carrying out Fourier transform on the image to obtain a spectrogram which is also a distribution map of image gradients, and if dark points in the spectrogram are relatively more, making the image softer; if the number of bright points is relatively large, the image must be sharp, i.e., the boundary is distinct and the difference between pixels on both sides is large. For example, the text image of a common cross-plate, the text lines are generally darker and the spaces between the lines are brighter, which naturally creates alternating light and dark features. By fourier transformation, a bright line perpendicular to the line can be generated on the spectrogram, the line is detected by hough (a parameter estimation technique using voting principle), the included angle between the line and the horizontal line is calculated, and then the image to be detected can be corrected by rotating the image to be detected by a proper angle. In this embodiment, the rotation correction uses an OpenCV module (Intel open source computer vision library) in the image field to implement correction of the picture.
The present embodiment does not limit the model for extracting the feature vector. For example, feature vectors may be extracted using a pre-trained ViT model (Visual Transformer, visual task backbone); or may be other CNN (convolutional neural network) -based network models such as VGGNet (Visual Geometry Group, deep convolutional neural network), res net (Residual Neural Network ) network models.
Further, in order to better cluster the images, the feature extraction is performed on the preprocessed images to obtain feature vectors, which may include the following steps:
and extracting the feature vector of the preprocessed image by using a pre-trained ViT model.
The embodiment adopts a ViT model to extract semantic features of the picture, and has the following advantages compared with other network models based on CNN: global features can be obtained from the shallow representation; the jump connection structure in ViT better protects the characterization transfer from the bottom layer to the high layer; viT retains more spatial information; the representation of the shallow and deep layers is more similar; high quality intermediate features can be learned. The training process of the ViT model in this embodiment may include: the large-scale classification dataset pre-trains the ViT model, and the ViT model training steps are as follows:
step one: viT the model uses a transducer model structure to process image data, and in order to enable a transducer (a model that uses the attention mechanism to increase the model training speed) to process image data, the image data is first converted into patterns sequence data, for example, a picture x e R is input H×W×C Wherein H represents long, W represents wide, and C represents high. If the patch size is p, then N patches of images can be created, whereinThe size of each patch is +.>For example, a 224X 3 picture is input according toPatches of size 16X 16 are divided into sections, which result (224/16) 2 =14×14=196 patches, where each patch has a size of 16×16×3.
Step two: the patch is tiled and position information is added (Position embedding, one way to convert discrete variables into a continuous vector representation), the patch is first tiled, then the tiled vector is transformed into specified dimensions by a linear transform layer, for example, 16×16×3 patch tiled dimensions are 1×768, and the dimensions are changed to 1×256 by a 768×256 linear transform layer. The trainable relative position information is adopted as the position information, namely, the position information obtained by training is superimposed on the original vector to obtain patch-embedding.
Step three: the class token is added, and a special label class token is added before the input model Transformer Encoder (encoder), the purpose of which is that the ViT model takes the output of the class token label in Transformer Encoder as the code of the input picture, and the label of the picture is used for loss calculation in the subsequent input MLP module.
Step four: the patch-enabling sequence and the class-token are spliced and then input into Transformer Encoder for training, wherein the Encoder comprises the following modules:
layer Norm: and the normalization layer has the main function of normalizing the output of each layer.
Multi-Head Attention: the multi-head attention mechanism has the main function of extracting semantic features of the patch sequence.
Dropout: and the random inactivation is used for carrying out random inactivation on the nodes in the network structure and preventing the model from being over fitted.
MLP Block: the output layer uses the feedforward network + the activation function (Tanh) +the feedforward network as the output layer of the model.
In this embodiment, the clustering algorithm consumes a relatively large amount of time complexity, and particularly, the clustering algorithm consumes a relatively large amount of time complexity when calculating on a large-scale data set, which is relatively low in efficiency. Therefore, in order to improve the efficiency of the clustering algorithm, the feature vector may be subjected to dimension reduction processing, and the embodiment is not limited to a specific dimension reduction method. For example, feature vectors may be subjected to a dimension reduction process using PCA (Principal Component Analysis ); or the LDA linear discriminant analysis can be adopted to carry out dimension reduction processing on the feature vector.
Further, in order to preserve effective features and improve clustering effect, after extracting the feature vector of the image to be detected, the method may further include the following steps:
step 21: performing dimension reduction processing on the feature vector by using a PCA algorithm to obtain a processed feature vector;
correspondingly, the clustering of the feature vectors to obtain a clustering result includes:
and clustering the processed feature vectors to obtain a clustering result.
S102: and clustering the feature vectors to obtain a clustering result.
The present embodiment does not limit the clustering algorithm for the feature vectors. For example, a k-means (an unsupervised clustering algorithm) clustering algorithm may be employed; alternatively, a KNN (K-nearest-Neighbor) clustering algorithm may be employed. The k-means algorithm consumes a large amount of time complexity when clustering data, and particularly consumes a large amount of time complexity when calculating on a large-scale data set, so that the efficiency is low. Therefore, the present embodiment adopts the methods of data compression and acceleration calculation.
Further, in order to fully ensure the effectiveness of the retained features and the high efficiency of the clustering, the clustering of the processed feature vectors to obtain the clustering result, and judging whether the image to be detected is an abnormal image according to the clustering result may include the following steps:
step 31: and cutting the processed feature vector to obtain a sub-vector.
Step 32: and clustering the sub-vectors by using a K-means algorithm to obtain a codebook.
Step 33: based on the codebook, calculating the distance between the data by using an ADC (analog to digital converter) similar search algorithm, and if the distance is larger than a preset distance, determining the image to be detected as an abnormal image.
In this embodiment, considering that a core problem of clustering is distance calculation, for calculating distances between semantic representations of high-dimensional space in a clustering algorithm, the consumed space-time complexity is high, resulting in low model efficiency, so that during clustering, feature vectors can be subjected to dimension reduction, then a PQ (product quantization, here, a cartesian product) is used, the vectors are represented in a form of a cartesian product, then clustering and compression are performed in each piece of data to obtain a codebook, and then a ADC (Asymmetric Distance Computation) similarity search algorithm is used to calculate similarity between the data to obtain a final clustering result, and whether the obtained result is an abnormal picture is judged.
For a better understanding of the clustering process of this embodiment, examples are as follows: the dimension of the input vector is 128, and the input vector is segmented into 4 subspaces, and then the dimension of each subspace is 32 dimensions. Each subspace is clustered (e.g., into 16 classes) using K-means, so that each subspace can obtain a codebook. So that each sub-segment of the sample can be identified with the cluster center ID (index 1 ,index 2 ,...,index i ) To approximate. When a query vector queries a sample, the sample is divided into identical subsections according to the process of generating a codebook by the sample, then in each subspace, the distances from the subsections to all cluster centers in the subspace are calculated, namely, 4 x 16 distances, for example, the distances from the vector of the sample to the query vector are calculated and encoded as (12, 6, 12, 12), the distances numbered as 12 in the first calculated 16 distances are taken out, and the distances from all subsections are summed up to obtain the asymmetric distance from the sample to the query vector.
S103: and judging whether the image to be detected is an abnormal image or not according to the clustering result.
Clustering is the partitioning of a data set into different classes or clusters according to some specific criteria (e.g., distance) such that the similarity of data objects within a cluster is as large as possible, while the differences between data objects that are not within a cluster are also as large as possible. The clustered data of the same class are gathered together as much as possible, and the data of different classes are separated as much as possible. Because normal data are often large in the field of abnormal data detection, abnormal data are few, and abnormal data types are diversified, normal data samples can be clustered, and abnormal data detection is realized according to a clustering result.
S104: if the image to be detected is a normal image, the normal image is secondarily detected by using a reconstruction probability method, and whether abnormal data exist in the normal image is determined.
Further, in order to accurately detect the abnormal data of the picture, the above method for performing secondary judgment on the normal image by using the reconstruction probability may include the following steps:
step 41: obtaining probability distribution by using the trained variation self-encoder;
step 42: calculating to obtain the mean value and variance of the normal image after carrying out Monte Carlo sampling on the probability distribution for a plurality of times;
step 43: obtaining the reconstruction probability of the normal image according to the mean value and the variance;
step 44: when the reconstruction probability is greater than the threshold value, abnormal data exists in the normal image;
step 45: when the reconstruction probability is smaller than the threshold value, the normal image has no abnormal data.
In this embodiment, a VAE model (variational self-encoder) trained by a normal sample is used to calculate the mean and variance of an image to be detected, so that the sample is subjected to normal distribution, parameters of the distribution are sampled, the reconstruction probability of the sample is calculated based on the sampled distribution, and the reconstruction probability is determined to be abnormal if the reconstruction probability is greater than a threshold value.
The present embodiment can detect the reconstruction probability of abnormal data using a variable self-encoder (VAE) based on vgg 16. The VAE model is a directed probability map model whose posterior probabilities are approximated by a neural network, similar in structure to a self-encoder (AE), unlike a self-encoder, which is a deterministic discriminant model, and a randomly generated model that provides the probabilities after calibration. First useAnd training the VAE model by the normal data set to obtain a trained VAE model. The reconstruction probability formula for calculating the sample in this embodiment isWherein L represents the number of samples; x is x i Image data representing an input image to be detected; />Representing the mean; />Representing the variance.
In this embodiment, for the data set required by the training model, the data in the data set may be randomly disturbed, and may be split into a training set, a development set and a test set according to the ratio of 8:1:1. In cases where the amount of data is not sufficient, optimization may also be considered using data enhancement, K-fold cross-validation. The K-fold cross validation is to divide the data set into K parts, alternately train K-1 parts of the data set as a training set, and test the rest 1 part as test data. The data enhancement method is to expand the data, and the expansion method comprises the following steps:
(1) The picture is scaled to a certain extent.
(2) And (5) intercepting the random position of the picture.
(3) The picture is randomly flipped horizontally or vertically.
(4) And (5) carrying out random angle rotation on the picture.
(5) The picture is subjected to random variations in brightness, contrast and color.
Further, in order to process the abnormal data in time, after the abnormal data exists in the normal image, the method may include the following steps:
and sending the abnormal data information to the client for reminding an auditor to confirm or automatically reject and recording the log.
By applying the abnormal data detection method provided by the embodiment of the application, the feature vector of the image to be detected is extracted; clustering the feature vectors to obtain a clustering result; judging whether the image to be detected is an abnormal image or not according to the clustering result; if the image to be detected is a normal image, the normal image is secondarily detected by using a reconstruction probability method, and whether abnormal data exist in the normal image is determined. According to the two-stage abnormal data detection method based on the combination of clustering and reconstruction probability, the accuracy of abnormal picture information detection is improved. The clustering method is suitable for scenes with larger image feature differences, and has poor effect on images with smaller feature differences, so that after the clustering-based detection method, the method based on the reconstruction probability is used for secondary judgment, and the recognition accuracy of abnormal data is improved. Moreover, the feature vectors are subjected to dimension reduction and clustering by adopting PCA and PQ algorithms, so that the clustering efficiency is improved, and the problem of space-time complexity caused by clustering is solved; moreover, a ViT model based on a transducer network structure is adopted to carry out vector representation on the image to be detected, so that semantic representation of the image is greatly improved; in addition, the image is subjected to rotation correction, so that the effectiveness of extracting vector features can be effectively ensured; and moreover, the reconstruction probability of the abnormal data is detected by adopting a variation self-encoder, so that the accuracy of abnormal detection is ensured.
The abnormal data detection device provided in the embodiment of the present application is described below, and the abnormal data detection device described below and the abnormal data detection method described above may be referred to correspondingly to each other.
Referring to fig. 2 specifically, fig. 2 is a schematic structural diagram of an abnormal data detection device according to an embodiment of the present application, which may include:
the feature extraction module 100 is used for extracting feature vectors of the image to be detected;
the clustering module 200 is used for clustering the feature vectors to obtain a clustering result;
the first judging module 300 is configured to judge whether the image to be detected is an abnormal image according to the clustering result;
and the second judging module 400 is configured to, if the image to be detected is a normal image, perform secondary detection on the normal image by using a method of reconstruction probability, and determine whether illegal data exists in the normal image.
Based on the above embodiment, the abnormal data detecting apparatus may further include:
the dimension reduction module is used for carrying out dimension reduction processing on the feature vector by using a PCA algorithm to obtain a processed feature vector;
accordingly, the clustering module 200 may include:
and clustering the processed feature vectors to obtain the clustering result.
Based on the above embodiment, the clustering module 200 and the first determining module 300 may include:
the segmentation unit is used for segmenting the processed feature vector to obtain a sub-vector;
the clustering unit is used for clustering the sub-vectors by using a K-means algorithm to obtain a codebook;
and the distance calculation unit is used for calculating the distance between the data by using an ADC (analog to digital converter) similar search algorithm based on the code book, and if the distance is larger than a preset distance, the image to be detected is an abnormal image.
Based on the above embodiment, the second determining module 400 may include:
the probability distribution calculation unit is used for obtaining probability distribution from the encoder by utilizing the trained variation;
the mean value and variance calculation unit is used for calculating the mean value and variance of the normal image after the probability distribution is subjected to Monte Carlo sampling for a plurality of times;
a reconstruction probability calculation unit, configured to obtain a reconstruction probability of the normal image according to the mean and the variance;
a first result unit, configured to, when the reconstruction probability is greater than the threshold value, cause abnormal data to exist in the normal image;
and the second result unit is used for enabling the normal image to have no abnormal data when the reconstruction probability is smaller than the threshold value.
Based on the above embodiment, the feature extraction module 100 may include:
the preprocessing unit is used for carrying out rotation correction on the image to be detected to obtain a preprocessed image;
and the feature extraction unit is used for extracting features of the preprocessed image to obtain the feature vector.
Based on the above embodiment, the preprocessing unit may include:
the Fourier transform subunit is used for carrying out Fourier transform on the image to be detected based on an OpenCV module to obtain bright lines perpendicular to the rows on the spectrogram;
an included angle calculating subunit, configured to detect the bright line through hough, and calculate an included angle between the bright line and a horizontal line;
and the rotating subunit is used for rotating the image to be detected according to the included angle.
Based on the above embodiment, the feature extraction unit may include:
and the feature vector extraction subunit is used for extracting the feature vector of the preprocessed image by utilizing a pre-trained ViT model.
The order of the modules and units in the abnormal data detection device can be changed without affecting the logic.
The abnormal data detection device provided by the embodiment of the application is used for extracting the feature vector of the image to be detected through the feature extraction module 100; the clustering module 200 is used for clustering the feature vectors to obtain a clustering result; the first judging module 300 is configured to judge whether the image to be detected is an abnormal image according to the clustering result; the second judging module 400 is configured to, if the image to be detected is a normal image, perform secondary detection on the normal image by using a method of reconstruction probability, and determine whether abnormal data exists in the normal image. According to the two-stage abnormal data detection method based on the combination of clustering and reconstruction probability, the accuracy of abnormal picture information detection is improved. The clustering method is suitable for scenes with larger image feature differences, and has poor effect on images with smaller feature differences, so that after the clustering-based detection method, the method based on the reconstruction probability is used for secondary judgment, and the recognition accuracy of abnormal data is improved. Moreover, the feature vectors are subjected to dimension reduction and clustering by adopting PCA and PQ algorithms, so that the clustering efficiency is improved, and the problem of space-time complexity caused by clustering is solved; moreover, a ViT model based on a transducer network structure is adopted to carry out vector representation on the image to be detected, so that semantic representation of the image is greatly improved; in addition, the image is subjected to rotation correction, so that the effectiveness of extracting vector features can be effectively ensured; and moreover, the reconstruction probability of the abnormal data is detected by adopting a variation self-encoder, so that the accuracy of abnormal detection is ensured.
The abnormal data detecting apparatus provided in the embodiment of the present application will be described below, and the abnormal data detecting apparatus described below and the abnormal data detecting method described above may be referred to correspondingly to each other.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an abnormal data detection apparatus according to an embodiment of the present application, which may include:
a memory 10 for storing a computer program;
a processor 20 for executing a computer program to implement the abnormal data detection method described above.
Memory 10, processor 20, communication interface 31, and communication bus 32. The memory 10, the processor 20, and the communication interface 31 all communicate with each other via a communication bus 32.
In the embodiment of the present application, the memory 10 is used for storing one or more programs, the programs may include program codes, the program codes include computer operation instructions, and in the embodiment of the present application, the memory 10 may store programs for implementing the following functions:
extracting a feature vector of an image to be detected;
clustering the feature vectors to obtain a clustering result;
judging whether the image to be detected is an abnormal image or not according to the clustering result;
if the image to be detected is a normal image, the normal image is secondarily detected by using a reconstruction probability method, and whether abnormal data exist in the normal image is determined.
In one possible implementation, the memory 10 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, and at least one application program required for functions, etc.; the storage data area may store data created during use.
In addition, memory 10 may include read only memory and random access memory and provide instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, where the operating instructions may include various operating instructions for performing various operations. The operating system may include various system programs for implementing various basic tasks as well as handling hardware-based tasks.
The processor 20 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a fpga or other programmable logic device, and the processor 20 may be a microprocessor or any conventional processor. The processor 20 may call a program stored in the memory 10.
The communication interface 31 may be an interface of a communication module for connecting with other devices or systems.
Of course, it should be noted that the configuration shown in fig. 3 does not limit the abnormal data detecting apparatus according to the embodiment of the present application, and the abnormal data detecting apparatus may include more or less components than those shown in fig. 3 or may combine some components in practical applications.
The storage medium provided by the embodiments of the present application will be described below, and the storage medium described below and the abnormal data detection method described above may be referred to correspondingly.
The present application also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the abnormal data detection method described above.
The storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
Finally, it is further noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The above description of the method, the device, the apparatus and the storage medium for detecting abnormal data provided by the present application applies specific examples to illustrate the principles and the implementation of the present application, and the above description of the examples is only used to help understand the method and the core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. An abnormal data detection method, comprising:
extracting a feature vector of an image to be detected;
clustering the feature vectors to obtain a clustering result;
judging whether the image to be detected is an abnormal image or not according to the clustering result;
if the image to be detected is a normal image, performing secondary detection on the normal image by using a reconstruction probability method, and determining whether abnormal data exists in the normal image.
2. The abnormal data detection method according to claim 1, further comprising, after the extracting of the feature vector of the image to be detected:
performing dimension reduction processing on the feature vector by using a PCA algorithm to obtain a processed feature vector;
correspondingly, the clustering the feature vectors to obtain a clustering result includes:
and clustering the processed feature vectors to obtain the clustering result.
3. The abnormal data detection method according to claim 2, wherein the clustering the processed feature vectors to obtain the clustering result, and determining whether the image to be detected is an abnormal picture according to the clustering result includes:
cutting the processed feature vector to obtain a sub-vector;
clustering the sub-vectors by using a K-means algorithm to obtain a codebook;
and calculating the distance between the data by using an ADC (analog to digital converter) similar search algorithm based on the codebook, and if the distance is larger than a preset distance, determining that the image to be detected is an abnormal image.
4. The abnormal data detection method according to any one of claims 1 to 3, wherein the performing the secondary judgment on the normal image using the reconstruction probability method comprises:
obtaining probability distribution by using the trained variation self-encoder;
calculating to obtain the mean value and variance of the normal image after the probability distribution is subjected to Monte Carlo sampling for a plurality of times;
obtaining the reconstruction probability of the normal image according to the mean value and the variance;
when the reconstruction probability is greater than the threshold value, abnormal data exists in the normal image;
and when the reconstruction probability is smaller than the threshold value, the normal image has no abnormal data.
5. The abnormal data detection method according to claim 1, wherein the extracting the feature vector of the image to be detected includes:
performing rotation correction on the image to be detected to obtain a preprocessed image;
and extracting the characteristics of the preprocessed image to obtain the characteristic vector.
6. The abnormal data detection method according to claim 5, wherein the performing rotation correction on the image to be detected includes:
performing Fourier transform on the image to be detected based on an OpenCV module to obtain bright lines perpendicular to the rows on a spectrogram;
detecting the bright line through hough, and calculating an included angle between the bright line and a horizontal line;
and rotating the image to be detected according to the included angle.
7. The abnormal data detection method according to claim 5, wherein the feature extraction of the preprocessed image to obtain the feature vector comprises:
extracting the feature vector of the preprocessed image by using a pre-trained ViT model.
8. An abnormal data detection apparatus, comprising:
the feature extraction module is used for extracting feature vectors of the image to be detected;
the clustering module is used for clustering the feature vectors to obtain a clustering result;
the first judging module is used for judging whether the image to be detected is an abnormal image or not according to the clustering result;
and the second judging module is used for carrying out secondary detection on the normal image by using a reconstruction probability method if the image to be detected is the normal image, and determining whether illegal data exist in the normal image.
9. An abnormal data detecting apparatus, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the abnormal data detection method according to any one of claims 1 to 7 when executing the computer program.
10. A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the abnormal data detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310314985.4A CN116612308A (en) | 2023-03-28 | 2023-03-28 | Abnormal data detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310314985.4A CN116612308A (en) | 2023-03-28 | 2023-03-28 | Abnormal data detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116612308A true CN116612308A (en) | 2023-08-18 |
Family
ID=87680665
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310314985.4A Pending CN116612308A (en) | 2023-03-28 | 2023-03-28 | Abnormal data detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116612308A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116912529A (en) * | 2023-09-12 | 2023-10-20 | 深圳须弥云图空间科技有限公司 | Method and device for detecting camera abnormality based on image |
| CN117218149A (en) * | 2023-11-08 | 2023-12-12 | 南通度陌信息科技有限公司 | Image reconstruction method and system based on self-coding neural network |
-
2023
- 2023-03-28 CN CN202310314985.4A patent/CN116612308A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116912529A (en) * | 2023-09-12 | 2023-10-20 | 深圳须弥云图空间科技有限公司 | Method and device for detecting camera abnormality based on image |
| CN117218149A (en) * | 2023-11-08 | 2023-12-12 | 南通度陌信息科技有限公司 | Image reconstruction method and system based on self-coding neural network |
| CN117218149B (en) * | 2023-11-08 | 2024-02-20 | 南通度陌信息科技有限公司 | Image reconstruction method and system based on self-coding neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113554089B (en) | Image classification countermeasure sample defense method and system and data processing terminal | |
| US20140270489A1 (en) | Learned mid-level representation for contour and object detection | |
| CN116612308A (en) | Abnormal data detection method, device, equipment and storage medium | |
| US20020067857A1 (en) | System and method for classification of images and videos | |
| US20210319240A1 (en) | Generator exploitation for deepfake detection | |
| Sequeira et al. | A realistic evaluation of iris presentation attack detection | |
| US11348364B2 (en) | Method and system for neural fingerprint enhancement for fingerprint recognition | |
| Shrein | Fingerprint classification using convolutional neural networks and ridge orientation images | |
| Pflug et al. | 2D ear classification based on unsupervised clustering | |
| Talab et al. | A novel statistical feature analysis-based global and local method for face recognition | |
| Alhabib et al. | Exploiting wavelet transform, principal component analysis, support vector machine, and k-nearest neighbors for partial face recognition | |
| CN114332500A (en) | Image processing model training method and device, computer equipment and storage medium | |
| Dagher et al. | Improving the SVM gender classification accuracy using clustering and incremental learning | |
| Farsi et al. | Eye gaze detection based on learning automata by using SURF descriptor | |
| CN116884070A (en) | Efficient image processing algorithm and application thereof in face recognition | |
| Zhuang et al. | A handwritten Chinese character recognition based on convolutional neural network and median filtering | |
| CN114565798A (en) | Power device wear fault diagnosis method and system based on ferrographic image analysis | |
| US11475684B1 (en) | Methods and systems for performing noise-resistant computer vision techniques | |
| Chen et al. | A finger vein recognition algorithm based on deep learning | |
| Gu et al. | Selection of optimal features for iris recognition | |
| AU2021101517A4 (en) | A system for object recognition for visual surveillance | |
| CN112070009B (en) | Convolutional neural network expression recognition method based on improved LBP operator | |
| CN111695117B (en) | Webshell script detection method and device | |
| Li et al. | A pre-training strategy for convolutional neural network applied to Chinese digital gesture recognition | |
| Huang et al. | Skew correction of handwritten Chinese character based on ResNet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |