KR20230163183A - Anomaly detection method and appratus using neural network - Google Patents

Abstract

An electronic device for detecting outliers using a neural network is disclosed. An electronic device according to an embodiment includes a memory, a transceiver, and a processor, wherein the processor determines an abnormality corresponding to the input image based on deep features calculated based on an input image and an average of the deep features. Based on a neural network model previously trained to detect values, outliers in the input image can be detected.

Description

Outlier detection method and device using neural network {ANOMALY DETECTION METHOD AND APPRATUS USING NEURAL NETWORK}

The following embodiments relate to an outlier detection method and device using a neural network.

Anomaly detection refers to a data analysis method or process that distinguishes between normal and abnormal samples, and is applied and studied in various fields such as manufacturing, CCTV, medical care, and social networks. Outlier detection methods are useful for detecting abnormal process patterns and can be applied to detect foreign substances in the food industry.

With the recent development of deep learning technology, outlier detection methods using neural networks have been proposed in various application technologies, and the data reconstruction function of the auto-encoder has been applied. The number of cases attempting to detect outliers is increasing.

However, since data restoration performance depends on the degree of compression, such as the code size of the autoencoder, it is important to include how to define the difference between the input and output of the autoencoder and what loss function to use to train the autoencoder. There is a problem that the accuracy of pass/fail judgment may be somewhat low depending on a number of factors. Therefore, there is a need for outlier detection methods using deep learning neural networks to be improved in terms of stability and performance.

The purpose of the embodiments is to provide an outlier detection method and device using a neural network.

However, the technical problems that the present disclosure seeks to solve are not limited to the above-described technical problems, and other technical problems may exist.

An outlier detection device that detects outliers using a neural network according to an embodiment includes: a memory; transceiver; and a processor, wherein the processor detects an outlier corresponding to the input image based on deep features calculated based on the input image and an average of the deep features. , it is possible to detect outliers in the input image.

The neural network model may include an encoder trained in advance to calculate a relationship score using deep features calculated through the encoder and a relationship vector generated based on the average of the deep features as input.

The neural network includes a first encoder that calculates deep features based on the input image; a second encoder that calculates deep features based on the input image; And it may include a third encoder that outputs a relationship score by inputting the deep features calculated from the first encoder and the relationship vector generated based on the average of the deep features.

The neural network model is learned in a way that minimizes the loss value calculated through the loss function of Equation 1, and Equation 1 is: And, the above is the loss function, is the output of the second encoder, is the output of the third encoder, N is the number of training data, The correct answer value of the final output of the neural network for the ith learning data, is a relationship score corresponding to the i-th learning data, and may mean the output of the neural network for the i-th learning data.

An outlier detection method for detecting outliers using a neural network according to an embodiment includes calculating deep features based on an input image; calculating an average of the deep features; and detecting outliers in the input image based on a neural network model previously trained to detect outliers corresponding to the input image based on the average and the deep features.

An electronic device according to one embodiment includes memory; transceiver; and a processor, wherein the processor detects an outlier corresponding to the image based on a deep feature calculated based on an image included in learning data and a loss function calculated based on an average of the deep feature. You can train a neural network model to do this.

1 shows a schematic block diagram of an electronic device for detecting an outlier according to an embodiment.
Figure 2 is a diagram for explaining the encoder included in the neural network model.
Figure 3 is a diagram for explaining an outlier detection algorithm performed by an outlier detection device according to an embodiment.
Figure 4 shows the sequence of the algorithm for training a neural network model.
Figures 5 and 6 are diagrams showing the performance of the outlier detection device when normal data is a strawberry jam image.
Figures 7 and 8 are diagrams showing the performance of an outlier detection device when normal data is a fig image.
Figures 9 and 10 are diagrams showing the performance of an outlier detection device when normal data is a mouse gun image.
Figure 11 is a flowchart explaining a method for detecting an outlier according to an embodiment.
FIG. 12 is a flowchart illustrating a method of learning a neural network model used in an outlier detection device according to an embodiment.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are merely illustrative for the purpose of explaining the embodiments according to the concept of the present invention. They may be implemented in various forms and are not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can make various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, for example, a first component may be named a second component, without departing from the scope of rights according to the concept of the present invention, Similarly, the second component may also be referred to as the first component.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. Expressions that describe the relationship between components, such as “between”, “immediately between” or “directly adjacent to”, should be interpreted similarly.

The terminology used herein is only used to describe specific embodiments and is not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate the presence of a described feature, number, step, operation, component, part, or combination thereof, and one or more other features or numbers, It should be understood that this does not exclude in advance the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

Processor in this specification may mean hardware that can perform functions and operations according to each name described in this specification, or may mean computer program code that can perform specific functions and operations. , or it may mean an electronic recording medium loaded with computer program code that can perform specific functions and operations.

In other words, a processor may mean a functional and/or structural combination of hardware for carrying out the technical idea of the present invention and/or software for driving the hardware.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or limited by these examples. The same reference numerals in each drawing indicate the same members.

1 shows a schematic block diagram of an electronic device for detecting an outlier according to an embodiment.

Referring to FIG. 1, the electronic device 10 that detects outliers can detect outliers in an input image. The electronic device 10 that detects outliers may be referred to as an ‘outlier detection device’. Additionally, the electronic device 10 may be referred to as a 'learning device' during the process of training the neural network model described below.

As an example, the outlier detection device 10 may detect a foreign substance such as a red thread as an outlier when normal data is an image containing strawberry jam. The outlier detection device 10 can receive an image as an input and process the image using a neural network model to detect outliers in the image and output an outlier detection result. The outlier detection result may be expressed as an image, a specific value, or a graph, but the technical idea of the present disclosure is not limited thereto. An image includes an image of an object created by refraction or reflection of light, and may mean representing the shape of the object using lines or colors. The image may consist of information in a form that can be processed by a computer, and according to one embodiment, the image may be a hyperspectral image. Hyperspectral analysis is a technology that analyzes material properties by finely separating and detecting light up to the invisible spectrum region. Hyperspectral images can refer to image(s) recorded by dividing an object by wavelength into invisible light.

Neural network models (or artificial neural networks) can include statistical learning algorithms that mimic neurons in biology in machine learning and cognitive science. The outlier detection device 10 according to an embodiment can detect outliers from an input image through a neural network model. A neural network model can refer to an overall model in which artificial neurons (nodes), which form a network through the combination of synapses, change the strength of the synapse connection through learning and have problem-solving capabilities. The neural network model according to one embodiment may have a structure in which a plurality of encoders are combined, and details about the structure of the neural network model will be described later.

Neurons in a neural network model can contain combinations of weights or biases. A neural network model may include one or more layers consisting of one or more neurons or nodes. A neural network model can infer the result to be predicted from arbitrary input by changing the weight of neurons through learning.

Neural network models may include deep neural networks. Neural network models include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN ( Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial) Network), LSM (Liquid State Machine), ELM (Extreme Learning Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network) , a Kohonen Network (KN), and an Attention Network (AN), but those skilled in the art will understand that it may include any neural network, but is not limited thereto.

The outlier detection device 10 may be implemented with a printed circuit board (PCB) such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the outlier detection device 10 may be implemented with an application processor.

Additionally, the outlier detection device 10 may be implemented in a personal computer (PC), a data server, or a portable device.

Portable devices include laptop computers, mobile phones, smart phones, tablet PCs, mobile internet devices (MIDs), personal digital assistants (PDAs), and enterprise digital assistants (EDAs). , digital still camera, digital video camera, portable multimedia player (PMP), personal navigation device or portable navigation device (PND), handheld game console, e-book ( It can be implemented as an e-book) or a smart device. A smart device may be implemented as a smart watch, smart band, or smart ring.

The outlier detection device 10 includes a transceiver 100 and a processor 200. The outlier detection device 10 may further include a memory 300.

The transceiver 100 may transmit or receive hyperspectral data including hyperspectral images. The transceiver 100 may include a transmission interface or a reception interface. Illustratively, the transceiver 100 may output the input hyperspectral data to the processor 200. As another example, the transceiver 100 may output data obtained from input hyperspectral data (for example, vectorized data of a hyperspectral image) to the processor 200.

The processor 200 may detect outliers from hyperspectral data input through a neural network model, and the neural network model may include a structure in which a plurality of encoders are combined.

The processor 200 may process data stored in the memory 300. The processor 200 may execute computer-readable code (eg, software) stored in the memory 300 and instructions triggered by the processor 200 .

The “processor 200” may be a data processing device implemented in hardware that includes a circuit with a physical structure for executing desired operations. For example, the intended operations may include code or instructions included in the program.

For example, data processing devices implemented in hardware include microprocessors, central processing units, processor cores, multi-core processors, and multiprocessors. , ASIC (Application-Specific Integrated Circuit), and FPGA (Field Programmable Gate Array).

The processor 200 can train the neural network model described above and detect outliers from the input hyperspectral data using the learned neural network model. The method of learning the neural network model is explained in more detail through the accompanying drawings below.

The processor 200 inputs normal and/or abnormal sample data (hereinafter referred to as 'data') included in the received hyperspectral data into an encoder included in the neural network model. Deep features can be calculated. Deep features can be defined as variables that are not directly visible in the data but play a role in creating a specific data distribution. The encoder can compress the input into an internal representation, such as deep features. Deep features may be referred to as codes or latent variables.

Processor 200 may calculate an average of deep features of normal and/or abnormal sample data included in the received hyperspectral data. As an example, the processor 200 may calculate deep features of sample data corresponding to N pixels included in a hyperspectral image and calculate an average of the calculated deep features.

The processor 200 may calculate a relationship score using deep features and averages calculated through an encoder included in the neural network model. The relationship score may contain information about the relationship between deep features and the mean. 'Relationship between deep features and the average' may mean a logical relationship between each deep feature and the average.

According to one embodiment, the processor 200 may detect outliers in the received hyperspectral data based on the calculated relationship score. More specifically, if the calculated relationship score is less than a predetermined threshold, the processor 200 may determine that an outlier (for example, a foreign substance in a food image, etc.) has been detected.

The processor 200 may train a neural network model using a loss function calculated based on the relationship score. The loss function according to an embodiment may include a relationship score as a parameter, and details about the loss function according to an embodiment of the present disclosure are described in more detail in the drawings below.

The processor 200 updates parameters (e.g., weights or biases of a neural network model) to minimize the loss value calculated through the loss function, that is, to have high accuracy. direction can be learned. As an example, using a neural network model learned to minimize loss, the processor 200 can determine whether data obtained from an image is an outlier and output an outlier detection result (AD). As an example, the processor 200 may detect foreign substances (outliers) in food using a hyperspectral image and output a foreign substance detection result.

The memory 300 may store instructions (or programs) executable by the processor 200. For example, the instructions may include instructions for executing the operation of the processor and/or the operation of each component of the processor.

The memory 300 may be implemented as a volatile memory device or a non-volatile memory device.

Volatile memory devices may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM).

Non-volatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (MRAM), and Conductive Bridging RAM (CBRAM). , FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM (Nanotube RRAM), Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, molecular electronic memory device, or insulation resistance change memory.

Figure 2 is a diagram for explaining the encoder included in the neural network model.

The neural network model according to one embodiment may be implemented in a form that includes a plurality of encoders included in the auto encoder 20. The autoencoder 20 is an unsupervised learning technique in deep learning and can be defined as a structure for manifold learning. The autoencoder 20 can compress a high-dimensional manifold into a low-dimensional manifold while maintaining deep features of the data. A manifold can be defined as a subspace that can encompass all data with as little error as possible.

The auto encoder 20 may include an encoder 22. Additionally, the auto encoder 20 may include a decoder 24. The auto encoder 20 may have the same number of neurons in the input (x) and output (x') layers, converts the input (x) into a latent variable (z) through the encoder 22, and decoders Through (24), an output (x') similar to the input (x) can be generated.

For example, when normal sample data corresponding to a good product is input to the learned autoencoder 20, the decoder 24 restores it well, so there is almost no difference between the input (x) and the output (x'). When abnormal sample data corresponding to foreign matter is input, the auto encoder 20 restores it as if it were a normal sample, so a large difference between input (x) and output (x') may occur. In this way, abnormal sample data can be detected.

The loss function (L) can be defined as a function that quantifies the difference (loss) between the output value x' (i.e., predicted value) of the autoencoder 20 and the input value x. The autoencoder 20 can be trained to find a weight parameter that minimizes the loss value, which is the result of the loss function. Illustratively, the loss function (L) may be mean squared error (MSE) or cross entropy error (CEE).

Referring to FIG. 2, the loss function (L) can be defined as the square of the L2 norm of the input (x) and output (x'). The neural network model according to one embodiment may be implemented in a structure that includes a plurality of encoders 22 of the pre-trained autoencoder 20. As an example, the auto-encoder 20 may be trained in advance by determining the weight in a direction that minimizes the loss value of the loss function L calculated based on the learning data input x consisting of a hyperspectral image, and the learned auto-encoder The encoder (22) in (20) can be used to implement a neural network model.

Figure 3 is a diagram for explaining an outlier detection algorithm performed by an outlier detection device according to an embodiment.

According to one embodiment, Figure 3 shows a process of outputting an outlier detection result (AD) 46 according to an outlier detection algorithm.

The outlier detection device 10 according to one embodiment uses a neural network model 40 including a first encoder 41, a second encoder 42, and a third encoder 43 to detect the input image 4. Outliers can be detected. The outlier detection device 10 may input the input image(s) 4 (or data obtained from the image(s) 4) to the first encoder 41 and the second encoder 42. . As an example, image(s) 4 may be hyperspectral image(s). The first encoder 41 is part of an auto encoder and may correspond to the encoder 22 in FIG. 2. The second encoder 42 and the third encoder 43 are encoders whose weights have been adjusted through additional learning for the encoder 22. Learning for the second encoder 42 and the third encoder 43 is attached below. This is explained in more detail through the drawings.

The outlier detection device 10 may calculate the average (C) of the deep features of the image 4 calculated through the first encoder 41. Additionally, the outlier detection device 10 may calculate deep features (Z) for the image 4 through the second encoder 42. The outlier detection device 10 may input a relationship vector ([C, Z]) 44 generated based on the calculated average (C) and deep features (Z) to the third encoder 43. The outlier detection device 10 may calculate a relationship score (R) 45 by inputting the previously described relationship vector into the third encoder 43. The outlier detection device 10 may output an outlier detection result 46 based on the calculated relationship score 45. The detection result is a value close to 1 in the case of a normal state in which no outliers are detected, and a value close to 0 in the case of an abnormal state in which an outlier is detected. The relationship score (R) is determined and the outlier detection result (46) is determined and output. You can. As an example, an outlier detection result may be output as a normal state only when the relationship score (R) exceeds a predetermined threshold.

The outlier detection device (or learning device) 10 according to an embodiment can train the neural network model 40 in a way that minimizes the loss value calculated through the loss function according to Equation 1 below. More specifically, the outlier detection device (or learning device) 10 is a process of updating the weights of the second encoder 42 and the third encoder 43 in a direction that minimizes the loss value according to Equation 1. Learning about the neural network model 40 can be performed. During the learning process, the weights for the second encoder 42 and the third encoder 43 may be updated in a direction that minimizes the result of Equation 1, but the weight of the first encoder 41 is as described in FIG. 2 previously. The learning results can remain the same.

In Equation 1, is the output of the second encoder 42, is the output of the third encoder 43, N is the number of learning data, is the relationship score corresponding to the ith learning data and is the final output of the neural network 40, is the correct answer value of the final output of the neural network 40 for the i-th learning data, and means the label for the i-th learning data, may mean a loss function. N is the total number of data and may mean the total number of pixels included in the training data (eg, hyperspectral images).

In one embodiment, may be a label expressed as a value of 0 if the i-th data is an outlier, and as 1 if it is not an outlier but a normal value.

Figure 4 shows the sequence of the algorithm for training the neural network model 40.

Referring to FIG. 4, the input image(s) of the first encoder 41 and the second encoder 42 of the neural network model 40 (e.g., image 4 shown in FIG. 3) x is N The dimension (corresponding to the number of pixels of the input image) and the C dimension (example corresponding to the number of feature values corresponding to each pixel of the input image) are elements of the real number set, and the output y of the neural network model 40 is It may be an element of an N-dimensional real number set. At this time, x may be in the form of a matrix with a size of NxC and y has a size of Nx1.

In the first step, the outlier detection device 10 may receive x as input and output deep feature(s) z through an encoder. For example, the encoder (e.g., the first encoder 41 and the second encoder 42 shown in FIG. 3) may generate hyperspectral image(s) (e.g., the image(s) shown in FIG. 3). When (4)) includes data corresponding to N pixels, x in the C dimension can be input and output as z in the W dimension. N can be a natural number greater than W. The encoder can output deep feature(s) (z) of the data by reducing time and space complexity through dimensionality reduction in the first step.

In the second and third steps, the outlier detection device 10 initially sets c, an element of the W-dimensional real number set, to an empty array through an encoder (e.g., the first encoder 41 shown in FIG. 3). You can use the for statement to calculate the average (c) of each deep feature for i with a value in the range from 1 to W.

In the fourth step, the outlier detection device 10 detects all i in the range from 1 to W. After calculating , the for statement can be terminated.

In the fifth step, the outlier detection device 10 uses the deep feature(s) (z) of the calculated data and the average (c) of each deep feature to create a relationship score (R) (e.g., shown in Figure 3). The relationship score (45)) can be calculated. R is an element of an N-dimensional real number set and may be in the form of a matrix with a size of Nx1.

In the sixth step, the outlier detection device 10 may calculate a loss value (Loss) using a loss function including the variables calculated in the first to fifth steps. The loss value may be a probability loss value.

The outlier detection device 10 updates the weights of the encoders (e.g., the second encoder 42 and the third encoder 43 shown in FIG. 3) in a direction that minimizes the calculated loss value, You can perform learning on a neural network model.

According to an embodiment of the present disclosure, a neural network model can be learned using a relational network according to a relationship-based outlier detection algorithm, and an outlier detection result (AD) can be output.

Figures 5 and 6 are diagrams showing the performance of the outlier detection device when normal data is a strawberry jam image.

5 and 6, all encoders used in the experiment are the same, and when the total data used in the experiment is 100%, the amount of all learning data used in the experiment is 10% of the total data, and all verification data used in the experiment are 10% of the total data. It is assumed that the amount is 40% of the total data, and the amount of all test data used in the experiment is 100% of the total data. Additionally, all results are calculated as the average of each result sampled 5 times.

Referring to Figure 5, the ground truth (GT) 60 shows a red thread included in strawberry jam. The predicted image 62 is the result of a general autoencoder-based outlier detection algorithm and includes not only a red thread at the same location as the actual image but also other outlier detection result values. In contrast, it can be confirmed that the predicted image 64 output using the neural network model according to the present invention is more similar to the actual image 60 than the predicted image 62.

In the case of an outlier detection algorithm that detects foreign substances in strawberry jam according to an embodiment, if it is actual strawberry jam and the prediction result is also strawberry jam, it is True Positive (TP), and if it is actual strawberry jam and the prediction result is not strawberry jam, it is False Negative (FN). , Actual and predicted results can be classified into four situations, including False Positive (FP) if it is not actual strawberry jam but the predicted result is strawberry jam, and True Negative (TN) if it is not actual strawberry jam and the predicted result is strawberry jam. .

Precision is one of the classification evaluation indicators and is an indicator of how accurate the prediction value is. It refers to the ratio of the actual GT being True among those predicted to be True as a result of the algorithm, and the precision can be calculated as TP/(TP+FP).

Sensitivity (Recall) is called recall rate and is one of the classification evaluation indicators, indicating how well the correct answer was actually answered. This refers to the ratio of those whose GT is actually True among those predicted to be actually True. Sensitivity can be calculated as TP/(TP+FN).

In addition to precision and sensitivity, F1-score is also one of the classification evaluation indicators. It can be defined as an indicator that can evaluate the performance of the algorithm as a single value by obtaining the harmonic average of sensitivity and precision. F1-score can be calculated as 2*(Recall*Preicision)/(Recall+Precision). F1-score can be an indicator that can intuitively evaluate the performance of the algorithm when all data corresponding to TP, FN, FP, and TN are unbalanced.

Referring to FIG. 5, the predicted image 62 output according to the baseline outlier detection algorithm corresponding to the comparative example has a precision of 0.485, a sensitivity of 0.734, and an F1-score of 0.584. On the other hand, the predicted image 64 output according to the outlier detection algorithm according to the present invention has a precision of 0.571, a sensitivity of 0.75, and an F1-score of 0.649. Therefore, it can be seen that the proposed outlier detection algorithm shows relatively high performance in all evaluation indicators.

Referring to FIG. 6, the image evaluation results are shown as a graph showing the F1-score according to the threshold.

For the outlier detection algorithms of the present disclosure, it may be more desirable to evaluate performance based on F1-score rather than precision or recall due to data imbalance. The threshold value on the According to the graph in Figure 6, it can be seen that the F1-Score value of the proposed outlier detection algorithm is generally higher than that of the base outlier detection algorithm.

Figures 7 and 8 are diagrams showing the performance of an outlier detection device when normal data is a fig image.

The conditions of the experiment may be the same as previously described in FIG. 5.

Referring to FIG. 7, the ground truth (GT) 80 shows green yarn, yellow paper, black cable ties, green plastic, and bugs included as foreign substances in the fig. The predicted image 82 output according to the baseline outlier detection algorithm corresponding to the comparative example includes not only foreign substances at the same location as the actual image but also other outlier detection result values. In contrast, it can be confirmed that the predicted image 84 output through the neural network model according to the present invention is more similar to the actual image 80 than the predicted image 82.

The predicted image 82 output according to the baseline outlier detection algorithm corresponding to the comparative example has a precision of 0.789, a sensitivity of 0.933, and an F1-score of 0.855. On the other hand, the predicted image 84 has a precision of 0.886, a sensitivity of 0.883, and an F1-score of 0.885. Therefore, it can be seen that the proposed outlier detection algorithm shows relatively high performance according to the F1-score.

Referring to FIG. 8, the image evaluation results are shown as a graph showing the F1-score according to the threshold.

According to the graph in Figure 8, it can be seen that the F1-Score value of the proposed outlier detection algorithm is generally higher than that of the comparative example.

Figures 9 and 10 are diagrams showing the performance of an outlier detection device when normal data is a mouse gun image.

The conditions of the experiment may be the same as previously described in FIG. 5.

Referring to FIG. 9, the actual image (Ground Truth: GT) 100 shows fishing line, tree branches, twine, straw, and thread included as foreign substances in the rat bag. It can be confirmed that the predicted image 104 output using the neural network model according to the present invention is more similar to the actual image 100 than the predicted image 102 corresponding to the comparative example.

Referring to FIG. 9, the prediction image 102 has a precision of 0.651, a sensitivity of 0.822, and an F1-score of 0.727. The predicted image 104 has a precision of 0.649, a sensitivity of 0.838, and an F1-score of 0.731. Therefore, it can be seen that the proposed outlier detection algorithm shows relatively high performance according to the F1-score.

Referring to FIG. 10, the image evaluation results are shown as a graph showing the F1-score according to the threshold.

According to the graph of FIG. 10, it can be seen that the F1-Score value according to the present invention is higher than that of the comparative example.

Figure 11 is a flow chart to explain a method for detecting an outlier according to an embodiment.

At S1101, an electronic device that detects outliers may receive an image. As an example, the image may be hyperspectral images containing information corresponding to N pixels (N is a real number).

At S1102, the electronic device may calculate deep features of data obtained from the image. Deep features of data obtained from an image may be calculated based on the first encoder described above.

At S1103, the electronic device may calculate the average of the deep features. The average of the deep features may be calculated based on the deep features obtained through the first encoder described above.

In S1104, the electronic device may calculate a relationship score using the deep features calculated through the second encoder and the average of the deep features calculated through the previous step. More specifically, the electronic device may calculate the relationship score based on the third encoder described above. The method of calculating the average and relationship scores of deep features may be the same as the method described through the previous drawing.

In S1105, the electronic device may output a final output value including information on whether the electronic device is an outlier. As described above, the final output value may be a value expressing whether or not it is an outlier as 0 or 1.

FIG. 12 is a flowchart illustrating a method of learning a neural network model used in an outlier detection device according to an embodiment.

Referring to FIG. 12, in S1201, the electronic device may calculate deep features of images included in the learning data. More specifically, the electronic device can calculate deep features of images included in the learning data through the first encoder described above. The first encoder may be an encoder of an auto encoder that has undergone prior learning through the method described in FIG. 2 above.

At S1202, the electronic device may calculate an average of the deep features. The average of the deep features may be calculated based on the deep features obtained through the first encoder described above.

In S1203, the electronic device calculates a relationship score by inputting the deep features calculated through the second encoder and the average of the deep features calculated through the previous step to the third encoder, and enters the loss function calculated based on the relationship score. Based on this, a neural network model can be trained. The second and third encoders may be encoders of the auto encoder learned in the manner of FIG. 2, and the weights of the second and third encoders may be updated in a direction that minimizes the loss value of Equation 1, In this process, the weight of the first encoder can be maintained. Learning of a neural network model can be done based on the method described above. The method of calculating the average and relationship scores of deep features may be the same as the method described through the previous drawing.

An electronic device for detecting an anomaly according to an embodiment may be combined with a computer (or computing device) as hardware and include a computer program stored in a computer-readable recording medium to perform steps S1101 to S1203 described above.

It may be implemented as a computing device including at least one processor that executes instructions of programs loaded into memory, and a program including instructions described to execute S1201 to S1208 described above may be loaded into memory. there is.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments and equivalents of the claims also fall within the scope of the following claims.

Claims (6)

An outlier detection device that detects outliers using a neural network,
Memory;
transceiver; and
processor
Including,
The processor,
Detecting outliers in the input image based on deep features calculated based on the input image and a neural network model previously trained to detect outliers corresponding to the input image based on the average of the deep features, Outlier detection device.
According to paragraph 1,
The neural network model is,
An outlier detection device comprising an encoder trained in advance to calculate a relationship score using deep features calculated through an encoder and a relationship vector generated based on an average of the deep features as input.
According to paragraph 2,
The neural network is,
a first encoder that calculates deep features based on the input image;
a second encoder that calculates deep features based on the input image; and
A third encoder that outputs the relationship score using the deep features calculated from the first encoder and the relationship vector generated based on the average of the deep features as input.
Including an outlier detection device.
According to paragraph 3,
The neural network model is,
The loss value calculated through the loss function in Equation 1 is learned in a way that is minimized,
Equation 1 above is:
ego,
remind is the loss function, is the output of the second encoder, is the output of the third encoder, N is the number of training data, is the correct answer value of the final output of the neural network for the ith learning data, is a relationship score corresponding to the i-th learning data, which means the output of the neural network for the i-th learning data, an outlier detection device.
An outlier detection method that detects outliers using a neural network,
calculating deep features based on the input image;
calculating an average of the deep features; and
Detecting outliers in the input image based on a neural network model previously trained to detect outliers corresponding to the input image based on the average and the deep features.
Including an outlier detection method.
As an electronic device,
Memory;
transceiver; and
processor
Including,
The processor,
An electronic device that trains a neural network model to detect outliers corresponding to images based on deep features calculated based on images included in learning data and a loss function calculated based on the average of the deep features. .