WO2020119169A1

WO2020119169A1 - Computer readable storage medium, input data checking method and computing device

Info

Publication number: WO2020119169A1
Application number: PCT/CN2019/102013
Authority: WO
Inventors: 宋基永
Original assignee: 数优(苏州)人工智能科技有限公司
Priority date: 2018-12-13
Filing date: 2019-08-22
Publication date: 2020-06-18

Abstract

According to an embodiment of the present disclosure, the present disclosure discloses a computer readable storage medium storing a computer program. When executed in more than one processor, the computer program performs the following work for checking input data, the work comprises: the work of exporting a first checking result of the input data; the work of classifying the input data by using a classification model learned by learning data labeled with cluster information; and the work of outputting a final checking result based on at least a portion of the classification result of the input data of the classification model.

Description

Computer readable storage medium, input data checking method and computing device

Technical field

The present disclosure relates to data processing using a computing device, and more particularly, to data processing of artificial intelligence embodied by using a computing device.

Background technique

In order to solve complex or unknown problems, research has been conducted on applying the recognition method known to humans to equipment. As one of such studies, there is a neural network that models human biological nerve cells. Neural networks use algorithms that mimic the learning capabilities held by humans. The neural network performs the mapping between the input mode and the output mode by learning. Also, based on the learning results, the neural network has the generalization ability to generate relatively correct output for input patterns that are not used for learning.

Summary of the invention

The present disclosure is for providing a method of processing data using a computing device.

An embodiment of the present disclosure for solving the problem discloses a computer-readable storage medium that stores a computer program. When the computer program is executed in more than one processor, the following work for checking the input data is performed, the work includes: a work for deriving the results of the first check of the input data; The task of classifying the input data by the classification model learned by the learning data with cluster information; and the work of outputting the final inspection result based on at least a part of the classification result of the input data of the classification model.

Alternatively, the classification model is learned using a learning data set that contains multiple subsets of learning data. It is possible to classify similar data into the same cluster in the learning data contained in the learning data set, and classify non-similar The data is classified into different clusters to learn the classification model.

Alternatively, the work of deriving the first inspection result of the input data may include processing the input data using a pre-learned inspection model containing more than one network function, thereby, the input data Perform anomaly detection or perform anomaly detection on the input data based on the comparison between the input data and the reference data.

Alternatively, the classification model may be learned using a learning data set containing a subset of learning data, the learning data set containing learning data labeled with different cluster information.

Alternatively, the classification model may be learned using a learning data set that includes a subset of learning data, the learning data set including target data, target similar data, and target non-similar data.

Alternatively, the target data and the target similar data may be data marked with first cluster information, and the target non-similar data may be data marked with second cluster information.

Alternatively, in the case where the target data is data containing abnormalities, the target similar data may be data containing types of abnormalities similar to the abnormalities contained in the target data, and the target is not similar The data may be data that does not contain abnormalities.

Alternatively, in the case where the target data is an image containing abnormality, the target similar data may be an image cropped in a manner that includes at least a part of the abnormality included in the target data, and the target is not Similar data can be images that do not contain anomalies.

Alternatively, the target data may be an image containing the abnormality at the center of the image.

Alternatively, the target non-similar data may be an image in which at least a part of the part other than the abnormality of the target data is repeated.

Alternatively, the target data and the target similar data may be classified into the same cluster, and the target non-similar data may be classified into a cluster different from the cluster to which the target data and the target similar data belong. Way to learn the classification model.

Alternatively, the target data includes an image related to an abnormal condition that can occur separated from the object of the object, and the target similar data may include an image having at least a part of the target data.

Alternatively, the target data and the target similar data may be data containing false positives.

Alternatively, outputting the final inspection result based on at least a part of the classification result of the input data of the classification model may include when the classification model classifies the input data as belonging to a specific cluster , Generate the inspection result matching the specific cluster as the second inspection result, and output the final inspection result based on the second inspection result, and, in the case where the classification of the input data fails, convert The output of the first inspection result as a final inspection result.

Alternatively, the second inspection result may include information about at least one of whether there is an abnormality, the location of the abnormality, the type of the abnormality, and the type of misdetection.

Alternatively, classifying the input data by using the classification model learned by the learning data marked with cluster information may include: using the pre-learned classification model to process the input data, so that all The work of mapping the features of the input data to the solution space of the pre-learned classification model; and based on the position of the input data on the solution space and based on whether the input data belongs to the solution space One of the more than one clusters to classify the work of the input data.

Alternatively, the solution space may be composed of more than one dimension, and may contain more than one cluster, and each cluster is based on the characteristics of each target data and the position of the features based on similar data of the target on the solution space. constitute.

An embodiment of the present disclosure for solving the problem discloses an input data checking method executed in a computing device including more than one processor. The method includes the steps of deriving the first inspection result of the input data; the step of classifying the input data by using a classification model learned by the learning data marked with cluster information; and the step of classifying the classification model The step of outputting the final inspection result based on at least a part of the classification results of the input data.

Another embodiment of the present disclosure of the present disclosure for solving the problem discloses a computing device. The computing device includes: more than one processor; and a memory for storing instructions that can be executed in the processor, the processor performs the following tasks: exporting the first inspection result of the input data; by using Classifying the input data by a classification model learned by the learning data marked with cluster information; and outputting a final inspection result based on at least a part of the classification results of the input data of the classification model.

The present disclosure may provide a method of processing data using a computing device.

BRIEF DESCRIPTION

FIG. 1 is a block diagram illustrating a computing device that performs an input data checking method according to an embodiment of the present disclosure.

FIG. 2a is a diagram illustrating a network function according to an embodiment of the present disclosure.

FIG. 2b is a schematic diagram illustrating a convolutional neural network according to an embodiment of the present disclosure.

3a, 3b, 3c, 3d, 3e, and 3f are diagrams showing learning data according to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating a method of training a classification model according to an embodiment of the present disclosure.

FIG. 5 is a diagram showing the solution space of the classification model according to an embodiment of the present disclosure.

6 is a flowchart of a method for checking input data inspection according to an embodiment of the present disclosure.

7 is a block configuration diagram showing a unit for embodying a method of input data inspection according to an embodiment of the present disclosure.

FIG. 8 is a block configuration diagram showing a module for embodying an input data checking method according to an embodiment of the present disclosure.

FIG. 9 is a block diagram showing logic for embodying an input data checking method according to an embodiment of the present disclosure.

FIG. 10 is a block configuration diagram showing a circuit for embodying an input data checking method according to an embodiment of the present disclosure.

11 is a brief and general diagram showing an exemplary computing environment that can be embodied by an embodiment of the present disclosure.

detailed description

Various embodiments are described with reference to the drawings. In this specification, various descriptions are disclosed to provide an understanding of the present invention. However, such an embodiment can be implemented even without such specific description.

The terms "component", "module", "system", etc. used in this specification refer to computer-related entities, hardware, firmware, software, software, and a combination of hardware or execution of software. For example, the component is a procedure executed on a processor, a processor, an object, an execution thread, a program, and/or a computer, but it is not limited thereto. For example, both the application executed on the computing device and the computing device can be components. More than one component may reside in the processor and/or thread of execution. One component can be logicalized in one computer. A component can be distributed between more than two computers. And, such components can be executed from various computer-readable media having various data structures stored internally. For example, a component is based on a signal with more than one data packet (for example, in a logic system, a decentralized system, through data and/or signals from a component interacting with other components, transmitted through a network such as other systems and the Internet Data), to communicate via local and/or remote processing.

At the same time, the term "or" is not an exclusive "or", but includes the "or". That is, "X uses A or B" means one of naturally inclusive substitutions when it is not specifically defined or clearly expressed in context. That is, when X uses A, or X uses B, or X uses A and B, the case of "X uses A or B" applies. Also, the term "and/or" used in this specification includes all possible combinations of the listed related items.

Also, the terms "include" and/or "include" mean the presence of corresponding features and/or structural elements. However, the terms "include" and/or "include" do not exclude the presence or addition of more than one other feature, structural element, and/or combination. In addition, when there is no special definition or the context does not clearly indicate the singular form, the singular number usually means "one or more" in the scope of the specification and the invention.

Those of ordinary skill in the art to which the present invention pertains know that various exemplary logical blocks, structures, modules, circuits, units, logic, and algorithms added to the description of the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both reflect. To clearly illustrate the interchangeability of hardware and software, various exemplary components, blocks, structures, units, logics, modules, circuits, and components have the same functional aspects as above. Whether such functionality is embodied as hardware or software depends on the particular application and design constraints presented in the system. A person of ordinary skill in the technical field to which the present disclosure pertains embodies the illustrated functionality through various methods for each specific application. However, the determination of this embodiment does not go beyond the scope of this explanation.

The description of the disclosed embodiments is provided to enable those of ordinary skill in the art to which this disclosure belongs to utilize or implement this disclosure. Various modifications to this embodiment will be apparent to those of ordinary skill in the art to which this disclosure belongs. The general principles defined herein are applicable to other embodiments without exceeding the scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments described herein. This disclosure needs to be interpreted in the broadest scope consistent with the principles and new features disclosed.

In the present disclosure, network functions and artificial neural networks and neural networks can be exchanged with each other.

The structure of the computing device 100 shown in FIG. 1 is an illustration briefly shown. In an embodiment of the present disclosure, the computing device 100 may include other structures for performing the embodiments of the present disclosure.

The computing device 100 may include a processor 110, a memory 120, a communication module 130, and a camera module 140.

The processor 110 may be composed of more than one core, and may include a central processing unit (CPU) of a computing device, a general graphics processing unit (GPGPU, general purpose graphics processing unit), and a tensor processing unit (TPU, tensor processing unit) ) And other data analysis, deep learning processors. The processor 110 reads the computer program stored in the memory 120 to execute the input data checking method according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 110 may perform calculations for neural network learning. The processor 110 may perform input data processing for deep learning (DN, deep learning), feature extraction from input data, error calculation, and weighting using backpropagation neural network Calculation of neural network learning such as value update. At least one of the CPU, GPGPU, and TPU of the processor 110 may be in the learning of network functions. For example, the CPU and GPGPU work together to learn network functions and use the network functions to classify data. Moreover, in an embodiment of the present disclosure, the processors of a plurality of computing devices are used together to learn network functions and classify data using the network functions. Moreover, the computer program executed by the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

In an embodiment of the present disclosure, the computing device 100 utilizes at least one of CPU, GPGPU, and TPU to decentralize and process network functions. Moreover, in an embodiment of the present disclosure, the computing device 100 decentralizes network functions together with other computing devices for processing.

In an embodiment of the present disclosure, the image processed by the network function may be an image stored in the storage medium of the computing device 100, an image captured by the camera module 140 of the computing device 100, and/or a communication module 130, from the image Images transmitted by other computing devices such as databases. Moreover, in an embodiment of the present disclosure, the image processed by the network function may be an image stored in a computer-readable storage medium (for example, may include flash memory, etc., but the present disclosure is not limited thereto). The computing device 100 receives an image file stored in a computer-readable storage medium through an input-output interface (not shown).

In an embodiment of the present disclosure, the computing device 100 receives image data and uses pre-learned network functions to check the input data. The input data check of the present disclosure may include anomaly detection for determining whether there is an abnormality in the input data. The input data inspection of the present disclosure may include misdetection judgment related to abnormality detection and abnormality detection.

The memory 120 may store a computer program for executing the input data checking method according to an embodiment of the present disclosure. The stored computer program is read and driven by the processor 110.

The communication module 130 transmits and receives an abnormality determination method for performing image data according to an embodiment of the present disclosure to other computing devices, servers, and the like. The communication module 130 transmits and receives data for image data and other embodiments of the present disclosure to other computing devices, servers, and the like. For example, the communication module 130 receives learning image data in a learning image database or the like. And, the communication module 130 may enable communication between a plurality of computing devices, thereby performing learning dispersion of network functions in each of the plurality of computing devices. The communication module 130 enables communication between a plurality of computing devices, so that data classification using network functions can be distributed.

The camera module 140 captures an inspection object to generate image data in order to execute the image data abnormality determination method according to an embodiment of the present disclosure. In an embodiment of the present disclosure, the computing device 100 may include more than one camera module 140.

FIG. 2a is a diagram illustrating a network function 200 according to an embodiment of the present disclosure.

In this specification, the calculation model, neural network, network function, and neural network may have the same meaning. A neural network may be composed of a collection of interconnected computing units generally referred to as "nodes". Such "nodes" may also be called "neurons". The neuron includes at least one or more nodes. The nodes (or neurons) that make up the neural network are connected by more than one "connection unit".

In the neural network, more than one node connected by the connection unit forms the relationship between the input node and the output node. The concept of input node and output node is relativity. For a node, any node that has an output node relationship has a relationship with an input node in the relationship with other nodes, and vice versa. As described above, the relationship between the input node and the output node is generated with the connection unit as the center. At one input node, more than one output node is connected through the connection unit, and vice versa.

In the relationship between the input node and the output node connected by a connection unit, the output node determines the value based on the data input to the input node. Among them, the nodes connecting the input node and the output node with each other may have a weight. The weighting value can be changed, and the neural network can be changed by the user or algorithm in order to perform the required function. For example, in the case where one output node is connected to more than one input node through each connection unit, the output node has the value input to the input node connected to the output node and the value corresponding to each input node The weighted value set by the connection unit determines the output node value based on the weighted value.

As described above, in a neural network, more than one node is connected to each other through more than one connection unit, thereby forming an input node and an output node relationship in the neural network. In the neural network, the characteristics of the neural network are set according to the number of nodes and connection units and the connection relationship between the nodes and connection units, and the weighted value restored to the connection unit. For example, there are the same number of nodes and connection units, and when there are two neural networks with different weighting values between the connection units, the two neural networks are different.

The neural network may include more than one node. A part of the nodes constituting the neural network form a layer based on the distance from the initial input node. For example, starting from the initial input node, the set of nodes with distance n may constitute n layers. The distance from the initial input node is determined by the minimum number of connection units that need to be traversed in order to reach the corresponding node from the initial input node. However, the definition of this layer is used to illustrate that within the neural network, the difference in layers can be defined by a method different from that described above. For example, the layer of a node can be defined by the distance of the final node.

Initially, the input node is a node in the neural network. In the relationship with other nodes, more than one node that directly inputs data without going through the connection unit. Or, in the neural network, in the relationship between the nodes based on the connection unit, the nodes of other input nodes connected through the connection unit are not blocked. Similarly, the final output node is one of the nodes in the neural network, and in the relationship with other nodes, there is more than one node that does not have an output node. In addition, the hidden node is a node that does not constitute the neural network for the first input node and the last output node. In the neural network according to an embodiment of the present disclosure, the number of nodes in the input layer is the same as the number of nodes in the output layer. In the input layer, along the hidden nodes, the number of nodes decreases and increases again. Moreover, in the neural network of another embodiment of the present disclosure, the number of nodes in the input layer is greater than the number of nodes in the output layer, and in the input layer, the number of nodes decreases along hidden nodes. In addition, in the neural network according to another embodiment of the present disclosure, the number of nodes in the input layer is greater than the number of nodes in the output layer. In the input layer, the number of nodes increases along hidden nodes. The neural network of another embodiment of the present disclosure is a neural network of a combined form of neural networks.

A deep neural network (DNN, deep neural network) is a neural network that includes multiple hidden layers in addition to the input layer and the output layer. If the deep neural network is used, the latent structures of the data can be grasped. That is, the underlying structure of photos, texts, videos, voices, and music can be grasped (for example, which objects exist in the photos, what are the contents and feelings of the text, and what are the contents and feelings of the speech, etc.). Deep neural networks may include convolutional neural networks (CNN, convolutional neural networks), recurrent neural networks (RNN, recurrent neural networks), autoencoders (autoencoders), generative adversarial networks (GAN, Generative Adversarial Networks), restricted Boltzmann machine (RBM, restricted boltzmann machine), deep belief network (DBN, deep Belief network), Q network, U network, Siam network, etc. The description of the deep neural network is an example, and the present disclosure is not limited thereto.

In an embodiment of the present disclosure, the network function 200 may further include an auto encoder. The automatic encoder may be one type of artificial neural network for outputting output data similar to the input data. The autoencoder may include at least one hidden layer, and a singular hidden layer may be disposed between the input and output layers. The number of nodes in each layer is reduced to the middle layer called the bottleneck layer (coding) in the number of nodes in the input layer, and then in the bottleneck layer, it is expanded symmetrically with the reduction to the output layer (symmetrical to the input layer) . In this case, the illustration of FIG. 2 shows that the dimension reduction layer and the dimension restoration layer are symmetrical, and the present disclosure is not limited thereto, and the nodes of the dimension reduction layer and the dimension restoration layer may be symmetric or asymmetric. The automatic encoder can perform non-linear dimensionality reduction. The number of input layers and output layers corresponds to the number of sensors remaining after the preprocessor of the input data. In the structure of an autoencoder, the number of nodes of the hidden layer in the encoder decreases in a direction away from the input layer. In the case where the number of nodes in the bottleneck layer (the layer with the fewest nodes between the encoder and the decoder) is too small, a sufficient amount of information cannot be transferred, and therefore, it can be maintained above a certain number (for example, half of the input layer) Etc.).

Neural networks can be learned in at least one of supervised learning, unsupervised learning and semi-supervised learning. The learning of the neural network is used to minimize the output error. During the learning process of the neural network, repeatedly input the learning data to the neural network, and calculate the output and target errors of the neural network related to the learning data, along the direction used to reduce the errors, remove the errors of the neural network from the output layer of the neural network Backward propagation in the direction of the input layer to update the weighted value of each node of the neural network. In the case of supervised learning, learning data marked with correct answers in each learning data (ie, marked learning data) is used, and in the case of unsupervised learning, each learning data is not marked with correct answers. That is, for example, the learning data in the case of supervised learning related to data classification is data marked with categories in the learning data. The labeled learning data is input to the neural network, and the output (category) of the neural network and the label of the learning data are compared to calculate errors. As another example, the input learning data in the case of unsupervised learning related to data classification is compared with the neural network output, thereby calculating errors. Calculated errors propagate backward in the neural network (that is, in the direction of the output layer to the input layer). With the reverse propagation, the connection weight of each node of each layer of the neural network will be updated. The updated connection weight value of each node may change according to the learning rate. The calculation of the neural network related to the input data and the back propagation of errors can constitute a learning loop (epoch). The learning rate can be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, at the beginning of the learning of the neural network, a high learning rate is applied. The neural network quickly ensures a specified level of performance to improve efficiency, and at the later stage of learning, a low learning rate is used to improve accuracy.

In the learning of neural networks, usually, the learning data is a partial collection of actual data (ie, data processed by the learned neural network), therefore, there is a learning cycle that reduces errors related to the learning data or adds errors to the actual data . As described above, overfitting over-learns, resulting in an increase in errors related to actual data. For example, one type of overfitting is that after seeing a yellow cat, the neural network that learned the cat cannot recognize cats other than the yellow cat. Overfitting increases the error of the machine running the algorithm. To prevent this overfitting algorithm, a variety of optimization methods can be used. In order to prevent overfitting, increase the learning data, or regularization (regularization), omitting the dropout of a part of the network nodes during the learning process, etc.

The convolutional neural network (CNN, convolutional neural network) shown in FIG. 2b is a kind of deep neural network, including a neural network with a convolutional layer. Convolutional neural networks are a type of multilayer receptors designed with minimal preprocess. The convolutional neural network can be composed of one or more convolutional layers and an artificial neural network layer combined with this, and the weighted value and pooling layer are additionally used. Because of this structure, the convolutional neural network can fully use the input data of the two-dimensional structure. Convolutional neural networks are used to identify targets in images. The convolutional neural network can divide the image data into rows and columns with dimensions to be processed. For example, in the case of image data encoded in red-green-blue (RGB, red-green-blue), it is divided into two-dimensional (for example, two-dimensional image) rows and columns according to red, green, and blue colors. That is, the color of each pixel of the image data can be divided into components of rows and columns, and the size of the rows and columns is the same as the size of the image. Therefore, the image data can be divided into three two-dimensional rows and columns (three-dimensional data array).

In the convolutional neural network, the convolution filter is moved to multiply the convolution filter and the row and column components at various positions of the image to perform the convolution process (input and output of the convolution array). The convolution filter may be in the form of n*n rows and columns, and generally, it may be composed of a filter of a fixed form that is smaller than the number of overall pixels of the image. That is, when an m*m image is input to a convolution layer (for example, a convolution layer with a size of n*n of a convolution filter), the rows and convolutions of n*n pixels including each pixel of the image are represented The filter components are multiplied (ie, the components of the rows and columns are multiplied). By multiplying with the convolution filter, the components matching the convolution filter can be extracted from the image. For example, in the image, the 3*3 convolution filter used to extract the upper and lower straight line components is composed of [[0, 1, 0], [0, 1, 0], [0, 1, 0]]. The convolution filter is applied to the input image, and the upper and lower straight line components matching the convolution filter are extracted from the image and output. The convolution layer may use a convolution filter in each row and column related to each channel (that is, in the case of R, G, and B coating images, R, G, and B colors). The convolutional layer uses a convolution filter on the input image to extract features in the input image that match the convolution filter. The filter value of the convolution filter (that is, the value of each component of the row and column) is updated by inverse propagation during the learning process of the convolutional neural network. The output of the convolutional layer is connected to the sub-sampling layer, thereby simplifying the output of the convolutional layer to reduce memory usage and calculation values. For example, when the output of the convolutional layer is input to the collection layer having the 2*2 maximum collection filter, each pixel of the image outputs the maximum value included in each patch to compress the image for each 2*2. The aggregate output minimum value of the patch, or the average value of the output patch, any aggregation method may be included in the present disclosure.

The convolutional neural network may include more than one convolutional layer and sub-sampling layer. The convolutional neural network repeatedly performs a convolution process and a sub-sampling process (for example, the maximum aggregation, etc.) to extract features from the image. Through repeated convolution and sub-sampling processes, the neural network can extract the global features of the image.

The output of the convolutional layer or sub-sampling layer can be input to a fully connected layer. A fully connected layer is a layer connected to all neurons of a layer adjacent to all neurons of one layer. A fully connected layer is a structure in a neural network where all nodes of each layer are connected to all nodes of other layers.

In an embodiment of the present disclosure, in order to perform segmentation of image data, the neural network may include a deconvolutional neural network (DCNN). The deconvolution neural network performs similar work to calculate the convolutional neural network in the reverse direction, and outputs the features extracted from the convolutional neural network as features related to the original data.

In an embodiment of the present disclosure, the processor 110 may derive the first inspection result of the input data. The first inspection result may include abnormal information related to the input data, and the abnormal information may include arbitrary information related to the abnormality existing in the input data. For example, the first inspection result may include the presence or absence of an abnormality, the location of the abnormality in the input data, and the type of abnormality. In the case where the input data is an image, the first inspection result may include information about whether the input data is abnormal, which pixel of the input data is abnormal, and what type of abnormality is included in the input data ( For example, in the case of images related to semiconductors, poor soldering, defective wiring, defective components, etc.).

The processor 110 may derive the first check result related to whether there is an abnormality in the input data. The processor 110 processes the input data using a pre-learned inspection model containing more than one network function, thereby performing an abnormality inspection related to the input data to derive the first inspection result. The inspection model is a semi-supervised learning model that judges whether there is an unlearned new pattern (ie, abnormality) in the input data by learning only from normal data, thereby determining whether there is an abnormality in the input data. In addition, the inspection model is learned from the marked normal data and abnormal data, so that it is possible to determine whether the input data contains abnormal supervised learning models. Checking the model may include any machine learning model that determines whether there is an abnormality in the input data.

And, the processor 110 performs abnormality detection related to the input data based on the comparison of the input data and the reference data to derive the first inspection result. When the input data is an image, the processor 110 determines whether there is an abnormality in the input data based on the comparison of the reference data containing the abnormality or the reference data not containing the abnormality and the input data. The processor 110 is based on an arbitrary machine vision (machine vision) solution to judge the abnormality in the input data.

The processor 110 classifies the input data using the classification model learned using the learning data marked with cluster data. The classification model uses learning data sets that contain multiple subsets of learning data to learn. The learning data contained in the learning data set classifies similar data into the same cluster and classifies dissimilar data into different clusters. The learning data subset may contain multiple learning data. The learning data subset may contain multiple learning data, and each learning data is marked with cluster information. In the present disclosure, the learning data is a subset of data used in one iteration in the learning of the network function. In the present disclosure, the learning data set is the entire data used for looping in the learning of the network function. The learning data subset may contain dissimilar learning data, and the dissimilar learning data contained in the learning data subset may be classified into different clusters through a classification model.

That is, the learning data subset includes a plurality of learning data, each of the learning data is labeled with cluster information, and the learning data subset may further include learning data labeled with different cluster information. Specifically, the learning data subset may include target data, target similar data, and target non-similar data, and may be labeled with cluster information such as target data and target similar data, and target non-similar data may be marked with different data from target data and target similar data. Cluster information. The target data and target similar data contained in the learning data subset may be marked with first cluster information, and the target non-similar data may be marked with second cluster information.

For example, when the target data is data containing abnormalities, the target similar data is data containing abnormalities, and the target non-similar data is data containing no abnormalities. The target non-similar data is data in which at least part of the part other than the abnormality is repeated in the target data. For example, in the case where the target data is an image, the target non-similar data is an image of at least a part of the part other than the abnormal part in the image containing the abnormality. And, for example, the target similar data is data including an abnormality similar to the abnormality included in the target data. For example, in the case where the target data is an image containing abnormality, the target similar data is cropped in such a manner as to include at least a part of the abnormality included in the target data. For example, the abnormality of the similar type included in the target data may include the abnormality at the same position on the object of the inspection object of the abnormality contained in the target data, and the same type of defect as the object of the inspection object of the abnormality included in the target data (For example, when the object to be inspected is a circuit board and the abnormality included in the target data is poor soldering, the abnormality included in the target similar data is also poor soldering).

The target similar data may include at least a part of the image of the target data, and the pixel part corresponding to the abnormality in the image of the target data is the data that must be included.

Target non-similar data are images that do not contain abnormalities. Among them, the target data is an image that contains abnormality in the center of the image. The target non-similar data is an image in which at least a part of the part other than the abnormality of the target data is repeated.

When the data is an image, the abnormal part of the image containing the abnormality and the image not containing the abnormality are different, and other parts may be similar, so that in the case of classification by the usual method, it can be classified as a similar image. For example, in the 100*100 pixel image acquired in the semiconductor process, when the abnormal part is 5*5 pixels, and the abnormal part is usually classified at 0.25% of the whole, 99.75% of the two images are similar, Therefore, it is classified as similar by the usual classification method. However, in the case of using the target data, target similar data, and target dissimilar data respectively labeled with cluster information according to an embodiment of the present disclosure to learn a classification model, even if a part of the image is different, it is classified as Non-similar data.

In the classification model, the processor 110 classifies the target data and target similar data into the same cluster, and classifies the target non-similar data into clusters that are different from the cluster to which the target data and target similar data belong.

In the present disclosure, among the abnormalities that the target data can include, in the case of the target data bit image, at least one of the abnormalities related to the inspection target object included in the image and the abnormalities separated from the inspection target object. For example, in the case where the target data is an image acquired from a semiconductor process, the abnormalities that the target data may include may be abnormalities that can occur in semiconductor products (eg, defects, etc.) and abnormalities that are not related to the object of the inspection object (eg, images Acquire at least one of the problem of the device, the lens foreign object, etc.). That is, from the viewpoint of abnormality detection, the target data and the target similar data may include false positives. More specifically, when there is no abnormality in the inspection object and there is a situation that can occur that is separate from the inspection object, the target data and the target similar data may contain such a false alarm. Therefore, the classification model classifies abnormal types and false positives into clusters.

In the case where the input data contains anomalies irrelevant to the target object, the result of the first inspection may contain a false positive. In this case, the input data is classified for the second time using the classification model, and the processor 110 uses the classification model to classify the result of the input data. By using the classification model learned by the learning data containing false positives, the second time In the classification, it can be judged whether it is an abnormality related to the actual inspection object or an abnormality not related to the inspection object (ie, a false alarm).

Hereinafter, the learning data of the present disclosure will be described in detail with reference to the drawings.

3a, 3b, 3c, 3d, 3e, and 3f are diagrams showing learning data according to an embodiment of the present disclosure. The images shown in FIGS. 3 a to 3 e are only examples of learning data, and the present disclosure may include arbitrary learning data and arbitrary images.

FIG. 3a is a diagram showing a target base image 300 that can be used as learning data. The target base image 300 itself can be used as the target image for learning the classification model, and the target base image 300 is cropped for the target image 310. For example, the target base image 300 may include a defect 301 abnormality of the inspection object. The plurality of target

similar images

311, 313 may be images cropped in such a manner as to contain the defective 301 portion of the target image 310. The plurality of

crop lines

310, 311, and 313 in FIG. 3a are crops of the target image 310 and

crops

311 and 313 of the target similar image.

FIG. 3b shows a target image 310 (target image) and target similar images 311, 313 (target similar image) cropped by the cropping line of FIG. 3a. Referring to FIG. 3b, the target image 310 and the target

similar images

311, 313 both contain the abnormality 301, and may include other parts of the inspection object.

FIG. 3c is an illustration showing the target non-similar base image 320. The target non-similar base image 320 shown in FIG. 3c itself can be used as a target dissimilar image, a part of which is cropped and used as a target non-similar image. In the case where a cropped target non-similar image is used in order to learn the classification model of the present disclosure, the target non-similar image 320 may have resolutions such as the target image 310 and the target similar image 311. The resolutions of the target images, target similar images, and target non-similar images are also examples, and the resolutions of these may be the same or different. The target non-similar image 320 may be an image that does not contain abnormalities. The target non-similar image 320 is an image that does not contain abnormality in a portion 321 corresponding to the target image.

FIG. 3d is an illustration showing a subset of learning data used to learn the classification model of an embodiment of the present disclosure.

The learning data subset may include a target image 310 with an anomaly 301, a target similar image 311, and a target non-similar image 320 without an anomaly 321. The target image 310 and the target similar image 311 are marked with the same cluster information in a manner of being classified into the same cluster. The target non-similar image 320 is marked with cluster information that is different from the cluster information marked on the target image 310 and the target similar image 311. For example, the target image 310 and the target similar image 311 may be marked with first cluster information, and the target non-similar image 320 may be labeled with second cluster information.

In the case of using the learning data subset shown in FIG. 3d to learn a classification model, the classification model may classify the images with abnormalities and the images without abnormalities into different clusters.

For example, in the 100*100 pixel image acquired in the semiconductor process, when the abnormal part is 5*5 pixels, and the abnormal part is usually classified at 0.25% of the whole, 99.75% of the two images are similar, Therefore, it is classified as similar by the usual classification method. However, in the case of using the target data, target similar data, and target dissimilar data respectively labeled with cluster information according to an embodiment of the present disclosure to learn a classification model, even if a part of the image is different, it is classified as Non-similar data.

FIG. 3e is an illustrative diagram showing other learning data subsets for learning the classification model of an embodiment of the present disclosure.

The target image 330 and the target similar image 333 of FIG. 3e may be images containing false positives. More specifically, the abnormality 331 included in the target image 330 and the target similar image 333 is separated from the inspection target object and an abnormality condition (for example, a lens foreign object, etc.) can occur. The target image 330 may be an image related to a normal product or an abnormal image that occurs according to an abnormal condition separated from the normal product. According to the conventional abnormality inspection method, when an abnormal image containing such a false alarm is included, it is determined to be abnormal, and an erroneous result is output. However, in an embodiment of the present disclosure, the classification model is used to generate clusters related to false alarms using images containing false alarms, whereby it is possible to detect whether the inspection object included in the input image to be inspected actually contains abnormalities The object to be inspected does not contain anomalies, but for other reasons, the input data is judged to be anomaly detection. In the illustration of FIG. 3e, the target non-similar image 340 is an image that does not contain abnormalities.

In the case of using the learning data subset shown in FIG. 3e to learn a classification model, the classification model may classify images with false alarms and images without false alarms into different clusters.

FIG. 3f is an illustrative diagram showing another learning data subset for learning the classification model of an embodiment of the present disclosure.

The target image 330 and the target similar image 333 of FIG. 3f may be images containing false positives. More specifically, the abnormality 331 included in the target image 330 and the target similar image 333 is an abnormal condition (for example, a lens foreign object, etc.) that can occur when separated from the inspection target object. The target image 330 may be an image related to a normal product or an abnormal image that occurs according to an abnormal condition separated from the normal product. According to the conventional abnormality inspection method, when an abnormal image containing such a false alarm is included, it is determined to be abnormal, and an erroneous result is output. However, in an embodiment of the present disclosure, the classification model is used to generate clusters related to false alarms using images containing false alarms, whereby it is possible to detect whether the inspection object included in the input image to be inspected actually contains abnormalities The object to be inspected does not contain anomalies, but for other reasons, the input data is judged to be anomaly detection. In the illustration of FIG. 3e, the target non-similar image 310 is an image that includes an abnormality 301 (that is, an abnormality related to the inspection target object), and does not include a false alarm. In the case of using the learning data subset shown in FIG. 3f to learn the classification model, the classification model can classify images with false alarms and images with abnormalities in the actual inspection object (ie, images classified as abnormal in the abnormality inspection ) Is classified into different clusters.

The target image 330 including a false alarm is added according to the type of false alarm. In other words, at the industrial site, in the process of executing the abnormality detection solution, when a new misdetection occurs, the new misdetection is used as a target image including a false alarm to additionally learn the classification model. The classification model of the present disclosure can cluster well-known anomaly types and well-known misdetection types, and can determine which type the input data belongs to.

The classification model of the present disclosure clusters similar data on the solution space 400. More specifically, the classification model includes target data 401 and target similar data 402 in one cluster 410, and target non-similar data 403 is included in a different cluster from target data 401 and target similar data 402.

In the solution space of the learned classification model, each cluster has a prescribed distance margin 420.

The classification model receives a subset of learning data containing target data 401, target similar data 402, and target non-similar data 403 to match each data with the solution space, and updates the included classification in a clustered manner based on the cluster information marked on the solution space The weighted value of more than one network function of the model. That is, the classification model increases the distance in the solution space between the target data 401 and the target similar data 402 and the target non-similar data 403 in such a manner that the distance in the solution space of the target data 401 and the target similar data 402 becomes shorter. For example, the classification model uses a triplet-based cost function to learn. Separate the input data from the third input data of other classifications based on the triple cost function, and the first distance between the input data of the same classification (ie, the size of the cluster 410) and one of the input data of the same classification and the third The difference between the second distance between the input data (ie, the distance between 401 or 402 and 403) is at least the distance margin 420, and the method of learning the classification model includes reducing the first distance to the distance margin Steps below the scale. Among them, the distance margin 420 is always a positive number. In order to reach the distance margin 420, the weighted value of more than one network function included in the classification model may be updated, and the weighted value update is performed every iteration or one epoch. Regarding such distance margins, details are disclosed in "FaceNet: A Unified Embedding for Face Recognition and Clustering" of Schroff et al. and Korean Patent Publication 10-2018-0068292 incorporated by reference in this specification.

Moreover, the classification model is learned by a model based on magnet loss considering not only cluster classification of dissimilar data, but also the semantic relationship between data in a cluster or other clusters. In the solution space of the classification model, the initial distance of the center point difference of each cluster is executed during the learning process. After the classification model matches the data on the solution space, the position on the solution space of each data is adjusted based on the similarity to the cluster to which each data belongs and the data within and outside the cluster. The details related to the learning of the classification model based on the magnet loss are incorporated as "METRIC LEARNING WITH ADPTIVE DENSITY DISCRIMINATION" of O. Rippel et al.

FIG. 5 is a diagram showing the solution space 400 of the classification model. The solution space 400 shown in FIG. 5 is merely an example, and the classification model may include any number of clusters and any number of data of each cluster. The

data

431, 433, 441, and 443 in the cluster shown in FIG. 5 are similar data.

In the present disclosure, the solution space is composed of a space of more than one dimension, and includes more than one cluster, and each cluster is constructed based on the features based on the respective target data and the positions on the solution space based on the features based on the target-like data.

In the solution space, the first cluster 430 and the second cluster 440 may be clusters related to dissimilar data. In addition, the third cluster 450 may be a cluster related to data that is not similar to the first and second clusters. The

distance

445, 435 between clusters is a scale that shows the difference in data belonging to each cluster.

The twelfth distance 445 between the first cluster 430 and the second cluster 440 is a measure of the difference between the data belonging to the first cluster 430 and the data belonging to the second cluster 440. In addition, the thirteenth distance 435 between the first cluster 430 and the second cluster 440 is a measure of the difference between the data belonging to the first cluster 430 and the data belonging to the third cluster 450. In the illustration shown in FIG. 5, the data belonging to the first cluster 430 is more dissimilar to the data belonging to the second cluster than the data belonging to the third cluster 450. That is, when the distance between clusters is long, the data belonging to each cluster is more dissimilar, and when the distance between clusters is short, the data belonging to each cluster is more similar. The

distances

435, 445 between several districts are larger than the radius of the cluster by a predetermined ratio. The processor 110 uses the classification model to calculate the input data, thereby classifying the input data based on the position where the feature of the input data matches the solution space of the classification model.

The processor 110 processes the input data using the pre-learned classification model, thereby matching the characteristics of the input data with the solution space of the pre-learned classification model. The processor 110 classifies the input data based on the position on the solution space of the input data, based on which of the one or more clusters on the solution space the input data belongs to.

The processor 110 determines the final inspection result based on at least a part of the classification result related to the input data of the classification model.

In the case where the split model classifies the input data as belonging to a specific cluster, the processor 110 generates the inspection result matching the specific cluster as the second inspection result, and determines the final inspection result with the second inspection result. In addition, when the classification of the classification model and the input data fails, the processor 110 determines the first inspection result as the final inspection result, where the second inspection result may include the presence or absence of the abnormality, the location of the abnormality, and the abnormality Information about at least one of the type, misdetection, and misdetection type.

The pre-learned classification model retains clusters in the solution space according to the type of anomaly. Therefore, the processor 110 uses the classification model to generate the second inspection result related to the input data. The processor 110 determines whether there is an abnormality in the input data based on which cluster on the solution space of the classification model the feature of the input data belongs to, and if there is an abnormality, it can be determined whether the abnormality is an abnormality on the inspection object Or an abnormality that has nothing to do with the object of inspection (ie, false detection). For example, in the case where the position of the input data feature on the solution space belongs to a cluster related to poor soldering of the substrate (ie, using the classification model, the input data is classified as poor soldering of the substrate), the processor 110 may generate The data exists as a second inspection result that is an abnormality of substrate soldering failure, and the final inspection result is determined based on the first inspection result and the second inspection result. And, for example, in the case where the position of the feature of the input data on the solution space belongs to the cluster related to the foreign object of the shooting lens of the target object, the processor 110 may generate a second corresponding to the input data that is a false detection and there is no abnormal normal data The results of the second inspection are based on the results of the first inspection and the second inspection to determine the final inspection results.

In the input data exemplified above, when it is determined that the substrate welding is defective in the first inspection, the results of the first and second inspections are the same. Therefore, the processor 110 determines that the substrate welding defects are related to the input data The final result of anomaly detection. In the illustration, in the case where the input data is classified as other abnormalities in the first inspection (that is, the first and second classification results are different), the processor 110 may inform the operator of the content. The processor 110 marks the corresponding input data in order to inform the operator of the difference between the first inspection result and the second inspection result. In addition, when the first classification result and the second classification result are different, the processor 110 ensembles the first inspection result and the second inspection result to generate a final inspection result. For example, the ensemble method may use a voting method, a method based on the weighted value of each neural network, and may include any other ensemble method.

In addition, in the case where the position of the input data feature on the solution space does not belong to any cluster, the processor 110 determines that the classification using the input data classification model failed, and generates the first inspection result as the final inspection result. For example, in the case where the input data includes a new abnormality related to the inspection target object with a defective circuit wire or an abnormality related to the inspection target object with a broken lens, the processor 110 (that is, the learned classification model has an unlearned new pattern, etc. ), the input data cannot be classified using the classification model, therefore, the final inspection result related to the corresponding input data is generated based on the first inspection result. In this case, the processor 110 may inform the operator of the content. In order to inform the operator of the content, the processor 110 needs to mark the corresponding input data. In addition, when the corresponding input data that cannot be classified by the classification model includes an abnormality, the processor 110 may generate the corresponding input data as new learning data. In other words, in the case where the abnormal data that cannot be classified using the classification model (that is, the case where the first check is abnormal, etc.) is an image, the processor 110 uses the corresponding input data as the target image, and can generate the corresponding The other part of the anomaly of the input data is used as a new learning data subset with similar data as the target. The processor 110 uses the newly generated subset of the learning data to generate clusters on the solution space of the classification model related to the new abnormal pattern and adds the learning classification model.

That is, in an embodiment of the present disclosure, the abnormality of the input data is determined by the first and second inspections, and the input data includes abnormalities related to abnormal conditions that occur through factors other than the object of the inspection object (ie, false positives) ), the inspection results can be derived, not for false detections. In addition, when a new type of misdetection or abnormality occurs, the classification model is learned again by using it, whereby the predetermined inspection performance corresponding to the newly generated misdetection and abnormality can be maintained.

Furthermore, the first inspection and the second inspection are separated, so that adding a new type of abnormality or a new type of misdetection to the abnormality detection solution does not affect the classification performance related to the previous data. That is, in the conventional solution, in the case of repeated false detections, the solution itself is modified to solve the problem of corresponding false detections, and thus other erroneous operations may occur. In the abnormality detection of an embodiment of the present disclosure, it may be broken Open the possibility of additional error work.

The computing device 100 may derive the first inspection result of the input data (step 610). In the present disclosure, the input data may include image data, and the computing device 100 may determine the first inspection result related to the captured image data or the image data transmitted from an external device. The result of the first inspection may contain abnormal information related to the input data.

The computing device 100 classifies the input data by using a classification model learned using the learning data labeled with cluster information (step S630). The classification model uses a learning data set that contains multiple subsets of learning data to learn. In the learning data contained in the learning data set, similar data is classified into the same cluster, and non-similar data is classified into other clusters.

The computing device 100 outputs a final inspection result based on at least a part of the classification result of the input data of the classification model (step 650). In the case where the classification model classifies the input data as belonging to a specific cluster, the computing device 100 generates the inspection result matching the specific cluster as the second inspection result, and determines the final result based on at least a part of the second inspection result test result. And, when the classification model fails to classify the input data, the computing device 100 determines the first inspection result as the final inspection result.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by the following unit.

According to an embodiment of the present disclosure, a method for inspecting input data may be embodied by the following units: a unit 710 that derives the first inspection result of the input data; a classification model learned by using learning data marked with cluster information A unit 730 to classify the input data; and a unit 750 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.

In another embodiment of the method for inspecting input data of the present disclosure, the unit 710 for deriving the first inspection result of the input data may include using a pre-learned inspection model containing more than one network function to The input data is processed to thereby perform abnormality detection on the input data, or a unit that performs abnormality check related to the input data based on the comparison of the input data and reference data.

In another embodiment of the method for inspecting input data of the present disclosure, a unit 750 for outputting the final inspection result based on at least a part of the classification result of the input data of the classification model may be included in the When the classification model classifies the input data as belonging to a specific cluster, the inspection result matching the specific cluster is generated as the second inspection result, and the final inspection result is output based on the second inspection result, and, When the classification of the input data fails, the first inspection result is output as a unit of the final inspection result.

In another embodiment of the present disclosure for checking and magnifying input data, the unit 730 for classifying the input data by using a classification model learned using the learning data marked with cluster information may include: The learned classification model processes the input data, thereby mapping the features of the input data to units of the solution space of the pre-learned classification model; and using the input data on the solution space Based on the location, the units of the input data are classified based on whether the input data belongs to one of more than one cluster on the solution space.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by the following modules.

According to an embodiment of the present disclosure, a method for inspecting input data may be embodied by the following modules: a module 810 for deriving the first inspection result of the input data; a classification model learned by using learning data marked with cluster information A module 830 to classify the input data; and a module 850 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by the following logic.

According to an embodiment of the present disclosure, the method for checking the input data may be embodied by the following logic: Logic 910 that derives the first check result of the input data; by using the classification model learned by the learning data marked with cluster information Logic 930 to classify the input data; and logic 950 to output a final inspection result based on at least a portion of the classification result of the input data of the classification model.

According to an embodiment of the present disclosure, the method for checking the input data may be embodied by the following circuit: a circuit 1010 that derives the first check result of the input data; a classification model learned by using learning data marked with cluster information A circuit 1030 to classify the input data; and a circuit 1050 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.

Those of ordinary skill in the art to which this disclosure belongs will know that various exemplary logical blocks, structures, modules, circuits, units, logic, and algorithm steps related to the embodiments disclosed herein are added through electronic hardware, computer software, or two Combined embodiment. In order to clearly illustrate the interchangeability of hardware and software, various exemplary components, blocks, structures, units, logic, modules, circuits, and components have the same functional aspects as above. Whether such functionality is embodied as hardware or software depends on the specific application and design constraints presented in the system. A person of ordinary skill in the technical field to which the present disclosure pertains embodies the illustrated functionality through various methods for each specific application. However, the determination of this embodiment does not go beyond the scope of this explanation.

This disclosure is described in relation to computer-executable instructions that are generally executed on more than one computer, as long as those of ordinary skill in the technical field to which this disclosure belongs can be embodied by combination with modules and/or a combination of hardware and software.

Generally, program modules include routines, programs, components, data structures, etc. that perform specific tasks or embody specific abstract data types. And, as long as it is a person of ordinary skill in the technical field to which the present disclosure belongs, the method of the present disclosure is known not only in single-processor or multi-processor computer systems, minicomputers, and mainframe computers, but also in personal computers, handheld computing devices, and microprocessor-based Implement other computer systems including devices such as devices, programmable home appliances, and others (these are connected to more than one related device to work).

The embodiments described in the present disclosure may be implemented in a decentralized computing environment executed by a remote processing device connected through a communication network through any task. In a distributed computing environment, program modules can be located in both local and remote storage devices.

Computers generally include a variety of computer-readable media. Through the computer, accessible media can become computer-readable media, such computer-readable media include volatile and non-volatile media, transitory (transitory) and non-transitory (non-transitory) media, mobile And non-removable media. As a non-limiting example, the computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes volatile and non-volatile media media, transitory and non-transitory media embodied by any method or technology that stores information such as computer-readable instructions, data structures, program modules, or other data, Removable and non-removable media. Computer storage media may include RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital video disk (DVD, digital) or other optical disk storage devices, magnetic tape, magnetic tape, magnetic disk storage devices or other magnetic storage devices or Any other medium that accesses and stores required information through a computer, but is not limited to this.

Computer-readable transmission media generally include a medium that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism and transmits all information. Modulated data signal terminology sets or changes more than one signal in the characteristics of the signal in such a manner as to encode information within the signal. As non-limiting examples, computer-readable transmission media include wired media such as a wired network or direct-wired connection, audio, RF, infrared, and other wireless media such as other wireless media. Any combination of such media is also included within the scope of computer-readable transmission media.

An exemplary environment 1100 embodying multiple aspects of the present disclosure including a computer 1102 is presented, which includes a processing device 1104, a system memory 1106, and a system bus 1108. The system bus 1108 connects system components including the system memory 1106 (not limited thereto) to the processing device 1104. The processing device 1104 may be any processor among various commonly used processors. Dual-processor and other multi-processor architectures can also be utilized as the processing device 1104.

The system bus 1108 may be one of several types of bus structures that are connected to each other using an arbitrary local bus in a memory bus, a peripheral device bus, and various common bus architectures. The system memory 1106 includes a read-only memory 2110 and a random access memory 2112. The basic input/output system (BIOS) is stored in non-volatile memory 2110 such as RROM, EPROM, EEPROM, etc., and includes a basic history of transferring information between structural elements in the computer 1102 when the input/output system is started. The random access memory 2112 may include a high-speed random access memory such as a static random access memory for caching data.

The computer 1102 includes an internal hard disk drive 2114 (HDD) (for example, EIDE, SATA, and the internal hard disk drive 2114 is also used as an external type in a suitable chassis (not shown)), and a magnetic floppy disk drive 2116 (FDD) (for example, Read from the removable floppy disk 2118 or use it to record thereon) and the optical disk drive 1120 (for example, read the CD-RON disk 1122 or read from other high-capacity optical media such as DVD or use it to record thereon). The hard disk drive 2114, the magnetic disk drive 2116, and the optical disk drive 1120 may be connected to the system bus 1108 through a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical disk drive interface 1128, respectively. The interface 1124 for external drive embodiment includes at least one or two of Universal Serial Bus (USB, Universal Serial Bus) and IEEE 1394 interface technologies.

These drives and related computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and others. In the case of the computer 1102, the drive and the medium store arbitrary data in an appropriate digital form. The description of the computer-readable medium refers to a hard disk drive, a removable magnetic disk, and a portable optical medium such as a CD or DVD, as long as it is a person of ordinary skill in the technical field to which this disclosure belongs, through a zip drive, a magnetic tape, Other types of media read by computers, such as flash memory cards, cassette tapes, and others, are also used in the exemplary operating environment, and any such media may contain computer-executable instructions for performing the present disclosure.

Multiple program modules including the operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136 may be stored in the drive and the random access memory 2112. All or part of the operating system, applications, modules, and/or data is also cached in the random access memory 2112. The present disclosure may be embodied as an operating system or a combination of operating systems available in various commercial aspects.

The user inputs commands and information to the computer 1102 through more than one wired or wireless input device, such as a pointing device such as a keyboard 2138 and a mouse 1140. Other input devices (not shown) are microphone, IR remote control, joystick, gamepad, stylus, touch screen, etc. These and other input devices are connected to the processing device 1104 through an input device interface 1142 connected to the system bus 1108, and connected through other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc. .

The display 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146 or the like. Attached to the display 1144, the computer usually includes speakers, printers, other peripheral output devices (not shown).

The computer 1102 operates in a networked environment using the logical connection of more than one remote computer, such as a remote computer 1148 via wired and/or wireless communication. The remote computer 1148 may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network nodes. Generally, the computer 1102 includes many of the described structural elements. For simplicity, only the storage device 1150 is shown. The illustrated logical connection includes a short-range communication network 1152 (LAN) and/or a larger network, for example, a wired or wireless connection of a long-range communication network 1154 (WAN). Such short-distance communication and long-distance communication network environments are generally used in offices and companies to make enterprise-wide computer networks such as intranets simple, which are connected to computer networks all over the world, for example, networks.

When used in a short-range communication network environment, the computer 1102 is connected to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156. The adapter 1156 can be wired or wireless for short-range communication 1152. The short-range communication 1152 includes a wireless access point provided for communication with the wireless adapter 1156. When used in a long-distance communication network environment, the computer 1102 may include a modem 1158, connected to a communication computing device on the long-distance communication 1154, or set other units of communication through the long-distance communication 1154 through a network or the like. The modem 1158, which may be an internal or external type and a wired or wireless device, is connected to the system bus 1108 through a serial interface 1142. In a networked environment, for the computer 1102, the described program module or a part thereof is stored in the remote memory/storage device 1150. The illustrated network connection is an exemplary embodiment, and other units for setting a communication connection between computers may also be used.

The computer 1102 performs any wireless device or individual configured and operated by wireless communication, for example, a printer, a scanner, a desktop computer and/or a portable computer, a personal data assistant (PDA, portable data), a communication satellite, and a wireless detection mark Related to any device or place and telephone communication work. This includes at least wireless fidelity and Bluetooth wireless technology. Therefore, as in the conventional network, the communication may be a predetermined structure or ad hoc communication of at least two devices.

Wireless fidelity (Wi-Fi, Wireless, Fidelity) can be connected through the network even if it is not wired. Wireless fidelity is such a device, for example, the wireless technology of a telephone in which a computer can send and receive data anywhere in the conversation circle of a base station, that is, a computer. The wireless fidelity network is reliable and uses IEEE 802.11 (a, b, g, other) wireless technologies to provide high-speed wireless connections. In order to connect the computer to the network and the wired network (using IEEE 802.3 or Ethernet), wireless fidelity is used. The wireless fidelity network operates in the unlicensed 2.4 and 5 GHz radio frequency bands, for example, at a data rate of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include dual bands.

Those of ordinary skill in the technical field to which this disclosure pertains know that information and signals are represented by a variety of different technologies and construction methods. For example, it can be embodied by data, instructions, instructions, information, signals, bits, flags and chips, voltages, current electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination of these that can be referred to in the description.

Those of ordinary skill in the art to which this disclosure pertains know that various exemplary logical blocks, modules, processors, units, circuits, and algorithm steps described in relation to the embodiments described herein pass through an electronic hard disk, (for convenience, they are referred to herein It is "software") various forms of programs or design codes or a combination of these. In order to clearly illustrate this mutual interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps are described above in relation to the functions of these. Whether such a function is embodied as hardware or software depends on the design limitations of a particular application and the overall system. A person of ordinary skill in the technical field to which the present disclosure pertains embodies the functions explained in various ways for each specific application, and such embodies are certainly not beyond the scope of the present disclosure.

The various embodiments disclosed herein are embodied by methods, devices, or articles using standard transformation and/or functional technologies. The term "article of manufacture" includes a computer program, carrier, or media accessed from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory devices (eg, EEPROMs, cards, strips, key drives, etc.), But it is not limited to this. Also, the various storage media disclosed herein include more than one device and/or other machine-readable media for storing information.

The specific order or hierarchical structure of the steps in the disclosed process is an example of illustrative proximity. Based on the limited order of design, the specific order or hierarchical structure of the steps of processes within the scope of the present disclosure may be rearranged again. The additional method invention claims that the elements of the multiple steps are provided in the order of the samples, and is not limited to the specific order or hierarchical structure disclosed.

The description of the disclosed embodiments is provided so that those of ordinary skill in the art to which the present disclosure pertains can simply utilize or implement the present disclosure. Various modifications to this embodiment will be apparent to those of ordinary skill in the art to which this disclosure belongs, and the general principles defined herein are applicable to other embodiments without exceeding the scope of this disclosure. Therefore, the present disclosure is not limited to the embodiments described herein, and needs to be interpreted in the widest scope consistent with the principles and new features disclosed herein.

Claims

A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed in more than one processor of a computing device, the following work for checking input data is performed, so The work described includes:

The work of exporting the first inspection result of the input data;

Classifying the input data by using the classification model learned by the learning data marked with cluster information; and

The work of outputting the final inspection result based on at least a part of the classification result of the input data of the classification model.
The computer-readable storage medium according to claim 1, wherein the classification model is learned using a learning data set including a plurality of learning data subsets, so that the learning data contained in the learning data set Similar data is classified into the same cluster, and the classification model is learned by classifying dissimilar data into different clusters.
The computer-readable storage medium according to claim 1, wherein the work of deriving the first inspection result of the input data includes using a pre-learned inspection model containing more than one network function to The input data is processed, thereby performing anomaly detection on the input data or performing anomaly detection on the input data based on the comparison between the input data and the reference data.
The computer-readable storage medium of claim 1, wherein the classification model is learned using a learning data set containing a subset of learning data, the learning data set containing learning data labeled with different cluster information.
The computer-readable storage medium of claim 1, wherein the classification model is learned using a learning data set including a subset of learning data, the learning data set including target data, target similar data, and target non-similar data.
The computer-readable storage medium according to claim 5, wherein the target data and the target similar data are data marked with first cluster information, and the target non-similar data are marked with second cluster information The data.
The computer-readable storage medium according to claim 5, wherein in the case where the target data is data containing abnormalities, the target similar data is of a type similar to the abnormalities included in the target data Abnormal data, and the target non-similar data is data that does not contain abnormality.
The computer-readable storage medium according to claim 5, wherein in the case where the target data is an image containing abnormality, the target similar data is to include at least a part of the abnormality included in the target data Cropped images, and the target non-similar data is images that do not contain abnormalities.
The computer-readable storage medium according to claim 8, wherein the target data is an image including the abnormality at the center of the image.
The computer-readable storage medium according to claim 8, wherein the target non-similar data is an image in which at least a part of the part other than the abnormality of the target data is repeated.
The computer-readable storage medium according to claim 5, wherein the target data and the target similar data are classified into the same cluster, and the target non-similar data is classified into the target data and The classification model is learned in a manner that the target similar data belongs to different clusters.
The computer-readable storage medium according to claim 5, wherein the target data includes an image related to an abnormal condition that can be separated from the target object, and the target similar data includes at least the target data Part of the image.
The computer-readable storage medium of claim 5, wherein the target data and the target similar data are data containing false positives.
The computer-readable storage medium according to claim 1, wherein the work of outputting the final inspection result based on at least a part of the classification result of the input data of the classification model includes When the input data is classified as belonging to a specific cluster, the inspection result matching the specific cluster is generated as the second inspection result, and the final inspection result is output based on the second inspection result. When the classification of the input data fails, outputting the first inspection result as the final inspection result.
The computer-readable storage medium according to claim 14, wherein the second inspection result includes information about at least one of whether there is an abnormality, an abnormal location, an abnormality type, and a misdetection type.
The computer-readable storage medium according to claim 1, wherein classifying the input data by using a classification model learned using learning data marked with cluster information includes:

The use of the pre-learned classification model to process the input data to map features of the input data to the solution space of the pre-learned classification model; and

The work of classifying the input data is based on the position of the input data on the solution space and based on whether the input data belongs to one of more than one clusters on the solution space.
The computer-readable storage medium according to claim 16, wherein the solution space is composed of a space of more than one dimension, including more than one cluster, and each cluster is based on the characteristics of each target data and the target-based Similar data features are constructed based on the position of the solution space.
An input data checking method, executed in a computing device including more than one processor, is characterized by including:

Steps for exporting the first inspection result of the input data;

The step of classifying the input data by using a classification model learned by the learning data marked with cluster information; and

The step of outputting the final inspection result based on at least a part of the classification result of the input data of the classification model.
A computing device, characterized in that it includes:

More than one processor; and

A memory for storing instructions that can be executed in the processor,

The processor performs the following work:

Export the results of the first inspection of the input data;

Classify the input data by using a classification model learned using learning data marked with cluster information; and

The final inspection result is output based on at least a part of the classification result of the input data of the classification model.