CN111325224A

CN111325224A - Computer-readable storage medium, input data checking method, and computing device

Info

Publication number: CN111325224A
Application number: CN201811528293.5A
Authority: CN
Inventors: 宋基永
Original assignee: Shuyou Suzhou Artificial Intelligence Technology Co ltd
Current assignee: Shuyou Suzhou Artificial Intelligence Technology Co ltd
Priority date: 2018-12-13
Filing date: 2018-12-13
Publication date: 2020-06-23
Also published as: KR20200075704A; KR20210020976A; KR102246085B1

Abstract

According to an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is disclosed. The computer program, when executed on one or more processors, performs the following operations for examining input data, the operations comprising: work to derive a result of a first examination of the input data; a work of classifying the input data by a classification model learned by using the learning data labeled with the cluster information; and an operation of outputting a final inspection result based on at least a part of the classification result of the input data to the classification model.

Description

Computer-readable storage medium, input data checking method, and computing device

Technical Field

The present disclosure relates to data processing with a computing device, and more particularly, to artificial intelligence data processing embodied by utilizing a computing device.

Background

In order to solve the complicated or unknown problems, studies have been made to apply the identification method known to humans to devices. As one of such studies, there is a neural network (neural network) that models biological nerve cells of humans. Neural networks utilize algorithms that mimic the learning abilities held by humans. The neural network performs idea (mapping) between an input pattern and an output pattern by learning. The neural network has a generalization capability of generating a relatively accurate output for an input pattern not used for learning based on a learning result.

Disclosure of Invention

The present disclosure is directed to providing a method for processing data with a computing device.

An embodiment of the present disclosure for solving the problems discloses a computer-readable storage medium storing a computer program. The computer program, when executed on one or more processors, performs the following operations for examining input data, the operations comprising: work to derive a result of a first examination of the input data; a work of classifying the input data by a classification model learned by using the learning data labeled with the cluster information; and an operation of outputting a final inspection result based on at least a part of the classification result of the input data to the classification model.

Alternatively, the classification model is learned using a learning data set including a plurality of learning data subsets, the classification model being able to be learned in such a manner that similar data is classified into the same cluster and non-similar data is classified into different clusters among learning data included in the learning data set.

Alternatively, the work of deriving the result of the first inspection of the input data may include work of processing the input data using a pre-learned inspection model including one or more network functions, thereby performing abnormality detection on the input data or performing abnormality detection on the input data based on comparison of the input data with reference data.

Alternatively, the classification model may be learned using a learning data set comprising a subset of learning data, the learning data set comprising learning data labeled with different cluster information.

Alternatively, the classification model may be learned using a learning data set that includes a subset of learning data, the learning data set including target data, target-like data, and target-non-like data.

Alternatively, the target data and the target similar data may be data marked with first cluster information, and the target non-similar data may be data marked with second cluster information.

Alternatively, in the case where the target data is data containing an abnormality, the target-like data may be data containing an abnormality of a type similar to the abnormality contained in the target data, and the target-non-like data may be data containing no abnormality.

Alternatively, in the case where the target data is an image including an abnormality, the target similar data may be an image cropped so as to include at least a part of the abnormality included in the target data, and the target non-similar data may be an image not including an abnormality.

Alternatively, the target data may be an image including the abnormality in a central portion of the image.

Alternatively, the target non-similar data may be an image in which at least a part of a portion other than an abnormality of the target data is repeated.

Alternatively, the classification model can be learned in such a manner that the target data and the target similar data are classified into the same cluster, and the target non-similar data is classified into a cluster different from the cluster to which the target data and the target similar data belong.

Alternatively, the target data may include an image relating to an abnormal situation that may occur separately from a subject object, and the target-like data may include an image having at least a portion of the target data.

Alternatively, the target data and the target-like data may be data containing false positives.

Alternatively, the work of outputting a final inspection result on the basis of at least a part of the classification result of the input data to the classification model may include work of generating an inspection result matching a specific cluster as a second inspection result and outputting a final inspection result on the basis of the second inspection result in a case where the classification model classifies the input data as belonging to the specific cluster, and outputting the first inspection result as a final inspection result in a case where the classification of the input data fails.

Alternatively, the second inspection result may include information on whether or not there is at least one of an abnormality, a location of the abnormality, a type of the abnormality, and a type of false detection.

Alternatively, the work of classifying the input data by a classification model learned using learning data labeled with cluster information may include: work of processing the input data with the pre-learned classification model to map features of the input data to a solution space of the pre-learned classification model; and classifying the input data based on a location of the input data on the solution space and based on whether the input data belongs to one of the one or more clusters on the solution space.

Alternatively, the solution space may be composed of more than one dimension, may include more than one cluster, and each cluster is composed based on the feature based on each target data and on the position of the feature on the solution space based on the similar data of the target.

An embodiment of the present disclosure for solving the problems discloses an input data checking method executed in a computing device including one or more processors. The method comprises the following steps: a step of deriving a result of a first check on the input data; a step of classifying the input data by a classification model learned by using learning data labeled with cluster information; and a step of outputting a final inspection result based on at least a part of a classification result of the input data to the classification model.

Another embodiment of the present disclosure for solving the problems discloses a computing device. The computing device includes: one or more processors; and a memory for storing instructions executable in the processor, the processor performing the following: deriving a first examination result of the input data; classifying the input data by a classification model learned using learning data labeled with cluster information; and outputting a final inspection result based on at least a portion of the classification result for the input data of the classification model.

The present disclosure may provide a method of processing data with a computing device.

Drawings

Fig. 1 is a block configuration diagram illustrating a computing device that performs an input data checking method according to an embodiment of the present disclosure.

Fig. 2a is a diagram illustrating network functions of one embodiment of the present disclosure.

Fig. 2b is a diagram illustrating a convolutional neural network (convolutional neural network) according to an embodiment of the present disclosure.

Fig. 3a, 3b, 3c, 3d, 3e, and 3f are diagrams illustrating learning data of an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating a method of training a classification model of an embodiment of the present disclosure.

FIG. 5 is a diagram illustrating a solution space of a classification model of an embodiment of the present disclosure.

FIG. 6 is a flow chart of a method for inspecting input data in accordance with an embodiment of the present disclosure.

FIG. 7 is a block diagram illustrating elements of a method for input data inspection embodying an embodiment of the present disclosure.

FIG. 8 is a block diagram illustrating modules used to implement an input data inspection method according to an embodiment of the present disclosure.

FIG. 9 is a block diagram illustrating the logic for an input data inspection method embodying an embodiment of the present disclosure.

Fig. 10 is a block configuration diagram showing a circuit for embodying an input data checking method of an embodiment of the present disclosure.

FIG. 11 is a simplified and generalized diagram illustrating an exemplary computing environment in which an embodiment of the present disclosure may be embodied.

Detailed Description

Various embodiments are described with reference to the drawings. In this specification, various descriptions are disclosed for the purpose of providing an understanding of the present invention. However, such an embodiment may be practiced without such specific description.

As used in this specification, the terms "component," "module," "system," and the like are a computer-related entity, either hardware, firmware, software, a combination of software and hardware, or software in execution. For example, a component is, but is not limited to being, a process (procedure) executing on a processor, an object, a thread of execution, a program, and/or a computer. For example, an application executing on a computing device and the computing device can both be a component. More than one component may reside within a processor and/or thread of execution. A component may be logical within a computer. A component may be distributed among more than 2 computers. Also, such components can execute from various computer readable media having various data structures stored therein. For example, components communicate by way of local and/or remote processes in accordance with a signal having more than one data packet (e.g., data and/or signals from one component interacting with another component in a logical system, distributed system, and transmitted over a network such as other systems and the Internet).

Also, the term "or" is not an exclusive "or," but includes "or. That is, "X employs a or B" means one of natural inclusive substitutions unless specifically defined or not explicitly indicated by context. That is, when X uses A, X uses B, or X uses A and B, "X uses A or B" applies. Also, the term "and/or" as used in this specification includes all possible combinations of the listed related items.

Also, the terms "comprises" and/or "comprising" mean the presence of the corresponding features and/or structural elements. Rather, the terms "comprises" and/or "comprising" do not preclude the presence or addition of one or more other features, structural elements, and/or groups thereof. Furthermore, the singular generally means "one or more" in the present specification and claims, unless specifically defined or otherwise clearly indicated herein.

Those of ordinary skill in the art would appreciate that the various illustrative logical blocks, structures, modules, circuits, units, logic, and algorithms described in connection with the embodiments disclosed herein may be embodied as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, structures, units, logic, modules, circuits, and components have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the system. Skilled artisans in the present disclosure may implement the described functionality in varying ways for each particular application. But such implementation decisions should not be made beyond the scope of this interpretation.

The description of the disclosed embodiments is provided to enable any person skilled in the art to which the disclosure pertains to make or use the disclosure. Various modifications to such embodiments will be readily apparent to those skilled in the art to which the disclosure pertains. The general principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Accordingly, the present disclosure is not limited to the embodiments described herein. The present disclosure is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

In the present disclosure, network functions and artificial neural networks (neural networks) may be interchanged.

The structure of the computing device 100 shown in fig. 1 is an example that is schematically illustrated, and in an embodiment of the present disclosure, the computing device 100 may include other structures for performing embodiments of the present disclosure.

The computing device 100 may include a processor 110, a memory 120, a communication module 130, a camera module 140.

The processor 110 may be configured with one or more cores, and may include a processor for data analysis and deep learning, such as a Central Processing Unit (CPU) of a computing device, a General Purpose Graphics Processing Unit (GPGPU), and a Tensor Processing Unit (TPU). The processor 110 reads the computer program stored in the memory 120 to execute the input data checking method according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 110 may perform calculations for neural net learning. The processor 110 may perform a process for input data for learning, a feature (feature) extraction in the input data, an error calculation, a calculation of learning of a neural network using a weight value update of a back propagation (back) neural network, and the like in deep learning (DN). At least one of the CPU, GPGPU, and TPU of processor 110 may be in the learning of network functions. For example, the CPU and GPGPU together handle learning of network functions, data classification using network functions. Also, in an embodiment of the present disclosure, processors of a plurality of computing devices are used together to perform learning of a network function, data classification using the network function. Also, the computer program executed by the computing device of an embodiment of the present disclosure may be a CPU, GPGPU or TPU executable program.

In an embodiment of the present disclosure, the computing device 100 utilizes at least one of a CPU, GPGPU, and TPU to scatter network functions and process. Also, in an embodiment of the present disclosure, the computing device 100 disperses network functions for processing together with other computing devices.

In an embodiment of the present disclosure, the image processed using the network function may be an image stored in a storage medium of the computing device 100, an image captured by the camera module 140 of the computing device 100, and/or an image transmitted from another computing device such as an image database through the communication module 130. Also, in an embodiment of the present disclosure, the image processed by the network function may be an image stored in a computer-readable storage medium (for example, a flash memory or the like may be included, but the present disclosure is not limited thereto). The computing device 100 receives an image file stored on a computer-readable storage medium through an input-output interface (not shown).

In an embodiment of the present disclosure, the computing device 100 receives image data and examines the input data using a previously learned network function. The input data check of the present disclosure may include anomaly detection for determining whether an anomaly exists in the input data. The input data inspection of the present disclosure may include false detection determinations related to anomaly detection and anomaly detection.

The memory 120 may store a computer program for performing an input data checking method of an embodiment of the present disclosure, and the stored computer program is read and driven by the processor 110.

The communication module 130 transmits and receives the image data to and from other computing devices, servers, and the like to execute the abnormality determination method according to an embodiment of the present disclosure. The communication module 130 transmits and receives data for the embodiments of the present disclosure, such as image data, to and from other computing devices, servers, and the like. For example, the communication module 130 receives the learning image data in a learning image database or the like. Also, the communication module 130 may enable communication between the plurality of computing devices, thereby performing learning dispersion of the network function in each of the plurality of computing devices. The communication module 130 enables communication between a plurality of computing devices, thereby decentralizing processing of number classification using network functions.

The camera module 140 captures an image of an inspection object to generate image data in order to execute the method of determining abnormality of image data according to the embodiment of the present disclosure. In an embodiment of the present disclosure, the computing device 100 may include more than one camera module 140.

Fig. 2a is a diagram illustrating a network function 200 according to one embodiment of the present disclosure.

In this specification, a calculation model, a neural network, a network function, and a neural network (neural network) may be synonymous. A neural network may be made up of a collection of interconnected computational units, commonly referred to as "nodes". Such "nodes" may also be referred to as "neurons". A neuron includes at least one node. Nodes (or neurons) that make up a neural network are connected by more than one "connecting unit".

In the neural network, one or more nodes connected by the connection means form a relationship between an input node and an output node. The notion of input nodes and output nodes is relativity, and for a node, any node that has an output node relationship has an input node relationship in the relationship with other nodes, and vice versa. As described above, the input node and output node relationship is generated centering on the connection unit. At one input node, more than one output node is connected via a connection unit, and vice versa.

In a relation between input nodes and output nodes connected by a connection unit, the output nodes determine values based on data input to the input nodes. Wherein a node interconnecting the input node and the output node may have a weight (weight). The weighting values may be varied and the neural network may be varied by a user or an algorithm in order to perform the desired function. For example, in the case where one output node is connected to one or more input nodes by the respective connection units, the output node determines an output node value based on a value input to the input node connected to the output node and a weighted value set at the connection unit corresponding to each input node.

As described above, one or more nodes in the neural network are connected to each other by one or more connection means, and thereby an input node and an output node relationship is formed in the neural network. In the neural network, characteristics of the neural network are set based on the number of nodes and connection units, the connection relationship between the nodes and the connection units, and a weighted value restored to the connection units. For example, there are the same number of nodes and connection units, and in the case where there are two neural networks whose weighted values between connection units are different, the two neural networks are not the same.

A neural network may include more than one node. Some of the nodes that make up the neural network make up a layer (layer) based on the distance from the original input node. For example, starting from the initial input node, a set of nodes at a distance of n may constitute n layers. The distance from the initial input node is determined by the minimum number of connection units that need to be traversed in order to reach the corresponding node from the initial input node. However, this layer definition is used to illustrate that within neural networks, layer differences may be defined by methods other than those described. For example, the level of a node may be defined by the distance of the final node.

The first input node is one or more nodes among nodes in the neural network, and data is directly input to the nodes in relation to other nodes without passing through the connection means. Alternatively, in the neural network, the nodes of the other input nodes connected by the connection means are not blocked in the relationship between the nodes with the connection means as a reference. Similarly, the final output node is one or more nodes that do not have an output node in the relationship with other nodes among the nodes in the neural network. The hidden node is a node constituting a neural network, which is not the first input node and the last output node. In the neural network according to an embodiment of the present disclosure, the number of nodes of the input layer is the same as the number of nodes of the output layer, and in the input layer, the number of nodes decreases and increases again along the hidden nodes. Also, in the neural network of another embodiment of the present disclosure, the number of nodes of the input layer is greater than the number of nodes of the output layer, and in the input layer, the number of nodes decreases along the hidden nodes. In addition, in the neural network according to another embodiment of the present disclosure, the number of nodes in the input layer is greater than the number of nodes in the output layer, and the number of nodes in the input layer increases along the hidden nodes. The neural net of another embodiment of the present disclosure is a neural net of a combined form of the neural nets.

A Deep Neural Network (DNN) is a neural network that includes a plurality of hidden layers in addition to an input layer and an output layer. If a deep neural network is utilized, the potential structure (latent structures) of the data can be grasped. That is, the underlying structure of the photograph, text, video, voice, music (e.g., what object is present in the photograph, what the content and emotion of the text are, what the content and emotion of the voice are, etc.) can be grasped. The deep neural network may include a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), an auto encoder (auto encoder), a Generative Adaptive Network (GAN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Q network, a U network, a siro network, and the like. The description of the deep neural network is an illustration, and the disclosure is not limited thereto.

In an embodiment of the present disclosure, the network function 200 may further include an auto-encoder. The auto-encoder may be one of artificial neural networks for outputting output data similar to the input data. The auto-encoder may comprise at least one hidden layer, a single hidden layer being configurable between the input and output layers. The number of nodes of each layer is reduced to an intermediate layer called a bottleneck layer (coding) among the number of nodes of the input layer, and thereafter, is expanded in the bottleneck layer symmetrically to the reduction to the output layer (symmetrical to the input layer). In this case, the dimension reduction layer and the dimension restoration layer are shown to be symmetrical in the illustration of fig. 2, but the present disclosure is not limited thereto, and the nodes of the dimension reduction layer and the dimension restoration layer may be symmetrical or asymmetrical. The auto-encoder may perform non-linear dimension reduction. The number of input layers and output layers corresponds to the number of sensors remaining after the preprocessor of the input data. In an auto-encoder architecture, the number of nodes of the hidden layer in the encoder decreases in the direction away from the input layer. In the case where the number of nodes of the bottleneck layer (layer having the minimum nodes located between the encoder and the decoder) is too small, a sufficient amount of information cannot be transferred, and thus, a certain number or more (e.g., half or more of the input layer, etc.) can be maintained.

The neural network may learn by at least one of supervised learning (unsupervised learning), unsupervised learning (unsupervised learning), and semi-supervised learning (semi-supervised learning). The learning of the neural network is used to minimize the error of the output. In the learning process of the neural network, learning data is repeatedly input to the neural network, an error between an output of the neural network and a target related to the learning data is calculated, and the error of the neural network is reversely propagated from an output layer to an input layer of the neural network in a direction for reducing the error, thereby updating a weighted value of each node of the neural network. In the case of supervised learning, learning data in which the respective learning data are labeled with correct answers (i.e., labeled with learning data) is used, and in the case of unsupervised learning, the respective learning data are not labeled with correct answers. That is, for example, learning data in the case of supervised learning related to data classification is data in which a class is labeled to the learning data. The labeled learning data is input to the neural network, and the output (class) of the neural network is compared with the label of the learning data to calculate an error (error). As another example, an error is calculated by comparing input learning data with a neural network output in the case of unsupervised learning related to data classification. The calculated error is propagated in reverse in the neural network in the reverse direction (i.e., in the direction of the output layer toward the input layer), and the connection weight values of the respective nodes of the respective layers of the neural network are updated as the error is propagated in reverse. The updated connection weight value of each node may vary according to a learning rate (learning rate). The computation of the neural network and the back propagation of errors with respect to the input data may constitute a learning loop (epoch). The learning rate may be variously applied according to the number of iterations of the learning cycle of the neural network. For example, a high learning rate is applied in the initial stage of learning of the neural network, the neural network quickly secures a predetermined level of performance to improve efficiency, and a low learning rate is used in the later stage of learning to improve accuracy.

In the learning of the neural network, generally, the learning data is a partial set of actual data (i.e., data processed by the learned neural network), and therefore, there is a learning cycle in which errors relating to the learning data are reduced or errors are added to the actual data. As described above, overfitting over learns, resulting in a phenomenon in which errors relating to actual data increase. For example, one of the overfitting is that after a yellow cat is seen, the neural network that learns about cats cannot identify cats other than the yellow cat. Overfitting increases the error of the machine running algorithm. To prevent such an overfitting algorithm, a variety of optimization methods may be used. Learning data is added to prevent overfitting, normalization (normalization), and a method of omitting a loss (dropout) of a part of a node of a network in the learning process.

The Convolutional Neural Network (CNN) shown in fig. 2b is a kind of deep neural network, and includes a neural network having convolutional layers. A convolutional neural network is one of the multi-layer receptors (multi layer receptors) designed in a way that minimal preprocessing is used. The convolutional neural network may be composed of one or more convolutional layers and an artificial neural network layer combined therewith, and a weighted value sum convergence layer (posing layer) is additionally used. With this structure, the convolutional neural network can fully use the input data of the two-dimensional structure. Convolutional neural networks are used to identify objects in an image. The convolutional neural network may process the image data as rows and columns with secondary elements. For example, in the case of image data encoded in red-green-blue (RGB), two-dimensional (e.g., in the case of two-dimensional images) lines are respectively divided by red, green, and blue colors. That is, the color of each pixel of the image data may be divided into components of rows and columns, and the size of the rows and columns is the same as the size of the image. Thus, the image data may be divided into 3 two-dimensional rows and columns (three-dimensional data array).

In a convolutional neural network, a convolution filter is moved to multiply the convolution filter with the row and column components of each position of an image to perform a convolution process (input and output of a convolution array). The convolution filters may be in rows and columns of n x n configuration and, in general, may be comprised of fixed configuration filters that are smaller than the number of whole pixels of the image. That is, when an m × m image is input to a convolution layer (for example, a convolution layer having a convolution filter size of n × n), a row and a column of n × n pixels representing each pixel of the image are multiplied by a convolution filter component (that is, each component of the row and the column is multiplied). By multiplication with a convolution filter, components matching the convolution filter can be extracted in the image. For example, if the 3 × 3 convolution filters for extracting the upper and lower straight line components in the image are configured as [ [0, 1, 0], [0, 1, 0] ] and the convolution filters are applied to the input image, the upper and lower straight line components matching the convolution filters are extracted from the image and output. The convolutional layer may use convolutional filters for each row and column associated with each channel (i.e., R, G, B colors in the case of R, G, B painted images). Convolution layers use convolution filters on the input image to extract features in the input image that match the convolution filters. The filter values of the convolution filter (i.e., the values of the individual components of the row and column) are updated by inverse propagation during the learning process of the convolutional neural network. The output of the convolutional layer is connected to the sub-sampling layer, thereby simplifying the output of the convolutional layer to reduce the amount of memory used and the calculated value. For example, when the output of the convolutional layer is input to the collection layer having a 2 × 2 maximum collection filter, the maximum value included in each patch is output every 2 × 2 in each pixel of the image, and the image is compressed. The assembly outputs a minimum value at the patch, or outputs an average value of the patch, and any assembly method may be included in the present disclosure.

The convolutional neural network may include more than one convolutional layer, sub-sampling layer. The convolutional neural network iteratively performs a convolution process and a sub-sampling process (e.g., the maximum assemble, etc.) to extract features in the image. Through an iterative convolution process and sub-sampling process, the neural network can extract global features of the image.

The output of the convolutional layer or the sub-sampling layer may be input to a fully connected layer (fully connected layer). A fully connected layer is a layer connected to all neurons of a layer adjacent to all neurons of one layer. The fully connected layer is a structure in which all nodes of each layer are connected to all nodes of other layers in the neural network.

In an embodiment of the present disclosure, to perform segmentation of image data, the neural network may include a deconvolution neural network (DCNN). The deconvolution neural network performs a similar operation of calculating the convolution neural network in the reverse direction, and outputs the features extracted in the convolution neural network as features related to the original data.

In an embodiment of the present disclosure, the processor 110 may derive a result of a first check on the input data. The first-time examination result may contain abnormality information about the input data, and the abnormality information may contain arbitrary information about an abnormality existing in the input data. For example, the first-time examination result may include the presence or absence of an exception, the location of an exception in the input data, the type of exception. In the case where the input data is an image, the first-time inspection result may include information on whether there is an abnormality in the input data, which pixel of the input data has an abnormality, which type of abnormality the abnormality contained in the input data is (for example, in the case of an image related to a semiconductor, a soldering failure, a wire failure, a component failure, and the like), and the like.

The processor 110 may derive a first check result regarding whether an exception exists in the input data. The processor 110 processes the input data using a pre-learned inspection model including one or more network functions, thereby performing an anomaly inspection with respect to the input data to derive a first inspection result. The inspection model is a semi-supervised learning model that determines whether or not an unlearned new pattern (i.e., abnormality) exists in the input data by learning only with normal data, and thereby can determine whether or not an abnormality exists in the input data. The inspection model is a supervised learning model that can determine whether or not an input data contains an abnormality by learning the marked normal data and abnormal data. Examining the model may include any mechanical learning model that determines whether an anomaly exists in the input data.

The processor 110 performs abnormality detection on the input data based on the comparison between the input data and the reference data to derive a first check result. In the case where the input data is an image, the processor 110 determines whether there is an abnormality in the input data based on a comparison of the input data with reference data containing an abnormality or reference data not containing an abnormality. The processor 110 determines an anomaly in the input data based on any machine vision (machine vision) solution.

The processor 110 classifies the input data using a classification model learned using learning data labeled with cluster data. The classification model is learned using a learning data set including a plurality of learning data subsets, and of the learning data included in the learning data set, similar data is classified into the same cluster, and non-similar data is classified into different clusters. The learning data subset may include a plurality of learning data. The learning data subset may comprise a plurality of learning data, each of which is labeled with cluster information. In the present disclosure, the learning data is data used by the subset in 1 iteration (iteration) in the learning of the network function. In the present disclosure, the learning data set is a data entirety used for a loop in learning of a network function. The learning data subsets may contain non-similar learning data, and the non-similar learning data contained in the learning data subsets are classified into different clusters by the classification model.

That is, the learning data subset includes a plurality of learning data each labeled with cluster information, and the learning data subset may further include learning data labeled with different cluster information. Specifically, the learning data subset may include target data, target-like data, target-non-like data, which may be labeled with cluster information such as the target data and the target-like data, and the target-non-like data is labeled with cluster information different from the target data and the target-like data. The target data and the target-like data included in the subset of learning data may be labeled with first cluster information, and the target-like data may be labeled with second cluster information.

For example, when the target data is data including an abnormality, the target similar data is data including an abnormality, and the target non-similar data is data not including an abnormality. The target non-similar data is data in which at least a part of a portion other than the abnormality in the target data is duplicated. For example, when the target data is an image, the target non-similar data is an image of at least a part of the image including the abnormality except for the abnormal portion. Also, for example, the target-like data is data that contains an anomaly of a type similar to that contained in the target data. For example, when the target data is an image including an abnormality, the target-like data is an image cropped (cropped) so as to include at least a part of the abnormality included in the target data. For example, the abnormality of the abnormality-like type included in the target data may include an abnormality at the same position on the inspection target object as the abnormality included in the target data, and an abnormality of the same type as the type of a defect of the inspection target object as the abnormality included in the target data (for example, when the inspection target object is a circuit board and the abnormality included in the target data is a soldering defect, the abnormality included in the target-like data is also a soldering defect).

The target-like data may contain at least a part of the image of the target data, and a pixel portion corresponding to an abnormality in the image of the target data is data that must be contained.

The target non-similar data is an image containing no abnormality. The target data is an image including an abnormality in the center of the image. The target non-similar data is an image in which at least a part of a portion other than the abnormality of the target data is repeated.

In the case where the data is an image, the abnormal portion of the image containing an abnormality and the abnormal portion of the image not containing an abnormality are different, and the other portions may be similar, so that in the case of classification by a general method, it is possible to classify into similar images. For example, in an image of 100 × 100 pixels acquired in a semiconductor process, when an abnormal portion is 5 × 5 pixels, and when the abnormal portion is normally classified by 0.25% of the entire image, 99.75% of the two images are similar, and therefore, the images are classified as similar by a normal classification method. However, in the case where the classification model is learned using the target data, the target similar data, and the target dissimilar data each of which is labeled with the cluster information according to the embodiment of the present disclosure, even when a part of the images is different, the images are classified as dissimilar data.

The processor 110 classifies the target data and the target-like data into the same cluster and classifies the target-non-like data into a cluster different from the cluster to which the target data and the target-like data belong in the classification model.

In the present disclosure, the target data may include at least one of an abnormality related to the inspection target object included in the image and an abnormality separated from the inspection target object in the case of the target data bit image. For example, in the case where the target data is an image acquired from a semiconductor process, the target data may include at least one of an abnormality (e.g., a defect or the like) that may occur in a semiconductor product and an abnormality that is not related to the inspection target object (e.g., a problem with the image acquisition apparatus, a foreign object in a lens, or the like). That is, the target data and the target-like data may include false positive (false positive) data from the viewpoint of abnormality detection. More specifically, in the case where there is no abnormality in the inspection target object and there is a situation that can occur separately from the inspection target object, the target data and the target-like data may include such false alarm. Thus, the classification model classifies anomaly types and false positives as clusters.

In the case where the input data contains an anomaly unrelated to the subject object, the first inspection result may contain a false positive. In this case, the input data is classified by the classification model for the second time, and the processor 110 classifies the result of the input data by the classification model, and by the classification model learned by the learning data including false alarms, it is possible to determine whether the input data is an abnormality related to the actual inspection target object or an abnormality unrelated to the inspection target object (i.e., false alarm) in the second classification of the input data.

Hereinafter, the learning data of the present disclosure will be described in detail with reference to the drawings.

Fig. 3a, 3b, 3c, 3d, 3e, and 3f are diagrams illustrating learning data of an embodiment of the present disclosure. The images shown in fig. 3a to 3e are merely examples of the learning data, and the present disclosure may include arbitrary learning data and arbitrary images.

Fig. 3a is a diagram illustrating a target base image 300 that may be used as learning data. The target base image 300 itself may be used as a target image for learning of the classification model, and the target base image 300 is clipped for the target image 310. For example, the target base image 300 may include a poor 301 anomaly of the object of the inspection object. The plurality of target

similar images

311, 313 may be images cut out so as to include the defective 301 portion of the target image 310. The plurality of trimming

lines

310, 311, 313 in fig. 3a are trimming of the target image 310 and trimming 311, 313 of the target similar image.

Fig. 3b shows a target image 310(target image) cut out by the trimming line of fig. 3a, and target similar images 311 and 313(target similar image). Referring to fig. 3b, the target image 310 and the target-

like images

311 and 313 each include an anomaly 301, which may include other portions of the object to be inspected.

Fig. 3c is an illustration showing a target non-similar base image 320. The target non-similar base image 320 shown in fig. 3c may itself be used as a target non-similar image (target non-similar image), with a portion cropped and used as the target non-similar image. In the case of using the cropped target non-similar image for learning the classification model of the present disclosure, the target non-similar image 320 may have a resolution as the target image 310, the target similar image 311. The resolutions of the target image, the target similar image, and the target non-similar image are also exemplified, and the resolutions of these may be the same or different. The target non-similar image 320 may be an image that does not contain anomalies. The target non-similar image 320 is an image that does not contain an abnormality in a portion 321 corresponding to the target image.

Fig. 3d is an illustration showing a learning data subset for learning a classification model of an embodiment of the present disclosure.

The learning data subset may include a target image 310 with an anomaly 301, a target similar image 311, and a target non-similar image 320 without an anomaly 321. The target image 310 and the target similar image 311 are labeled with the same cluster information in such a manner as to be classified into the same cluster. The target non-similar image 320 is marked with cluster information different from the cluster information marked on the target image 310 and the target similar image 311. For example, the target image 310 and the target similar image 311 may be marked with first cluster information, and the target non-similar image 320 may be marked with second cluster information.

In the case of learning the classification model using the subset of learning data shown in fig. 3d, the classification model may classify images with and without abnormalities into different clusters.

For example, in an image of 100 × 100 pixels acquired in a semiconductor process, when an abnormal portion is 5 × 5 pixels, and when the abnormal portion is normally classified by 0.25% of the entire image, 99.75% of the two images are similar, and therefore, the images are classified as similar by a normal classification method. However, in the case where the classification model is learned using the target data, the target similar data, and the target dissimilar data each of which is labeled with the cluster information according to the embodiment of the present disclosure, even when a part of the images is different, the images are classified as dissimilar data.

Fig. 3e is an illustration showing other learning data subsets for learning the classification model of an embodiment of the present disclosure.

The target image 330, target-like image 333 of FIG. 3e may be an image containing false positives. More specifically, an abnormality 331 included in the target image 330 and the target-like image 333 is an abnormal situation (e.g., a lens foreign object, etc.) that may occur separately from the inspection target object. The target image 330 may be an image related to a normal product or an image of an abnormality occurring according to an abnormal condition separated from the normal product. According to the conventional abnormality detection method, when such an erroneous abnormal image is included, it is determined that the abnormal image is abnormal, and an error result is output. However, in one embodiment of the present disclosure, by using an image including false alarms and generating clusters related to the false alarms by a classification model, it is possible to detect whether or not an inspection target object included in an input image as an inspection target actually includes an abnormality, and the inspection target object does not include an abnormality. In the example of fig. 3e, the target non-similar image 340 is an image that does not contain an anomaly.

In the case where the classification model is learned using the learning data subset shown in fig. 3e, the classification model may classify the image with the false alarm and the image without the false alarm into different clusters.

Fig. 3f is an illustration showing another learning data subset for learning a classification model of an embodiment of the present disclosure.

The target image 330, target-like image 333 of FIG. 3f may be an image containing false positives. More specifically, the abnormality 331 included in the target image 330 and the target-like image 333 is an abnormal situation (e.g., a lens foreign object, etc.) that may occur separately from the inspection target object. The target image 330 may be an image related to a normal product or an image of an abnormality occurring according to an abnormal condition separated from the normal product. According to the conventional abnormality detection method, when such an erroneous abnormal image is included, it is determined that the abnormal image is abnormal, and an error result is output. However, in one embodiment of the present disclosure, by using an image including false alarms and generating clusters related to the false alarms by a classification model, it is possible to detect whether or not an inspection target object included in an input image as an inspection target actually includes an abnormality, and the inspection target object does not include an abnormality. In the example of fig. 3e, the target non-similar image 310 is an image including an anomaly 301 (i.e., an anomaly related to the inspection target object), and does not include a false positive. In the case where the classification model is learned using the learning data subset shown in fig. 3f, the classification model may classify the image in which the false alarm exists and the image in which the abnormality exists in the actual inspection target object (i.e., the image classified as abnormal in the abnormality inspection) into different clusters.

The target image 330 including the false alarm is added according to the type of the false alarm. In other words, in the industrial field, when a new false positive occurs while an abnormality detection solution is being executed, the classification model is additionally learned using the new false positive as a target image including false positives. The classification model of the present disclosure may cluster well-known anomaly types and well-known false positive types, and may determine which type the input data belongs to.

The classification model of the present disclosure clusters similar data over the solution space 400. More specifically, the classification model includes the target data 401 and the target-like data 402 in one cluster 410, and the target non-like data 403 in a different cluster than the target data 401 and the target-like data 402.

Each cluster has a prescribed distance margin 420 in the solution space of the learned classification model.

The classification model receives a learning data subset including target data 401, target similar data 402, and target non-similar data 403 to match each data with a solution space, and updates weighting values of one or more network functions including the classification model in a clustering manner according to clustering information labeled on the solution space. That is, the classification model increases the distance in the solution space between the target data 401 and the target similar data 402 and the target non-similar data 403 so that the distance in the solution space between the target data 401 and the target similar data 402 becomes shorter. For example, the classification model learns using a triple-based cost function. The input data is separated from third input data of other classes based on a triple cost function, the difference between a first distance between the input data as the same class (i.e. the size of the cluster 410) and a second distance between one of the input data as the same class and the third input data (i.e. the distance between 401 or 402 and 403) being at least a distance margin 420, the method of learning the classification model comprising the step of reducing the first distance below a prescribed proportion of the distance margin. Where the distance margin 420 is always a positive number. To reach the distance margin 420, the weighting values of more than one network function included in the classification model may be updated, with the updating of the weighting values being performed at each iteration (iteration) or 1 epoch. In connection with such distance margins, the details are incorporated herein by reference as "FaceNet" by Schroff et al: a UnifiedEmbedding for Face Recognition and Clustering "and Korean laid-open patent No. 10-2018-0068292.

The classification model is learned by a magnetic loss (magnet loss) model that considers not only cluster classification of non-similar data but also semantic (semantic) relationships between individual data among one cluster or other clusters. On the solution space of the classification model, the initial distance of the center point difference of each cluster is executed in the learning process. After the classification model spatially matches the data, the spatial location of each data is adjusted based on similarities to the cluster to which each data belongs and to data within and outside the cluster. Details regarding learning of a classification model based on magnet losses are collectively referred to in this specification as "METRIC LEARNING WITHADAPTIVE DENSITY DISCRIMINATION" by o.rippel et al.

Fig. 5 is a diagram illustrating a solution space 400 of a classification model. The solution space 400 shown in FIG. 5 is merely exemplary, and a classification model may contain any number of clusters and any number of data for each cluster. The

data

431, 433, 441, 443 in the cluster shown in fig. 5 are similar data.

In the present disclosure, the solution space is composed of one or more dimensions and includes one or more clusters, each cluster being composed based on a feature based on each target data and on a position of the feature on the solution space based on target-like data.

In the solution space, the first cluster 430 and the second cluster 440 may be clusters related to non-similar data. Also, the third cluster 450 may be a cluster related to data that is not similar to the first and second clusters. The

distance

445, 435 between clusters is a measure that presents the difference in data belonging to each cluster.

A twelfth distance 445 between the first cluster 430 and the second cluster 440 is a measure of the difference in data belonging to the first cluster 430 and the second cluster 440. And, a thirteenth distance 435 between the first cluster 430 and the second cluster 440 is a measure of the difference in data belonging to the first cluster 430 and the third cluster 450. In the illustration shown in fig. 5, data belonging to the first cluster 430 is less similar to data belonging to the second cluster than to data belonging to the third cluster 450. That is, when the distance between clusters is long, the data belonging to each cluster is more dissimilar, and when the distance between clusters is short, the data belonging to each cluster is more similar. The

distances

435, 445 between the zones are greater than the radius of the cluster by more than a predetermined proportion. The processor 110 computes the input data using a classification model, thereby classifying the input data based on the locations of features that match the input data in a solution space of the classification model.

The processor 110 processes the input data using the pre-learned classification model, thereby matching features of the input data to a solution space of the pre-learned classification model. The processor 110 classifies the input data based on a position on the solution space of the input data, to which one of more than one cluster on the solution space the input data belongs.

The processor 110 determines a final inspection result based on at least a portion of the classification result related to the input data of the classification model.

In the case where the splitting model classifies the input data as belonging to a particular cluster, the processor 110 generates the inspection result matching the particular cluster as a second inspection result to determine a final inspection result. And, in case that the classification of the classification model and the input data fails, the processor 110 determines the first inspection result as a final inspection result, wherein the second inspection result may include information on at least one of the presence or absence of an abnormality, the location of the abnormality, the type of the abnormality, the presence or absence of false detection, and the type of false detection.

The pre-learned classification model retains clusters in the solution space by type of anomaly, and thus, the processor 110 utilizes the classification model to generate a second inspection result related to the input data. The processor 110 determines whether there is an abnormality in the input data based on which cluster on the solution space of the classification model the features of the input data belong to, and in the case where there is an abnormality, may determine whether the abnormality is the presence of an abnormality on the inspection target object or an abnormality unrelated to the inspection target object (i.e., false detection). For example, in a case where the position of the feature of the input data on the solution space belongs to a cluster related to the substrate soldering failure (i.e., in a case where the input data is classified as the substrate soldering failure using the classification model), the processor 110 may generate a second inspection result in which there is an abnormality as the substrate soldering failure in the input data, and determine a final inspection result based on the first inspection result and the second inspection result. For example, when the position of the feature of the input data in the solution space belongs to a cluster related to a foreign object in the subject object photographing lens, the processor 110 may generate a second inspection result corresponding to the input data as a false detection and normal data having no abnormality, and may determine a final inspection result based on the first inspection and the second inspection result.

In the exemplary input data, when it is determined that the substrate soldering failure is caused in the first inspection, the first and second inspection results are the same, and therefore, the processor 110 determines the substrate soldering failure as the final result of the abnormality detection related to the input data. In the illustrated example, the processor 110 may inform the operator of the input data as classified as other anomalies in the first inspection (i.e., where the first and second classification results differ). The processor 110 marks the corresponding input data in order to inform the operator of the difference between the first inspection result and the second inspection result. When the first classification result and the second classification result are not the same, the processor 110 combines the first inspection result and the second inspection result to generate a final inspection result. For example, the ensemble method may use a voting method, a method based on a weighted value of each neural net, and may include any other ensemble method.

In the case where the position of the feature of the input data in the solution space does not belong to any cluster, the processor 110 determines that the classification using the classification model of the input data has failed, and generates the first inspection result as the final inspection result. For example, in the case where the input data includes a new abnormality of the inspection target object with a defective circuit wiring or an abnormality of the inspection target object with broken lens, the processor 110 (that is, in the case where the learned classification model has a new pattern that is not learned, or the like) cannot classify the input data using the classification model, and therefore, the final inspection result regarding the corresponding input data is generated based on the first inspection result. In this case, the processor 110 may inform the operator of the content. The processor 110 needs to tag the corresponding input data in order to inform the operator of the content. Also, in the case where the corresponding input data that cannot be classified by the classification model includes an abnormality, the processor 110 may generate the corresponding input data as new learning data. In other words, in the case where data having an abnormality that cannot be classified by the classification model (i.e., the case where the first inspection is abnormal, etc.) is an image, the processor 110 may generate a new learning data subset having the other portion including the abnormality of the corresponding input data as target similar data, with the corresponding input data as a target image. The processor 110 additionally learns the classification model by generating clusters over the solution space of the classification model associated with the new anomaly using the newly generated subset of learning data.

That is, in one embodiment of the present disclosure, in the case where an abnormality of input data is determined by the first and second inspections, and the input data includes an abnormality (i.e., a false alarm) related to an abnormal situation occurring by a factor other than the inspection target object, the inspection result may be derived instead of the false detection. When a new false detection type or an abnormality type occurs, the classification model is learned again using the new false detection type or the abnormality type, whereby predetermined inspection performance corresponding to the newly occurring false detection and abnormality can be maintained.

Further, by separating the first check and the second check, addition of a new type of abnormality or a new type of false detection to the abnormality detection solution does not affect the classification performance with respect to the past data. That is, in the conventional solution, when repeated false detections occur, the solution itself is modified to solve the problem of handling the false detections, whereby other false operations may occur, and the possibility of additional false operations may be cut off in the abnormality detection of one embodiment of the present disclosure.

The computing device 100 may derive a first check result on the input data, step 610. In the present disclosure, the input data may include image data, and the computing apparatus 100 may determine a first check result related to the photographed image data or the image data transmitted from the external device. The first examination result may contain abnormality information about the input data.

The computing device 100 classifies the input data by the classification model learned using the learning data labeled with the cluster information (step S630). The classification model is learned using a learning data set including a plurality of learning data subsets, and among the learning data included in the learning data set, similar data is classified into the same cluster, and non-similar data is classified into other clusters.

The computing device 100 outputs a final inspection result based on at least a portion of the classification results for the input data of the classification model (step 650). In the case where the classification model classifies the input data as belonging to a particular cluster, the computing device 100 generates the inspection results that match the particular cluster as second inspection results, determining a final inspection result based on at least a portion of the second inspection results. When the classification of the classification model with respect to the input data fails, the computing apparatus 100 determines the first examination result as the final examination result.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by the following units.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by: a unit 710 for deriving a result of the first check on the input data; a unit 730 for classifying the input data by a classification model learned using the learning data labeled with the cluster information; and a unit 750 for outputting a final inspection result based on at least a portion of the classification result of the input data to the classification model.

In further embodiments of the method for inspecting input data of the present disclosure, the means 710 for deriving a result of a first inspection of the input data may comprise means for processing the input data using a pre-learned inspection model comprising one or more network functions, thereby performing anomaly detection on the input data, or performing anomaly inspection with respect to the input data based on a comparison of the input data and reference data.

In another embodiment of the method for inspecting input data of the present disclosure, the unit 750 for outputting a final inspection result based on at least a portion of the classification result of the input data of the classification model may include a unit for generating an inspection result matching a specific cluster as a second inspection result, outputting a final inspection result based on the second inspection result, if the classification model classifies the input data as belonging to the specific cluster, and outputting the first inspection result as the final inspection result, if the classification of the input data fails.

In another embodiment of the present disclosure for amplifying the inspection of the input data, the unit 730 for classifying the input data by using the classification model learned by the learning data labeled with the cluster information may include: means for processing the input data using the pre-learned classification model, thereby mapping features of the input data to a solution space of the pre-learned classification model; and a unit that classifies the input data based on whether the input data belongs to one of one or more clusters on the solution space based on a position of the input data on the solution space.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by the following modules.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by: a module 810 for deriving a result of a first check on the input data; a module 830 for classifying the input data by a classification model learned using the learning data labeled with cluster information; and a module 850 for outputting a final inspection result based on at least a portion of the classification result of the input data to the classification model.

According to an embodiment of the present disclosure, a method for examining input data may be embodied by the following logic.

According to an embodiment of the disclosure, a method for examining input data may be embodied by the following logic: logic to derive a result of a first check on the input data 910; logic 930 to classify the input data by a classification model learned using learning data labeled with cluster information; and logic 950 to output a final inspection result based on at least a portion of the classification result for the input data of the classification model.

According to an embodiment of the present disclosure, a method for checking input data may be embodied by: a circuit 1010 for deriving a result of a first check on the input data; a circuit 1030 that classifies the input data by a classification model learned using learning data labeled with cluster information; and a circuit 1050 that outputs a final inspection result based on at least a portion of the classification result for the input data of the classification model.

Those of ordinary skill in the art would appreciate that the various illustrative logical blocks, structures, modules, circuits, units, logic, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, structures, units, logic, modules, circuits, and components have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the system. Skilled artisans in the present disclosure may implement the described functionality in varying ways for each particular application. But such implementation decisions should not be made beyond the scope of this interpretation.

The present disclosure is described in connection with computer-executable instructions that are generally executed on one or more computers, which may be embodied by a combination of modules and/or hardware and software combined by one of ordinary skill in the art to which the present disclosure pertains.

Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or embody particular abstract data types. Moreover, those skilled in the art to which the present disclosure pertains will appreciate that the methodologies of the present disclosure may be practiced without limitation to single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as other computer system configurations, including personal computers, hand-held computing devices, microprocessor-based or programmable appliances, and the like, which may be operatively coupled to one or more associated devices.

The illustrated embodiments of the present disclosure may be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

A computer typically includes a variety of computer-readable media. Accessible media may become computer-readable media through a computer, and such computer-readable media include volatile and non-volatile media, transitory (transactional) and non-transitory (non-transactional) media, and removable and non-removable media. By way of non-limiting example, computer-readable media may comprise computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Video Disk (DVD) or other optical disk storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be accessed by a computer and which can store the desired information.

Computer readable transmission media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and conveys all information. Modulated data signal means that one or more signals are set or changed in characteristics within the signal in such a manner as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media include wired media such as a wired network or direct-wired connection, acoustic, RF, infrared, and wireless media such as other wireless media. Combinations of any of the above are also included within the scope of computer readable transmission media.

An illustrative environment 1100 embodying the various aspects of the disclosure includes a computer 1102, the computer 1102 including a processing device 1104, a system memory 1106 and a system bus 1108. The system bus 1108 couples system components including, but not limited to, the system memory 1106 to the processing device 1104. The processing device 1104 can be any of various commonly used processors. Dual processors and other multi-processor architectures may also be employed as the processing device 1104.

The system bus 1108 can be one of several types of bus structure that may further interconnect to another memory bus, a peripheral bus, and any of a variety of commonly used bus architectures. The system memory 1106 includes read only memory 2110 and random access memory 2112. The basic input/output system (BIOS), stored in a non-volatile memory 2110 such as RROM, EPROM, EEPROM, contains the basic routines that the BIOS will transfer information between elements within the computer 1102, when it is started. The random access memory 2112 may include a high-speed random access memory such as a static random access memory for caching data.

The computer 1102 includes an internal hard disk drive 2114(HDD) (e.g., EIDE, SATA, the internal hard disk drive 2114 also being external to and within a suitable chassis (not shown)), a magnetic floppy disk drive 2116(FDD) (e.g., to read from a removable diskette 2118 or of the type used for recording thereon), and an optical disk drive 1120 (e.g., reading a CD-RON disk 1122 or to read from or of the type used for recording on other high capacity optical media such as the DVD). The hard disk drive 2114, magnetic disk drive 2116, and optical disk drive 1120 can be connected to the system bus 1108 by a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical drive interface 1128, respectively. The interface 1124 for external drive implementations includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

The drives and their associated computer-readable media provide nonvolatile storage of data, data structures, computer-executable instructions, other data, and so forth. In the case of computer 1102, the drives and media store any data in a suitable digital form. Description of the computer-readable media refers to a hard disk drive, a removable magnetic diskette, and a removable optical media such as a CD or DVD, as well as other types of media which are readable by a computer such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, in the exemplary operating environment, as will be appreciated by those skilled in the art upon review of the present disclosure, and which may contain computer-executable instructions for performing the disclosed methods.

A number of program modules can be stored in the drives and random access memory 2112, including an operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136. All or portions of the operating system, applications, modules, and/or data are also cached in the random access memory 2112. The present disclosure may be embodied in an operating system or a combination of operating systems that are available in a variety of commercial contexts.

A user enters commands and information into the computer 1102 through one or more wired or wireless input devices, e.g., a keyboard 2138 and a finger device such as a mouse 1140. Other input devices (not shown) are a microphone, an IR remote control, a joystick, a game pad, a stylus pen, touch screen, or the like. These and other input devices are connected to the processing device 1104 via an input device interface 1142 that is coupled to the system bus 1108, and other interfaces such as a parallel port, IEEE 1394 serial port, game port, USB port, IR interface, and so forth.

A display 1144 or other type of display device is also connected to the system bus 1108 via an interface, such as a video adapter 1146. The display 1144 is typically attached to a computer, which typically includes speakers, a printer, and other peripheral output devices (not shown).

The computer 1102 operates in a networked environment using logical connections to one or more remote computers, such as a remote computer 1148, which can communicate via wires and/or wirelessly. The remote computer(s) 1148 can be a workstation, a computing device computer, a router, a personal computer, portable computer, microprocessor-based entertainment appliance, peer device or other common network node, and typically includes many or all of the elements described relative to the computer 1102, although, for purposes of brevity, only a storage device 1150 is illustrated. The logical connections depicted include wired, wireless connectivity to a short-range communications network 1152(LAN) and/or larger networks, e.g., a long-range communications network 1154 (WAN). Such a near-field communication and far-field communication network environment is generally used in offices and companies, and simplifies enterprise-wide computer networks (e.g., intranets) that are connected to computer networks all over the world, such as networks.

When used in a short-range communications network environment, the computer 1102 is connected to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156. The adapter 1156 may facilitate wired or wireless communication for short-range communications 1152, including a wireless access point disposed thereon for communicating with the wireless adapter 1156. when used in a long-range communications networking environment, the computer 1102 may include a modem 1158 coupled to a communications computing device on the long-range communications 1154, or other means for establishing communications over the long-range communications 1154, such as via a network, etc. The modem 1158, which can be internal or external and a wired or wireless device, is connected to the system bus 1108 via the serial port interface 1142. In a networked environment, program modules depicted relative to the computer 1102, or portions thereof, are stored in the remote memory/storage device 1150. The network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

The computer 1102 is operable to communicate with any wireless devices or individuals configured and operable via wireless communication, such as printers, scanners, desktop and/or portable computers, Personal Data Assistants (PDAs), communication satellites, any device or location associated with a wirelessly detectable tag, and telephones. This includes at least wi-fi and bluetooth wireless technologies. Therefore, as in the past, the communication may be a predetermined structure or ad hoc communication (ad hoc communication) of at least 2 devices.

Wireless Fidelity (Wi-Fi), even if not wired, can be connected over a network. Wireless fidelity is the wireless technology of such devices as telephones in which computers can transmit and receive data indoors and outdoors, i.e., anywhere within the circle of conversation of a base station. Wireless fidelity networks have reliability and use IEEE 802.11(a, b, g, other) wireless technologies in order to provide high-speed wireless connectivity. Wireless fidelity is used in order to connect computers to networks and wired networks (using IEEE 802.3 or ethernet networks). Wireless fidelity networks operate in the unlicensed 2.4 and 5GHz radio bands, for example, at 11Mbps (802.11a) or 54Mbps (802.11b) data rates, or in products that include dual bands.

Those of ordinary skill in the art to which this disclosure pertains will appreciate that information and signals may be represented by a variety of different technologies and techniques. For example, the instructions may be embodied in data, instructions, information, signals, bits, symbols, and chips that may be referenced throughout the above description, voltages, current electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

Those of ordinary skill in the art will appreciate that the various illustrative logical blocks, modules, processors, units, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hard disks, various forms of program or design code (referred to herein, for convenience, as "software"), or combinations of these. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those of ordinary skill in the art to which the disclosure pertains will appreciate that the functionality illustrated may be implemented in a variety of ways for each particular application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments disclosed herein are embodied in methods, apparatus, or articles of manufacture (articles) using standard morphing and/or functional techniques. The term "article of manufacture" includes a computer program, carrier, or media (media) accessible from any computer-readable device. For example, computer-readable media include, but are not limited to, magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, etc.), optical disks (e.g., CD, DVD, etc.), smart cards, and flash memory devices (e.g., EEPROM, cards, strips, key drives, etc.). Also, various storage media disclosed herein include one or more devices and/or other machine-readable media for storing information.

The particular order or hierarchy of steps in the processes disclosed is an illustrative approximation. The particular order or hierarchy of steps for the processes within the scope of the present disclosure may be rearranged based on a design-limited order. Additional methods the scope of the claimed invention is not limited to the specific order or hierarchy presented, but rather to the order in which the elements of various steps are presented.

The description of the disclosed embodiments is provided to enable any person skilled in the art to which the disclosure pertains to make or use the disclosure simply. Various modifications to such embodiments will be readily apparent to those skilled in the art to which the disclosure pertains, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A computer-readable storage medium storing a computer program, wherein the computer program, when executed in one or more processors of a computing device, performs the following operations for checking input data, the operations comprising:

work to derive a result of a first examination of the input data;

a work of classifying the input data by a classification model learned by using the learning data labeled with the cluster information; and

work of outputting a final inspection result based on at least a part of a classification result of the input data to the classification model.

2. The computer-readable storage medium of claim 1, wherein the classification model is learned using a learning data set comprising a plurality of subsets of learning data to classify similar data into the same cluster and to classify non-similar data into different clusters in learning data contained in the learning data set.

3. The computer-readable storage medium of claim 1, wherein deriving the results of the first inspection of the input data comprises processing the input data using a pre-learned inspection model that includes one or more network functions, whereby the input data is subjected to anomaly detection or is subjected to anomaly detection based on a comparison of the input data to reference data.

4. The computer-readable storage medium of claim 1, wherein the classification model is learned using a learning data set comprising a subset of learning data, the learning data set comprising learning data labeled with different cluster information.

5. The computer-readable storage medium of claim 1, wherein the classification model is learned using a learning data set comprising a subset of learning data, the learning data set comprising target data, target-like data, and target-non-like data.

6. The computer-readable storage medium of claim 5, wherein the target data and the target-like data are data labeled with first cluster information, and wherein the target-like data is data labeled with second cluster information.

7. The computer-readable storage medium according to claim 5, wherein, in a case where the target data is data containing an abnormality, the target-like data is data containing an abnormality of a type similar to the abnormality contained in the target data, and the target non-like data is data containing no abnormality.

8. The computer-readable storage medium according to claim 5, wherein, in a case where the target data is an image containing an abnormality, the target similar data is an image cropped so as to contain at least a part of the abnormality contained in the target data, and the target non-similar data is an image containing no abnormality.

9. The computer-readable storage medium of claim 8, wherein the target data is an image containing the anomaly in a center portion of the image.

10. The computer-readable storage medium of claim 8, wherein the target non-similar data is an image of at least a partial repetition of a portion other than an anomaly of the target data.

11. The computer-readable storage medium of claim 5, wherein the classification model is learned in a manner that classifies the target data and the target-like data into the same cluster, and classifies the target-like data into a cluster that is different from the cluster to which the target data and the target-like data belong.

12. The computer-readable storage medium of claim 5, wherein the target data includes an image relating to an abnormal situation that can occur separately from a subject object, and the target-like data includes an image having at least a portion of the target data.

13. The computer-readable storage medium of claim 5, wherein the target data and the target-like data are data that contain false positives.

14. The computer-readable storage medium according to claim 1, wherein the work of outputting a final inspection result based on at least a part of the classification result of the input data to the classification model includes work of generating an inspection result matching a specific cluster as a second inspection result and outputting a final inspection result based on the second inspection result in a case where the classification model classifies the input data as belonging to the specific cluster, and outputting the first inspection result as a final inspection result in a case where the classification of the input data fails.

15. The computer-readable storage medium of claim 14, wherein the second inspection result comprises information regarding whether there is at least one of an anomaly, a location of an anomaly, a type of anomaly, and a type of false positive.

16. The computer-readable storage medium of claim 1, wherein the act of classifying the input data by a classification model learned using learning data tagged with cluster information comprises:

work of processing the input data with the pre-learned classification model to map features of the input data to a solution space of the pre-learned classification model; and

work to classify the input data based on a location of the input data on the solution space and based on whether the input data belongs to one of more than one cluster on the solution space.

17. The computer-readable storage medium of claim 16, wherein the solution space is formed of more than one dimension and includes more than one cluster, and wherein each cluster is formed based on features based on each target data and on locations of features in the solution space based on similar data of the target.

18. An input data inspection method executed in a computing device including one or more processors, comprising:

a step of deriving a result of a first check on the input data;

a step of classifying the input data by a classification model learned by using learning data labeled with cluster information; and

a step of outputting a final inspection result based on at least a part of a classification result of the input data to the classification model.

19. A computing device, comprising:

one or more processors; and

a memory for storing instructions executable in the processor,

the processor performs the following:

deriving a first examination result of the input data;

classifying the input data by a classification model learned using learning data labeled with cluster information; and

outputting a final inspection result based on at least a portion of the classification results for the input data of the classification model.